Add end 2 end test sidebar behaviours on mobile and desktop. #6877

jorgefilipecosta · 2018-05-21T19:04:35Z

Description

Sometimes I notice regressions on the sidebar behaviors e.g.:#6876. So I'm adding an end to end test that tests simple sidebar behaviors on mobile and desktop.

To execute the test use:

npm run test-e2e test/e2e/specs/sidebar-behaviour.test.js

The tests will fail because currently we have a bug and #6876 should be merged first.

gziolo · 2018-05-22T10:24:08Z

test/e2e/support/utils.js

+	await setMobileViewPort();
+}
+
+export async function setDesktopViewPort() {
 	await page.setViewport( { width: 1000, height: 700 } );


Should we use emulate instead https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pageemulateoptions?

In our case, I think we don't have any difference using emulate. We just test for mobile using the resolution. But besides that, we need after emulating mobile set again the default behavior for the desktop. One option to do that would be using an emulate( DesktopDevice ) but puppeteer only offers mobile devices by default https://github.com/GoogleChrome/puppeteer/blob/master/DeviceDescriptors.js. So we would need to create a desktop mobile device, or change the tests to use a new page in each test case.

Creating a desktop mobile device is simple it is a JSON object so maybe it is a good idea.

gziolo · 2018-05-22T10:25:31Z

test/e2e/specs/sidebar-behaviour.test.js

+		await setMobileViewPort();
+
+		// the page was resized to mobile
+		const width = await page.evaluate(


Should it be verified inside the setMobileViewPort method?

True the code was updated 👍

aduth · 2018-05-31T13:15:36Z

test/e2e/support/utils.js

@@ -81,6 +108,13 @@ export async function getHTMLFromCodeEditor() {
 	return textEditorContent;
 }

+export async function clearLocalStorage() {
+	await page.$eval( 'body', ( body ) => {


We should be able to use page.evaluate here, which is run as within the context of the browser:

await page.evaluate( () => window.localStorage.clear() );

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pageevaluatepagefunction-args

Hi @aduth, thank you I applied this version and it works great.
Before I tried it and I had problems but I think the problem was related to other error that was meanwhile solved because now it works exactly as expected.

aduth · 2018-06-06T19:57:48Z

test/e2e/support/utils.js

-		// eslint-disable-next-line no-undef
-		fail( error );
-	} );
+export async function newMobileBrowserPage() {


Where is this function used?

It was not being used I added it to be consistent we had setMobileViewport, setDesktopViewport, and newDesktopBrowserPage so it made sense to create newMobileBrowserPage. I removed it for now if it is required later it can be easily added.

aduth · 2018-06-06T19:58:32Z

test/e2e/specs/sidebar-behaviour.test.js

+				return window.innerWidth;
+			}
+		);
+		expect( width ).toBeLessThan( 782 );


Aren't we expecting this to be guaranteed by setMobileViewPort ?

Yes it should be guaranteed, I removed this check.

After the removal of this check the test sometimes started to fail. It turns out it was related with puppeteer issue puppeteer/puppeteer#1751.

I added a waitForFunction to address it.

aduth · 2018-06-06T19:59:12Z

test/e2e/support/utils.js

 export async function newDesktopBrowserPage() {
 	global.page = await browser.newPage();
+	await setDesktopViewPort();


Capitalization should be "Viewport", if even just for consistency with Puppeteer:

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetviewportviewport

Capitalization corrected, thank your for the catch.

jorgefilipecosta · 2018-06-22T17:41:06Z

Hi @gziolo and @aduth thank you for your review, this PR is read for a new look.

gziolo

In general, it looks good and tests well. I had 2 minor comments where I feel we could remove some code duplications and increase readability of the code.

gziolo · 2018-06-25T10:05:31Z

test/e2e/support/utils.js

+	await setDesktopViewport();
+}
+
+export async function setDesktopViewport() {


Can we generalize this method as follows:

export async function setViewport( type ) { const allowedDimensions = { desktop: { width: 1000, height: 700 } }; const currentDimmension = allowedDimensions[ type ]; await page.setViewport( currentDimmension ); await waitForPageDimensions( currentDimmension.width, currentDimmension.height ); }

I'm wondering if also should align the names with what we use for the viewport module.

Feedback was addressed. Than you for the review!

gziolo · 2018-06-25T10:08:41Z

test/e2e/specs/sidebar-behaviour.test.js

+	} );
+
+	it( 'Should have the sidebar closed by default on mobile', async () => {
+		await setMobileViewport();


Won't it give false positives because we are setting the desktop viewport first?

We are setting the desktop viewport first as the default viewport but Gutenberg is not open. We are only opening Gutenberg in the newPost call.

gziolo · 2018-06-25T10:10:23Z

test/e2e/specs/sidebar-behaviour.test.js

+
+	it( 'Should reopen sidebar the sidebar when resizing from mobile to desktop if the sidebar was closed automatically', async () => {
+		await newPost();
+		await setMobileViewport();


Here we have a different order await setDesktopViewport(); and the new post is created and then changed to mobile. Maybe we should always set viewport in the test just before creating a new post for consistency?

Yes, I totally agree, in this test cases setting the viewport in the tests make things clear.

gziolo · 2018-06-25T15:39:07Z

Cool, thanks for iterating on this one 🚢

* 'master' of https://github.com/WordPress/gutenberg: (69 commits) fix: Show permalink editor in editor (WordPress#7494) Fix text wrapping in Firefox. (WordPress#7472) Try another approach to fixing the sibling inserter in Firefox (WordPress#7530) fix: Improve "add block" text in NUX onboarding (WordPress#7511) Implement core style of including revisions data on Post response (WordPress#7495) Testing: Add e2e test for PluginPostStatusInfo (WordPress#7284) Add end 2 end test for sidebar behaviours on mobile and desktop. (WordPress#6877) Only save metaboxes when it's not an autosave (WordPress#7502) Fix broken links in documentation (WordPress#7532) Remove post type 'viewable' compatibility shim (WordPress#7496) Fix typo. (WordPress#7528) Blocks: Remove wrapping div from paragraph block (WordPress#7477) fix: change import for InnerBlocks (WordPress#7484) Polish library just a teeeeensy bit (WordPress#7522) feat: Add snapshot update script (WordPress#7514) Display server error message when one exists (WordPress#7434) Fix issues with gallery in IE11. (WordPress#7465) Polish region focus style (WordPress#7459) Fix IE11 formatting toolbar visibility (WordPress#7413) Update plugin version to 3.1. (WordPress#7402) ...

aduth · 2018-10-22T17:39:00Z

test/e2e/specs/sidebar-behaviour.test.js

@@ -0,0 +1,80 @@
+/**


-behaviour is kinda redundant as a suffix, and doesn't add much value. Any test could be called as such, undo-behavior.test.js, writing-flow-behavior.test.js, etc.

Also, typo since it's spelled "behavior", not "behaviour" 🇺🇸

jorgefilipecosta added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label May 21, 2018

jorgefilipecosta self-assigned this May 21, 2018

jorgefilipecosta mentioned this pull request May 21, 2018

Fix: make sidebar closed on mobile when page loads even if it was opened before the reload #6876

Merged

jorgefilipecosta force-pushed the add/end-to-end-tests-sidebar-bahaviours branch from 2d7945f to aa86234 Compare May 21, 2018 21:01

gziolo reviewed May 22, 2018

View reviewed changes

jorgefilipecosta mentioned this pull request May 23, 2018

Popover: Refactor popover to clarify computations #6799

Merged

jorgefilipecosta force-pushed the add/end-to-end-tests-sidebar-bahaviours branch from aa86234 to 1eb5270 Compare May 23, 2018 18:50

aduth reviewed May 31, 2018

View reviewed changes

jorgefilipecosta force-pushed the add/end-to-end-tests-sidebar-bahaviours branch 2 times, most recently from a99920e to 2b399f4 Compare June 4, 2018 10:31

aduth reviewed Jun 6, 2018

View reviewed changes

jorgefilipecosta force-pushed the add/end-to-end-tests-sidebar-bahaviours branch from 2b399f4 to ad220c0 Compare June 22, 2018 16:58

jorgefilipecosta added the [Status] In Progress Tracking issues with work in progress label Jun 22, 2018

jorgefilipecosta force-pushed the add/end-to-end-tests-sidebar-bahaviours branch from ad220c0 to f84e091 Compare June 22, 2018 17:23

jorgefilipecosta removed the [Status] In Progress Tracking issues with work in progress label Jun 22, 2018

gziolo approved these changes Jun 25, 2018

View reviewed changes

Add end 2 end test sidebar behaviours mobile and desktop.

acdba6e

jorgefilipecosta force-pushed the add/end-to-end-tests-sidebar-bahaviours branch from f84e091 to acdba6e Compare June 25, 2018 15:36

jorgefilipecosta merged commit e222ccb into master Jun 25, 2018

jorgefilipecosta deleted the add/end-to-end-tests-sidebar-bahaviours branch June 25, 2018 16:13

jorgefilipecosta added this to the 3.2 milestone Jul 5, 2018

aduth reviewed Oct 22, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add end 2 end test sidebar behaviours on mobile and desktop. #6877

Add end 2 end test sidebar behaviours on mobile and desktop. #6877

jorgefilipecosta commented May 21, 2018 •

edited

Loading

gziolo May 22, 2018

jorgefilipecosta May 23, 2018

jorgefilipecosta May 23, 2018 •

edited

Loading

gziolo May 22, 2018

jorgefilipecosta May 23, 2018

aduth May 31, 2018 •

edited

Loading

jorgefilipecosta Jun 4, 2018

aduth Jun 6, 2018

jorgefilipecosta Jun 22, 2018

aduth Jun 6, 2018

jorgefilipecosta Jun 22, 2018

jorgefilipecosta Jun 22, 2018

aduth Jun 6, 2018

jorgefilipecosta Jun 22, 2018

jorgefilipecosta commented Jun 22, 2018

gziolo left a comment

gziolo Jun 25, 2018

jorgefilipecosta Jun 25, 2018

gziolo Jun 25, 2018

jorgefilipecosta Jun 25, 2018

gziolo Jun 25, 2018 •

edited

Loading

jorgefilipecosta Jun 25, 2018

gziolo commented Jun 25, 2018

aduth Oct 22, 2018

Add end 2 end test sidebar behaviours on mobile and desktop. #6877

Add end 2 end test sidebar behaviours on mobile and desktop. #6877

Conversation

jorgefilipecosta commented May 21, 2018 • edited Loading

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgefilipecosta May 23, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aduth May 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgefilipecosta commented Jun 22, 2018

gziolo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo Jun 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gziolo commented Jun 25, 2018

Choose a reason for hiding this comment

jorgefilipecosta commented May 21, 2018 •

edited

Loading

jorgefilipecosta May 23, 2018 •

edited

Loading

aduth May 31, 2018 •

edited

Loading

gziolo Jun 25, 2018 •

edited

Loading