Alpha usability test two: new content script

[karmatosed]

Background: To catch up on our current state of usability testing, check out this post: https://make.wordpress.org/design/2017/04/27/testing-the-gutenberg-user-test/. We already have the first test here, focusing on comparisons: #649.

Goal: What is it like to create new content using Gutenberg?

Task: Write a usability script using the alpha plugin. You should base the structure off the existing testing script, but it is a new task. You are creating a test that starts with no content and has users adding from zero. The content they should be able to create is:

• full width image
• text
• headlines
• lists
• quotes

As it will be creating new content we won't be using a demo, this would be on a local machine for testers or for experienced users have setup instructions. Remember the goal and focus the task on that.

Format: The script should be made into a google form like the first one, however first lets iterate on the script in this issue, then take to the form. This is not a task yet to run these tests. That will come.

The script should also come with a setup sheet for the testers. Our goal is to have as many people doing this test, as easily as possible.

[anna-harrison]

Love the idea for this test - another good set of data! Small suggestion on this one is to run the test with 2 or 3 people before deploying to make sure that the task is achievable.

In the past, I have found it to be really hard for participants to do a task from a "zero start", for example, I once had a total fail on a test with the task

"Write a story about a holiday you have taken, or would like to take. Include a photo, a video, a map and a short description of where you went, what you did and what you ate".

Seems simple enough, right? When I ran the test, no-one was able to complete the task!! I ended up changing it to "Type this story into the editor..." and providing a printout of what the final story needed to look like. This went better :)

[martinlugton]

Definitely agree about trialling with a small group before rolling it out more widely.

Seems that chunking the task up into its small sub-components would be less intimidating for users, and reduce their mental work-in-progress.

[rileybrook]

This is a first pass/first draft of a new content creation test script that could be iterated on.

Borrowing an idea from @annaephox:

I ended up changing it to "Type this story into the editor..." and providing a printout of what the final story needed to look like. This went better :)

A useful approach could be having the user recreate an example blog post or piece of content. This could help the user focus on the experience of creating new content, instead of brainstorming what the layout of their post should look like. As a first step, @joen suggested I post the script as a Github comment, so contributors can help morph it into something workable. There are only a small number of tasks included, but as many can be added to create an optimal test length. Please find full script below ↓

=========================================================================

Tester Set-up

• Open the interview form and resize to 1/3 of the width of the screen.
• Open the Gutenberg Prototype and resize to 2/3 the width of the screen.

User Introduction:
Today we’ll be testing the user experience of creating new content with the Gutenberg editor alpha release. As you work through the tasks, please think aloud. This means to please give a running commentary on what you’re doing as you work through the tasks. Describe what you’re trying to do and how you think you can do it. I’ll ask a lot of questions, but I'm not testing you. We are testing the editor. If you get stuck or confused, it’s not your fault. It helps us identify the problems in the editor that we need to fix. There are no right or wrong answers.

Instead of asking you to start creating content from zero - we will have you re-create a blog post about Einstein, or any other historical figure of your choosing. In the attached screenshot, we have included what a sample piece of content created in the editor might look like. Any images or attributed information of the person should be easily found from a google search using their name.

This sample blog post is based on Einstein. It includes a headline, an image, and an unordered list (birth date, claim to fame, date of passing.) Following this list is a famous quote attributed to them.

Objective:
Using the Gutenberg editor, please re-create the sample blog post focusing on Einstein or another historical figure of your choosing.

Tasks:

1. You want to add a new page headline at the top of page titled “The Life of Einstein”
2. Following the heading, you want to add an image block and upload a picture of Einstein.
3. Following the image, you want to add an unordered list with three bullet points.
4. You want to title the first bullet point “Date of Birth - March 14th, 1879.”
5. You want to title the second bullet point “Known for - Developing the scientific theory of relativity.”
6. You want to title the third bullet point “Date of Passing - April 19th, 1955.”
7. Following the list, you want to add a quotation titled “The true sign of intelligence is not knowledge but imagination. - Albert Einstein”
8. You want to change how the quote is presented. Change it to the alternate quote style.

[karmatosed]

@rileybrook thanks for this, great start. Here are my thoughts reading through. I am really interested in others' too.

• We don't have to say its alpha, so lets not. It could set expectations.
• How do we deal with those that don't want recording? We should allow that.
• What about some intro questions to find out how much they use WP.. the editor?
• User introduction is a lot, lets boil down to what we really need to say.
• Lets fix it on Einstein and not have them worry about other content. If they are actually creating a version, stick to that.
• Can we provide images to say e googling?
• Can we have a page somewhere they just copy?
• When you give task questions, its a good idea to mix in other ones such as 'how does this experience make you feel'. For our purpose this is valid, as is 'how easy was that'? We want to get snapshot insights. Often you don't, but I think for our purposes this could help.
• We are missing a chance of asking questions at the end of this test. Lets have some more general ones about the experience.

[karmatosed]

I'd love to see us focus on this and get complete for WCEU contribution day. Are you interested in that @rileybrook?

[rileybrook]

Definitely interested!

For anyone reading, I have incorporated @karmatosed's feedback into an edited script that now lives in this Google Doc. Anyone with the link should be able to edit, and comment. All ideas and edits welcome! Please add them on the document.

[karmatosed]

@rileybrook thanks for that. I have gone through the document. There is a lot of feedback, mainly as we are so close, lets really nail this. Once you iterate please let me know and we can hopefully be ready for the first test - I encourage you to also at least once test this to get feedback (once done iterations). Its important we try and get this done this week to be ready for WCEU. I hope that works - we are super close.

Once we have this, I will make a google doc and ensure everyone has access that needs it.

[rileybrook]

@karmatosed the document has been iterated. I implemented your suggestions and replied to your comments where more clarification might be required. I will be testing using the documents script a few hours from now.

[karmatosed]

Awesome, thanks so much @rileybrook. How did the test go?

[rileybrook]

@karmatosed It was great! I decided to do it in person, and without screen recording like some of the testers might do. The user was fairly new to Wordpress, but didn't have any obvious sticking points while using the editor.

[karmatosed]

I have made a form to start collecting tests, we can iterate on it, but wanted to have a starting point.

https://goo.gl/forms/4TWzY5Qo6Ku8ckV13

[karmatosed]

Closing as we have tests live.