GR6: User Testing and Final Report

Due in 8 days, on Wednesday, December 13th, 2006, by email and wiki.

Overview

In this group assignment, you will evaluate your interface with a small user test and write a final report.

User Testing

Find at least 3 representative users from your target population. None of your users should be enrolled in 6.831. All should be willing to participate voluntarily.

Prepare a briefing and three tasks. These may be the same ones that you used in paper prototyping, but you may need to improve them based on feedback from the paper prototyping.

You may, if you wish, also prepare a short demo of your interface that you can use to show your users the purpose of the system. The demo should be scripted, so that you do and say the same things for each user. It should use a concrete example task, but the example task should be sufficiently different from the test tasks to avoid bias. The demo option is offered because some interfaces are learned primarily by watching someone else use the interface. Think carefully about whether your interface is in this category before you decide to use a demo, because the demo will cost you information. Once you've demonstrated how to use a feature, you forfeit the chance to observe how the user would have learned to use it alone.

Pilot test your briefing, demo, and tasks, before you test your real users. You can use another member of the class for your pilot testing, if you wish.

Conduct a formative evaluation with each user:

Provide your briefing and (optionally) demo.
Then provide the tasks one at a time, observe, and take notes.

One member of your group should be the facilitator of the test, and the rest should be observers. Watch and record critical incidents.

We don't recommend that you videotape your users. However, if you want a record of the user test to supplement your notes, you may try using screen capture software, such as Camtasia Studio.

Collect the usability problems found by your user tests into a list. Assign each problem a severity rating (cosmetic, minor, major, catastrophic), and brainstorm possible solutions for the problems.

Final Report

Write a final report of approximately 10 pages describing your complete project. The report should have the following parts:

Problem (1 page). What user problem are you trying to solve? Who are the users? What are their important tasks?
Design (4-5 pages). Describe the final design of your interface. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, and user testing).
Implementation (1 page). Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface.
Evaluation (2 pages).Describe how you conducted your user test. Describe how you found your users and how representative they are of your target user population. Describe how users were briefed and what tasks they performed; if you did a demo for them as part of your briefing, justify that decision. List the usability problems you found, and discuss how you might solve them.
Reflection (1 page). Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations.

Wiki

Summarize the parts of your final report on your project's wiki page. Your wiki should have the following new sections:

Final Design. A description of your final interface, including a few screenshots.
Implementation. A summary of how your project was implemented.
User Testing. A paragraph about how you conducted your user testing, and a list of usability problems that were found.
Reflection. A summary of your reflection.

Tidy up your entire wiki to make it a complete and readable description of your term project. If your project changed direction or scope over the course of the semester, update earlier sections (such as the Problem section) to reflect your final project.

What to Hand In

This assignment has two deliverable components: a written report, and the update to your wiki page described above. Both must be complete by the deadline.

Email your report as a PDF file to 6831handin@csail.mit.edu and include GR6 in the subject line.

6.831 • User Interface Design and Implementation

Links