HCI 440

Fall 2022

Craig Miller

Weekly Notes

Assignments

Team Project 4
UX Evaluation
Due Sunday November 20 before 11:30pm

Overview

For the fourth phase of the team project, you will conduct usability tests, either with paper prototypes or software-based designs (e.g. HTML or Axure). You will then revise the design to address any usability problems you observed during the tests.

Assignment Requirements

Your tests should involve at least one participant per team member. The informed consent and the task "instructions" must be scripted, but all other aspects of the test may be informally conducted.

Test Roles

Rettig mentions several roles that team members can play while conducting a test: greeter, facilitator, computer, and observers. You will find descriptions of these roles in his article, "Prototyping for Tiny Fingers." To carry our your test, assign members of your team to these roles. For small groups, you will need to consolidate some of the roles. For example, the greeter and the facilitator is often the same person. While not ideal, it is possible to conduct the test with just one practitioner.

Test Design

In your last assignment, you describe testing scenarios for your designs. For this assignment, you should develop the testing procedure for these scenarios. We will discuss tests and conduct pilot runs in class. The text and the Rettig article also provides some suggestions.

Write out your instructions to the participant and decide in advance when and if you will intervene while the test is running. We will discuss how to write good instructions in class.

Evaluation Process

When conducting each test, make sure you do the following:
  1. Obtain the informed consent of each participant.
  2. Make the participant feel comfortable and ensure that he or she is not being tested.
  3. Follow your test script and read your instructions.
  4. Observers should record the user's actions that are relevant to your testing scenarios.
  5. When the test is over, thank the participant and answer any questions.

The Report

Your report should include the following sections.

  • Executive summary. This summary is a one-to-two paragraph summary describing the test, its results and changes made to your design.
  • Test Plan. This is a couple of paragraphs describing how you conducted your test.
  • Results Summary. This section summarizes events and measures (if any) that were observed from the test.
  • Findings and Recommendations Summary. This section summarizes interpretations and recommendations. They should be clearly based on what was observed from the results section.
  • Redesign summary. This section describes and motivates the changes you made to your designs. You may want to include and refer to screen shots when presenting your changes.
  • Appendix: Test materials. This final section is really an appendix consisting of the details and materials you used to conduct the usability test. It has the following documents:
    • Team Member Roles. List your team members and specify the roles they play. If they vary from the descriptions provided by Rettig, explain them.
    • Consent statement. Document how each participant consented to participate in your test.
    • Instructions. Provide the script of instructions that are read to each participant.
    • Detailed Results. Provide your documentation of the detailed results. It should include an organized set of notes from the usability tests.
    • Also provide a description of how team members contributed to this project in addition to their test roles.

Team Process

The following is a recommended set of team phases and deadlines:

  • Specify objective and task instructions. (Friday November 11). As a team, decide on at least one test objective, a task that addresses it and written scenario instructions for the test user.
  • Testing. (Tuesday November 15). Each team member should conduct at least one usability test.
  • Consolidation of results, findings and recommendation. (Friday November 18). List objective results and work out findings. Recommendation can be used to summarize changes in the interaction design.
  • Finalize. (Sunday November 20). Write executive summary. Edit and proofread. Submit!

Submission

Submit your report to D2L as a PDF document.

Grading

This project is worth 10% of the course grade. The following criteria will be used for scoring the report:

  • Report organization and presentation. The report is well written and organized so that it is easy to find required information. Results, findings and recommendations are clearly and appropriately separated.
  • Report completeness. The report has all required contents.
  • Process. The shows that appropriate methods were followed.
  • Supported design. Results support the findings. Findings suppor the changes in the design.