Exercise on Writing Test Results
Consider the exam lookup page on the DePaul website and the following details of a usability test for it:
- Evaluation goal: can students look up an exam date and time correctly and in under 30 seconds?
- Task instructions: Imagine that you plan to take a course next quarter in the Loop. It meets at 11:50 on Mondays and Wednesdays for an hour and a half each day that it meets. Use this tool to find when the final exam for the course would be held. Report your result when you think you have it.
Now assume that an informal usability test was conducted with 4 undergraduate DePaul students. The observed notes report the following (with pseudonyms):
- Max selects "MW" from the Day menu and Loop from the Campus menu. She then selects 11:10am from the Start menu. She takes about 10 seconds in deciding to select 1:20pm from the End time menu. She left the Term menu unchanged. She starts to report the day of the final exam, but then says "No, that can't be right. That is November." She then sees the Autumn Term selection and changes it to Winter. Many options appear and she says "I don't know which one is right." She then clicks to different selections in the Campus box before seeing only one option displayed. She reports "Ok, I think it's Monday 3/13 at 11:50am". Total time is about 40 seconds.
- Sarah makes the following selections: Winter Term, MW, Loop, 11:50am, 1:20pm. She then reports, "it looks like it's Monday 3/13 at 11:50am." It took her about 10 seconds, with a delay of a few seconds before selecting the end time.
- Sam makes the following selections: Winter Term, MW, Loop, 11:50am, 12:20pm (note mistaken time). When nothing appears, he says, "I'm not seeing any results." He makes other selections with the days and the start time but nothing appears. He concludes, "maybe there is no final exam."
- Mary makes the following selections (same as Sarah): Winter Term, MW, Loop, 11:50am, 1:20pm. She then reports, "it looks like it's Monday 3/13 at 11:50am." It took her about 20 seconds with each selection taking a similar amount of time.
- Write a summary statement of the results.
- Write a summary statement of findings. In addition to offering an interpretation, the finding may recommend a revision to the design.
- Did you report any quantitative results? How?
- Did you minimize judgment and interpretation when reporting results?
- For your findings, would a reader be able to distinguish between your explanation of a problem and what you recommend for fixing the problem?