Design
Evaluation
User Recruitment and Characteristics
We personally approached our recruited users and asked about their interest to a test user for WebAnnotator. All three of them agreed immediately so we did not have to look for a 4th candidate.
The first user is a CS researcher at MIT. The other two are undergraduates of different age and do not major in CS. We believe that this well represents our user population (recall: students and researchers). Moreover, Our users are quite diverse as their academic backgrounds are different.
Nonetheless, our users may lack the diversity in age as they are all under 25.
Procedure
Our users were asked to perform the following tasks. A short briefing of what WebAnnotator achieves was given at the beginning. However, no information about the interface was mentioned. We also encouraged our users to speak up about their thought process. No demo was performed.
We helped our users at time when they were stuck. Fortunately, this did not happen frequently.
Tasks
- Annotate a wikipedia page on your favorite topic. In WebAnnotator:
- Open the page for annotation.
- Highlight, underline and add notes to pieces of text in the page.
- Delete an annotation you have just made.
- Suppose you have annotated other webpages before using WebAnnotator.
- Find all your saved annotated pages.
- Reopen the one you just annotated.
- Open the one named Project 1: An ABC Music Player.
- Share your page with your best friend.
- Suppose the page is already shared with some people:
- Remove one of them
- Change one user's access rights from "can edit" to view-only.
- Add your best friend in.
- Suppose the page is already shared with some people: