GR6 User Testing - WebAnnotator

Back To Project Home

Design

Implementation

Front-end

Technology

The front-end UI is implemented in javascript and html. It is developed and tested primarily on the Chrome browser. Firefox should also work, though not extensively tested. Some new CSS 3 features are adopted. Browsers that don’t support CSS 3 features such as box-shadow might not display the web application correctly.

On the Javascript side, jquery, jquery-ui and a third-party Javascript library called rangy are used.

Logic

The annotations are modifications to the HTML code made by javascript on the client side, so that there is no delay between pressing of an annotation button (such as “Highlight”) and seeing the effects (text selection highlighted with a bright yellow background). An update to the server is sent immediately after, hence the “auto-save” feature.

Back-end

Technology

The back-end server is implemented in Java. HTTP requests sent from the front-end javascript are served by Java servlets. We are using Apache Tomcat latest version (7.0) to host the server.

Logic

HTTP requests sent from client to server:

• /newpage?url=...: Sent when a new page is created from a user-provided URL ready for annotation. Server returns a URL to the new page to be annotated.
• /saveAnnotatePage: Sent when an annotated page needs to be saved. The message body includes the entire modified HTML of the page being annotated.

When a “newpage” request is received by the servlet, it first downloads the page using wget, then a number of modifications are made on the page, including disabling links on the original page and adding our annotation button HTML code and Javascript files.

The saved pages are stored as files in the file system on the server, i.e. there is no formal database at the back-end. Server keeps track of the paths of the saved pages and serves them to the client when asked.

Design Decisions that Affect Usability

Server uses “wget” to download the webpage to be annotated, which has limitations. Some webpages dynamically generated by Javascript seems not to be downloaded correctly, and there are other pages whose format and styling are not preserved well.
The main effect on usability is that the user seems to be limited on the range of webpages that they can annotate. Not loading pages correctly frustrates the user.

Evaluation

User Recruitment and Characteristics

We personally approached our recruited users and asked about their interest to a test user for WebAnnotator. All three of them agreed immediately so we did not have to look for a 4th candidate.

The first user is a CS researcher at MIT. The other two are undergraduates of different age and do not major in CS. We believe that they well represent our user population (recall: students and researchers). Moreover, Our users are quite diverse as their academic backgrounds are different.

We also asked our users about their previous experience with annotating document. All of them have read annotated document before but have rarely annotated themselves. The advantage of this is that we could better evaluate the learnability of our interface because of their limited experience.

Nonetheless, our users may lack the diversity in age as they are all under 25.

Procedure

Our users were asked to perform the following tasks. A short briefing of what WebAnnotator achieves was given at the beginning. However, no information about the interface was mentioned. We also encouraged them to speak up about their thought process. No demo was performed.

We helped our users at time when they were stuck. Fortunately, this did not happen frequently.

Tasks

1. Annotate a page on your favorite topic. In WebAnnotator:
   1. Open the page for annotation.
   2. Highlight, underline and add notes to pieces of text in the page.
   3. Delete an annotation you have just made.
2. Suppose you have annotated other webpages before using WebAnnotator.
   1. Find all your saved annotated pages. (Note: about 30 saved pages were created beforehand in order to test the search box)
   2. Reopen the one you just annotated.
   3. Open the one named Project 1: An ABC Music Player.
3. Share your page with your best friend.
   1. Suppose the page is already shared with some people:
      1. Remove one of them
      2. Change one user's access rights from "can edit" to view-only.
      3. Add your best friend in.

Observation

User 1 (CS Researcher)

• Task 1
  • After highlighting, did not notice the popup annotation buttons at first. Was looking for a tool bar at the top and searching throughout the webpage.
  • Adding multiple annotations to the same piece of text seemed difficult.
  • Was looking for a "save" button before noticing the message "All changes saved" at the top.
  • The task was successfully completed otherwise.

• Task 2
  • Did not realize that the search box implements only prefix search. Entered keywords like "abc" and "player".
  • Had to locate the entry for "Project 1: An ABC Music Player" by manually searching through all the entries.
  • Had some difficulty to go back to the dashboard from an annotated page. Did not remember that the button "Saved Pages" would bring her back to the dashboard. Recommended changing the wording to "All pages".
  • The task was successfully completed otherwise.

• Task 3
  • In the share dialog box, the "plus" button was not pressed before clicking "done".
  • The task was successfully completed otherwise.

User 2 (Math Undergraduate)

• Task 1
  • Entered the URL for the Wikipedia main page. Tried to use the search box there to find a desired page before finding that the search box was disabled.
  • Thought that highlighting a piece of already highlighted text would un-highlight it. Took quite a while to realize the delete button would show up when clicking on it.
  • Tried to delete a note by highlighting the text in the note box.
  • The task was successfully completed otherwise.

• Task 2
  • Mistook the search box in the dashboard page with the functionality to "add a new page".
  • Did not realize that the search box implements only prefix search. Entered keywords like "abc" and "player".
  • Then tried to enter a URL in the search box to get to the page created in the last task.
  • Finally, had to locate the entry for "Project 1: An ABC Music Player" by manually searching through all the entries.
  • The task was successfully completed otherwise.

• Task 3
  • The task was completed successfully and quickly.

User 3 (Mechanical Engineering Undergraduate)

• Task 1
  • Entered the URL for the Wikipedia main page. Tried to use the search box there to find a desired page before finding that the search box was disabled.
  • Then tried to use the URL box as the google search box by entering a keyword. At this point the facilitator told her that a URL was needed.
  • Tried to click the shortcut link to jump to some section in the Wikipedia article. Realized that hyperlinks are disabled.
  • Didn't know how to add notes. Was looking for the button without success.
  • Recommended that clicking (in addition to highlighting) should also make the annotation toolbar appear.
  • The task was successfully completed otherwise.

• Task 2
  • First used the search box thinking that it implements substring search. Then realized it actually implements prefix search. Found page successfully.
  • The task was successfully completed otherwise.

• Task 3
  • In the share dialog box, the "plus" button was not pressed before clicking "done".
  • The task was successfully completed otherwise.

Interestingly, all three of our users chose a Wikipedia page even though nothing about Wikipedia was mentioned in any verbal or written description throughout the testing. This may indicate that Wikipedia articles are among the most popular webpages to be annotated and more effort should be geared to make our interface more usable for Wikipedia articles.

Usability Problems and Solutions

Reflections