User tests

To access our app: http://people.csail.mit.edu.ezproxy.canberra.edu.au/d_chang/smak/

User 1

Okay, actually going to have this done at 7pm

TODO: Andrew clean this up

TODO: severity rating for each problem

tasks: I told him not to pick Boston, but to pick florence.
he tried to click on a particular place shown by the map.
he looked around the map to pick places.
wants a search feature for the map where you can find restaurants.
he manually plans for walking time, though didn't include enough.
the autocomplete for places doesn't really succeed in finding things that well.
It would be good if you could click on the map to pick a place.
if they type a full name, I should search for the location with google maps
he thought of the idea that he could drag the thing to where he
wants to go. that worked well, but placing it there, it did not stay.
his loop is: seach in the map for interesting places, then type in the name.
he knows how delete works.
The tabs are pretty intuitive.
the start and end time should be shown 
"how do I save?" we should have a way to email the schedule or something at least.
the autocomplete should recognise a category name and show options for that.
the location of a specific place is not actually being selected.
It should pop to the nearest place matching that category.
clicking on the pin, the nature of the info displayed is non-obvious.
It is not obvious what the start and end times are.
took a while to discover that you can lock something.
It is hard to lock things because clicking the lock is hard (fatfingers).
redo doesn't completely reverse the operation.
selecting an event should scroll to the event.
Changing the time fucks everything up.
It removes the schedulegrid.
Undo fixes that.
And then he somehow breaks everything and cannot create activities.
"""There should be a button that just says "pick shit for me to do"
that picks the top-10 things.""" (I disagree with this.)
He did not use autoschedule.

User 2

User 2 was a grad student from the media lab.  We chose this user because the user has never been to Florence but has some experience planning trips.

Usability problems:

• User tried tapping instead of dragging the resize handles.  (minor)
  • Possible solution: better grip icons
• Earliest start and latest end times were unintuitive and unused by the user.  When the grey bars appeared, the user was unsure of their meaning. (minor)
  • Possible solution: remove feature altogether. 
• On multiple occasions, the user tried to add a new activity while an activity was selected. Because the uses the same form to add a new activity as to edit an activity, a new activity could not be added until the activity was deselected. (major)
  • Possible solution:  Do not reuse the form to edit activities. Make the form inside the activity or as a popup box. 
• User tried pressing the “Schedule these activities for me” when there were no activities in the unscheduled activities column.  It is unclear to users that auto-schedule does not help optimize the current schedule, but instead optimally moves unscheduled items into the schedule.  (major)
  • Possible solution: Make more descriptive  label for button

User 3

User 3 was a college graduate (early/mid twenties age range) who had just finished applying to law school.  This user was chosen because his personality was meticulous and thought-out but had very little experience planning trips.

Usability problems:

• User had a different iPad with a different resolution: the "end of day" pull down was not on the screen and there was no indication of where it was supposed to be. (minor)
  • Possible Solution: set default day to be sufficiently small so that the entire (default) schedule fits on one screen with high guarantee (easy fix).  OR adjust based on dynamic iPad resolution (more difficult).
• As described in the list of tasks, finding the specific locations on the map was difficult.  It was confusing to have to constantly zoom in and out of the map to pinpoint locations. (major)
  • Possible Solution: allow the users to input street addresses to locate places, as opposed to pinning the location down on the map itself.
• User complained that activities disappeared all of a sudden (the user had accidentally clicked the X button for that activity).  Fortunately, there is an undo button but the user did not see it (the facilitator had to give a hint).  (major)
  • Possible solution: ask for confirmation when an activity is to be moved from the List of Activities column to the trash.
• There was no way to specify "end of day" when setting the latest start time---one can only specify exact times. (minor)
  • Possible solution: add both "end of day" and "beginning of day" to the start time/end time pull-downs.
• There is no way to make an activity with no location.  User wanted to have lunch anywhere around 12p-2p (minor)
  • Possible solution: if the user specifies no location for an activity, that activity will still take up space in the schedule column but not appear on the map for distance routing purposes
• User did not use the lock button for the opera. (minor)
  • Possible solution: add an element in the "Add Activity" flow for specifying specific starting times.
• User did not use the auto-schedule button. (minor)
  • Possible solution: Couple the auto-schedule button more tightly with the schedule column (see below for details).

Usability successes:

• What does the drag me button do?  While this caused confusion for a few seconds, the user quickly realized that the currently selected activity corresponded to drag me.
• The user correctly discovered how to drag and resize activities.

The biggest issue with this user was that he did not use the lock or auto-schedule buttons.  In the debriefing he said that their purpose wasn't clear when he could just manually create the schedule by moving activities around (i.e. what does it mean to "Lock" something if you don't know about the auto-schedule button?).  As a broad solution to this, the user suggested eliminating the auto-schedule button, and letting users fill up the schedule as they please.  Then, the user can say "finished" at which point the app pops up a dialogue that says "thank you for scheduling.  would you like SMaK to try to optimize your schedule for you?"  This flow would make the auto-schedule button take more of a center stage.

Report

Design

Structure and flow

The final design is three-part/column and has the following structure:

1. Left column (Add activity): user can add an activity, and change properties related to that activity such as its location, name, earliest start time and latest end time (e.g., museum closing times).  When an activity is added, it appears in the list of activities (center) column.
2. Center column (List of Activities): A list of all activities that the user wants to do, but have not yet been scheduled.  This is conceptually a TODO list: activities in this column have a duration (which can be changed by the user), but haven't been given a specific start time.  Activities move between this and the right column as the schedule changes.
3. Right column (Schedule): The concrete schedule of activities.  This column has an hour-by-hour calendar feel; activities placed on it have a duration and start time and abide by the following constraints. First, no activity may overlap with another activity (you can't be in two places at the same time).  Second, no activity can start before its earliest start time (specified in the left column) or after its latest end time (you can't visit a museum after it has closed).  Each activity has a "locked?" button which indicates whether the start time is flexible (this will be explained below).

The final design has the following flow.  The user adds a set of activities in the left column.  Then, the user constructs a final schedule through an iterative process of moving activities between the list of activities and schedule column.  When the user moves an activity from the list of activities to the schedule, some activity from the schedule may be displaced and moved back to the list of activities.  The application provides a "schedule these activities for me" button, which tries to schedule all of the remaining activities in the list of activities in one click.  When the user is satisfied with the state of the schedule column, the scenario is complete.  

Here is a screenshot of the main page in the application.  Both shots are the same screen. The right-hand side is annotated.

Platform

SMak is meant to be used on the go, when the user is traveling.  The UI is currently a web browser application which has been ported to an iPad for portability reasons.

Design evolution

Our original storyboard design (GR2) was a mobile app consisting of a series of forms that allowed users to specify “constraints” such as location, duration and ordering of events. 

During our paper prototype (GR3), we quickly learned that the constraints list was very confusing and the navigating through the forms was too complicated.  In addition, users wanted to interact with the schedule once it was created. 

For our computer prototype (GR4), we decided to abandon the mobile platform due to problems with drag and drop, and instead created a web app that shows all the different views from the mobile app on one screen.  In addition, we decided to display and edit all constraints visually, such as allowing users to resize activities to change its duration or dragging activities around in the schedule to rearrange them.  We used an edit distance algorithm to move conflicting activities around if the user drops an activity into an occupied time lot.  This allowed users to rearrange their day without manually shifting each activity. 

For our final implementation (GR5), we realized we needed to make the web app work on an iPad to make it portable for traveler, so we figured out how to make drag and drop work on the iPad browser.  This new form factor meant the resize handles were very difficult to grab (since our fingers are a lot fatter than the mouse), so we created much larger round handles outside of the activity box to allow users to easily resize the activity.  We also added many features suggested in the heuristic evaluations including an auto-completing form to allow users to either pick from a list of activities or create their own activities, undo, locking activities in the schedule, optimized auto-scheduling using traveling salesman, earliest start time and latest end time, and draggable map pins.

Implementation

View

The interface was implemented using Jquery UI which included libraries to create draggable and resizable boxes, as well as the autocomplete input box in the form.  

When the page first loads, it renders the calendar grid based on the length of the schedule and the starting time.  Each time the start or end times of the day changes, the entire calendar grid and all items in the schedule get redrawn.  Each time an activity is added to the “List of Activities” either from the form or from being unscheduled, a new activity box is drawn in the “List of Activities” column.  This means that the “List of Activities” column is modified with each change, but not completely redrawn.  The one time the entire “List of Activities” is redrawn is when the undo and redo buttons are used.  On the other hand, whenever any part of the schedule column changes, the entire schedule is redrawn because often times, more than one element in the schedule is moving and it is hard to keep track of what changed.

In addition to rendering the boxes for each activity, the view also dynamically updates the activity boxes as they are moved and resized so that they show error messages, map labels and the current duration and times.  A single activity can be selected, at which point, it greys out parts of the schedule in which the selected activity cannot be done between.

TODO: Andrew: maps

Model

TODO: Andrew

- object array / uniqid

- invariant checking

- handler propegation

Scheduling algorithms (Controller)

When an activity moves from the Activities column to the Scheduling column, an important decision to make is how to "make room" for the new activity in the Scheduling column.  We implemented separate algorithms for the following operations:

1. Moving an activity onto/within the schedule, or resizing an activity already on the schedule.
2. Changing the schedule's start or end times.
3. Clicking the "Schedule these activities for me" button (a.k.a. auto-scheduling).

Broadly speaking, the algorithms rely on the principle of least destruction: when adding to/resizing/etc the schedule, don't move activities already in place unless they have to be moved.  We chose this heuristic for usability reasons: if the user is planning a schedule and moves some activity A, he/she will not want the rest of the schedule to change in unnecessary ways. So, if an activity is moved into an empty space, no other activities should have to move.  If one activity A is moved on top of another activity B, activity B should "scoot" or somehow else move to make room for A.  If changing the schedule makes placing some activities impossible (due to time constraints, etc), the activities are moved to the Activities column.  The algorithms try out different possible schedule configurations based on these principles and choose the best configuration based on a weight function.

The auto-scheduling algorithm, in addition to the above heuristics, tries to minimize the average euclidean distance between each activity.  This was also done to generate better schedules: given a set of activities, the user wants to complete as many as possible in a fixed amount of time.  The algorithm internals are traveling-salesman-like: we have a set of unscheduled items which we add to the schedule in different ways (while observing the "principle of least destruction").  Then, we choose the best complete configuration based on the weight function we described above plus the distance metric.  Despite our small problem scale, the brute force approach did not work time-wise; so our algorithm is greedy, with intelligence to avoid getting stuck in bad schedules.

Evaluation

We chose users who had a desire to travel to foreign places and had different travel.  For example, young people who travel through Europe do so "by the heals of their pants," jumping from hostel to hostel.  Older travelers, however, probably want a more set-in-stone schedule.  Older travelers are probably more experienced in traveling, as well.

Procedure

We read the briefing (below) to each user.  We then gave each task on an index card to each user.  The next card was given when the user completed the task on the current card.  No demo was given to any user.  After the user test completed, we debriefed each user and let them know how to overcome problems they were having (if any) with the interface.  We have listed usability problems during the user tests (and potential solutions) at the top of the page under the User tests section).

Briefing

SMak helps you plan your schedule.  When people travel to foreign countries, they typically know what you want to see but not how to go about doing it.  With SMak, your job is to specify what your activities are and give hints as to how you want to carry out those activities.  Smak's job is to help fill in the details of your schedule: how to get from place to place, in what order to do your activities---these sorts of things.

Tasks

1. You want to visit Florence.
2. You want to wake up at 8am and go to bed at 10pm.
3. You want to do the following activities throughout the day: eat breakfast/lunch/dinner, visit the dome, and go to the opera.
4. Now, you want to visit tourist places in Florence *and want to do as little walking as possible.*  You want to visit the dome (at the piazza del duomo) at some point in the day for two hours; it doesn't matter when you go.  The dome opens at 10am and closes at 5pm.  You also want to go to the Medici chapel for a special opera showing that starts at exactly 3pm and lasts for 2 hours.

Reflection

TODO: Andrew