User tests

To access our app: http://people.csail.mit.edu.ezproxy.canberra.edu.au/d_chang/smak/

User 1

Okay, actually going to have this done at 7pm

TODO: Andrew clean this up

TODO: severity rating for each problem

tasks: I told him not to pick Boston, but to pick florence.
he tried to click on a particular place shown by the map.
he looked around the map to pick places.
wants a search feature for the map where you can find restaurants.
he manually plans for walking time, though didn't include enough.
the autocomplete for places doesn't really succeed in finding things that well.
It would be good if you could click on the map to pick a place.
if they type a full name, I should search for the location with google maps
he thought of the idea that he could drag the thing to where he
wants to go. that worked well, but placing it there, it did not stay.
his loop is: seach in the map for interesting places, then type in the name.
he knows how delete works.
The tabs are pretty intuitive.
the start and end time should be shown 
"how do I save?" we should have a way to email the schedule or something at least.
the autocomplete should recognise a category name and show options for that.
the location of a specific place is not actually being selected.
It should pop to the nearest place matching that category.
clicking on the pin, the nature of the info displayed is non-obvious.
It is not obvious what the start and end times are.
took a while to discover that you can lock something.
It is hard to lock things because clicking the lock is hard (fatfingers).
redo doesn't completely reverse the operation.
selecting an event should scroll to the event.
Changing the time fucks everything up.
It removes the schedulegrid.
Undo fixes that.
And then he somehow breaks everything and cannot create activities.
"""There should be a button that just says "pick shit for me to do"
that picks the top-10 things.""" (I disagree with this.)
He did not use autoschedule.

User 2

• User tried tapping instead of dragging the resize handles.  (minor)
  • Possible solution: better grip icons
• Earliest start and latest end times were unintuitive and unused by the user.  When the grey bars appeared, the user was unsure of their meaning. (minor)
  • Possible solution: remove feature altogether. 
• On multiple occasions, the user tried to add a new activity while an activity was selected. Because the uses the same form to add a new activity as to edit an activity, a new activity could not be added until the activity was deselected. 
  • Possible solution:  Do not reuse the form to edit activities. Make the form inside the activity or as a popup box. (major)
• User tried pressing the “Schedule these activities for me” when there were no activities in the unscheduled activities column.  It is unclear to users that auto-schedule does not help optimize the current schedule, but instead optimally moves unscheduled items into the schedule.  (major)
  • Possible solution: Make more descriptive  label for button

User 3

TODO: Chris clean up

1.) Had to scroll down to get to the "end day at" pull down.  (the iPad used for this test had a different resolution than the iPad that was used for testing...).  No indication that the "end day Minor.
What is the drag me do?  playing with the drag me button--has to do with where the map zooms in eventually?  oh I see what happened--we dragged lunch over there!  let's try adding dinner.  crap I keep zooming in by accident.  visit dome ... add activity and go to opera.  add activity so let me see if I can order these things around.
each should take 1 hour... ok.  so I need to zoom in oh no I don't nvm... so let's see breakfast gets dragged over and ooh ok I can resize the breakfast icon to make it take longer (I was hoping it would do something like that!)  let's schedule lunch over here.  ok so now I need to zoom in (I can't see shit).  wait where did my breakfast go... it seems to have dissapeared completely (the user accidentally pressed the X delete button).  user doesn't see how to get breakfast back or how to undo.  <observer tells the user about undo>; gets breakfast back.  typed in eat breakfast vs. breakfast---user got confused about defaults.  it is confusing that activities are on top of one another by default (i.e. you can't see how many are really there).  what are the grey blocks?  oh I see they enforce the latest end time?  that's good.  now, I guess we don't care where lunch is eaten so lunch is .... so dinner doesn't have a latest time but there doesn't seem to be a way to specify that?  why can't I just say "none" or end of day?  hm I can't find C.  C was dinner where did C go?  C has dissapeared (ONLY C has dissapeared on the map--probably a bug).  User tries to undo to fix the problem, but then progress is lost.  lost C again... right after scheduled dinner.  C dissapeared again.
Now scheduling the opera.  C showed up again!  Showed up in the wrong spot (southeast of where it was supposed to be).  where is the medici chapel?  ok found it.  it marked dinner at centro storico and then it appeared at ... can't read where (icons are in the way).  when I added the opera.  did not use the locked button.  did not use the auto-schedule button.
<post-briefing>
would have been nice to just have the schedule column to have a sense for the schedule; and then there should be an optimize button.

Report

Design

Structure and flow

The final design is three-part/column and has the following structure:

1. Left column (Add activity): user can add an activity, and change properties related to that activity such as its location, name, earliest start time and latest end time (e.g., museum closing times).  When an activity is added, it appears in the list of activities (center) column.
2. Center column (List of Activities): A list of all activities that the user wants to do, but have not yet been scheduled.  This is conceptually a TODO list: activities in this column have a duration (which can be changed by the user), but haven't been given a specific start time.  Activities move between this and the right column as the schedule changes.
3. Right column (Schedule): The concrete schedule of activities.  This column has an hour-by-hour calendar feel; activities placed on it have a duration and start time and abide by the following constraints. First, no activity may overlap with another activity (you can't be in two places at the same time).  Second, no activity can start before its earliest start time (specified in the left column) or after its latest end time (you can't visit a museum after it has closed).  Each activity has a "locked?" button which indicates whether the start time is flexible (this will be explained below).

TODO: Chris, picture with no red lines

The final design has the following flow.  The user adds a set of activities in the left column.  Then, the user constructs a final schedule through an iterative process of moving activities between the list of activities and schedule column.  When the user moves an activity from the list of activities to the schedule, some activity from the schedule may be displaced and moved back to the list of activities.  The application provides a "schedule these activities for me" button, which tries to schedule all of the remaining activities in the list of activities in one click.  When the user is satisfied with the state of the schedule column, the scenario is complete.  

Platform

SMak is meant to be used on the go, when the user is traveling.  The UI is currently a web browser application which has been ported to an iPad for portability reasons.

Design evolution

Our original storyboard design (GR2) was a mobile app consisting of a series of forms that allowed users to specify “constraints” such as location, duration and ordering of events. 

During our paper prototype (GR3), we quickly learned that the constraints list was very confusing and the navigating through the forms was too complicated.  In addition, users wanted to interact with the schedule once it was created. 

For our computer prototype (GR4), we decided to abandon the mobile platform due to problems with drag and drop, and instead created a web app that shows all the different views from the mobile app on one screen.  In addition, we decided to display and edit all constraints visually, such as allowing users to resize activities to change its duration or dragging activities around in the schedule to rearrange them.  We used an edit distance algorithm to move conflicting activities around if the user drops an activity into an occupied time lot.  This allowed users to rearrange their day without manually shifting each activity. 

For our final implementation (GR5), we realized we needed to make the web app work on an iPad to make it portable for traveler, so we figured out how to make drag and drop work on the iPad browser.  This new form factor meant the resize handles were very difficult to grab (since our fingers are a lot fatter than the mouse), so we created much larger round handles outside of the activity box to allow users to easily resize the activity.  We also added many features suggested in the heuristic evaluations including an auto-completing form to allow users to either pick from a list of activities or create their own activities, undo, locking activities in the schedule, optimized auto-scheduling using traveling salesman, earliest start time and latest end time, and draggable map pins.

Implementation

View

The interface was implemented using Jquery UI which included libraries to create draggable and resizable boxes, as well as the autocomplete input box in the form.  

When the page first loads, it renders the calendar grid based on the length of the schedule and the starting time.  Each time the start or end times of the day changes, the entire calendar grid and all items in the schedule get redrawn.  Each time an activity is added to the “List of Activities” either from the form or from being unscheduled, a new activity box is drawn in the “List of Activities” column.  This means that the “List of Activities” column is modified with each change, but not completely redrawn.  The one time the entire “List of Activities” is redrawn is when the undo and redo buttons are used.  On the other hand, whenever any part of the schedule column changes, the entire schedule is redrawn because often times, more than one element in the schedule is moving and it is hard to keep track of what changed.

In addition to rendering the boxes for each activity, the view also dynamically updates the activity boxes as they are moved and resized so that they show error messages, map labels and the current duration and times.  A single activity can be selected, at which point, it greys out parts of the schedule in which the selected activity cannot be done between.

TODO: Andrew: maps

Model

TODO: Andrew

- object array / uniqid

- invariant checking

- handler propegation

Scheduling algorithms (Controller)

When an activity moves from the Activities column to the Scheduling column, an important decision to make is how to "make room" for the new activity in the Scheduling column.  We implemented separate algorithms for the following operations:

1. Moving an activity onto/within the schedule, or resizing an activity already on the schedule.
2. Changing the schedule's start or end times.
3. Clicking the "Schedule these activities for me" button (a.k.a. auto-scheduling).

Broadly speaking, the algorithms rely on the principle of least destruction: when adding to/resizing/etc the schedule, don't move activities already in place unless they have to be moved.  We chose this heuristic for usability reasons: if the user is planning a schedule and moves some activity A, he/she will not want the rest of the schedule to change in unnecessary ways. So, if an activity is moved into an empty space, no other activities should have to move.  If one activity A is moved on top of another activity B, activity B should "scoot" or somehow else move to make room for A.  If changing the schedule makes placing some activities impossible (due to time constraints, etc), the activities are moved to the Activities column.  The algorithms try out different possible schedule configurations based on these principles and choose the best configuration based on a weight function.

The auto-scheduling algorithm, in addition to the above heuristics, tries to minimize the average euclidean distance between each activity.  This was also done to generate better schedules: given a set of activities, the user wants to complete as many as possible in a fixed amount of time.  The algorithm internals are traveling-salesman-like: we have a set of unscheduled items which we add to the schedule in different ways (while observing the "principle of least destruction").  Then, we choose the best complete configuration based on the weight function we described above plus the distance metric.  Despite our small problem scale, the brute force approach did not work time-wise; so our algorithm is greedy, with intelligence to avoid getting stuck in bad schedules.

Evaluation

We chose users who had a desire to travel to foreign places and had different travel.  For example, young people who travel through Europe do so "by the heals of their pants," jumping from hostel to hostel.  Older travelers, however, probably want a more set-in-stone schedule.  Older travelers are probably more experienced in traveling, as well.

Procedure

We read the briefing (below) to each user.  We then gave each task on an index card to each user.  The next card was given when the user completed the task on the current card.  No demo was given to any user.  After the user test completed, we debriefed each user and let them know how to overcome problems they were having (if any) with the interface.  We have listed usability problems during the user tests (and potential solutions) at the top of the page under the User tests section).

Briefing

SMak helps you plan your schedule.  When people travel to foreign countries, they typically know what you want to see but not how to go about doing it.  With SMak, your job is to specify what your activities are and give hints as to how you want to carry out those activities.  Smak's job is to help fill in the details of your schedule: how to get from place to place, in what order to do your activities---these sorts of things.

Tasks

1. You want to visit Florence.
2. You want to wake up at 8am and go to bed at 10pm.
3. You want to do the following activities throughout the day: eat breakfast/lunch/dinner, visit the dome, and go to the opera.
4. Now, you want to visit tourist places in Florence *and want to do as little walking as possible.*  You want to visit the dome (at the piazza del duomo) at some point in the day for two hours; it doesn't matter when you go.  The dome opens at 10am and closes at 5pm.  You also want to go to the Medici chapel for a special opera showing that starts at exactly 3pm and lasts for 2 hours.

Reflection

TODO: Andrew