...

In this section we will highlight the different aspects of the interface, and comment on our reasoning for making the choices we did.

Login / Create an Account

Figure 1: Login

The first thing the user sees when opening BucketList is a plain bulletin board with a large piece of paper asking them to log in or create an account. We always planned on having the initial landing page be simple, and there were no problems with it during testing, so this part did not change very much based on user feedback. 

Main Page - Bulletin Board

...

We made sure that each of the users who participated in final user testing use to-do lists on a regular basis and have a need to keep track of group projects, thus they all fit perfectly into our target user population. Since BucketList targets such a wide range of users, we also made sure our test subjects came from varying demographics with respect to age, location, gender, education, and technological expertise, so as not to limit ourselves to other MIT students who think too similarly to us. Many users in our evaluation were MIT student (by far the easiest to find), but we also included professionals and high-school students (found by speaking with friends and family) who we believe would benefit from BucketList. 

Briefing

The following short briefing was given to each test subject before we began the evaluation:

"BucketList is a shared task-management application, allowing users to easily create group to-do lists.

Once you have an account, you can create "Buckets" - categories to put your tasks in. You can then add tasks to your existing buckets. This application is different than most task-management applications you've probably used in the past, because you can also share buckets with your friends. So, for example, you might create a "Final Group Assignment" bucket, which you will share with the rest of your group. You can then assign each person in the group individual tasks, like "do research" or "outline presentation", and monitor your progress, as a group, towards completing the project. You can also write and share notes about each task, to make sure you're all on the same page, and you can even send alerts to make sure your friends don't miss important notes.

We are going to describe a scenario for you, and ask you to use BucketList to help you accomplish your goals. This shouldn't take more than 10-15 minutes. Do you have any questions before we start?"

We decided not to do a demo as part of the briefing, as we really wanted to see if users would be able to figure out how to accomplish key tasks, so we didn't want to show them first. Our application is simple enough that users can figure it out themselves, without needing to see others use it first.

Tasks

Instead of giving users specific tasks to accomplish, we decided to give them a scenario with less guidance, and observer how they would use BucketList to help organize themselves. This allowed us to see which features are more important, and which features users don't attempt to use - something that would not be obvious if we specifically prompted them to use each feature. We structured the scenario in such a way that it would require the use of the entire system. 

...

Please imagine that you are a junior in college, taking 4 classes, and editor of the student newspaper. You already have a BucketList account with Buckets for 'Newspaper' and 'French Literature Project'. There are some notes on your board reminding you about the presentation for French Literature. 

1. You were just assigned a final project in your Computer Systems class. You need to write a report for next Friday, and give a presentation the following Monday.
   1. Your teacher reminds you that the presentation cannot be longer than 5 minutes, and must include a visual aid.
2. You just submitted the proposal for the group final project in your French Literature class.
   1. You remember that Tracey needs to return the book your group borrowed to the library.
   2. You decide Fatima will take over your responsibility to prepare the presentation.
3. You check what articles are scheduled to be written for this Thursday's newspaper publication.
   1. You decide that there should be a newspaper article about MIT 150 this week, and assign Jason and Matt to write it. It needs to be completed by Wednesday night.
4. You plan what you will do tomorrow, and then go to sleep.

Results of Usability

...

Testing

Reflection

• Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations.

The most important thing we learnt is that user testing is really important!

The Good

• Most of our users loved the look and feel of the website. The metaphor of using a bulletin board with stickies and paper.
• The idea of task notes and general notes was well received also
• The addition of due dates and the calendar was intuitive
• The idea of having collaborators in a bucket was understood by all users

The Bad and Ugly

• Bucket Metaphor
  • Sevierity - Minor
  • Explanation - Most of the users didn't not know what a bucket was, it had to be explained to them.
  • Possible Solution - We could revert to something more common like categories
• Long Names
  • Severity - Cosmetic
  • Explanation - Some users seem to use long titles in their tasks and did not like the fact that it showed a scroll widget after 2 lines. They wanted it to be symmetric with the short named tasks
  • Possible Solution - We could truncate to single line and have ... at the end. This decreases visibility, but opening a task will show the full name again
• Draggables
  • Severity - Major
  • Explanation - Sometimes, some draggable items on the screen do not get released and follow the mouse indefinitely. This annoyed a lot of the users
  • Possible Solution - More work on the JavaScript.
• Creation of Buckets
  • Severity - Minor
  • Explanation - Because the bucket metaphor failed, most users didn't create buckets for each classes in the task, they just added the tasks in one of the two pre-made buckets.
  • Possible Solution - Demo video or quick interactive instructions after login.
• Task Collaborators
  • Severity  - Minor
  • Explanation - Assigning tasks to collaborators was misunderstood by people. Most of the users failed to realize that a collaborator in a bucket isn't the same in a task. When asked to assign a person to a task, they would single click the task, then check a collaborator in the bucket view
  • Possible Solution - Explain to them - again, demo video or quick interactive instructions
• Notes Vs Tasks
  • Severity - Minor
  • Explanation - Even thought users loved the ideas of having notes and tasks, they weren't sure when to uses a note or make a task, or if other people would see the notes
  • Possible Solution - Explanation, demo video, or quick interactive instructions.
• Enter to complete
  • Severity - Minor
  • Explanation - A few users weren't sure how to commit their typed tasks, this is done by pressing enter. They were searching for a done or OK button. But after a while, it was learned. One user asked how to enter a task in.
  • Possible Solution - Either put a done button, which could decrease efficiency but increase familiarity. OR explain to users. Adding both is an unnecessary redundancy. 
• Note Edit
  • Severity - Cosmetic
  • Explanation - Most users expected that clicking in a note would edit it. They didn't actually notice the pencil edit icon on the bottom right corner.
  • Possible Solution - After explaining the reason for that was because of the dragging feature, they were ok with it.

Quotes

"How do i access my other bucket again"
"What is a bucket?"
"How do i do that?" <- After being asked to complete task 1
"You gotta fix this drag issue"
"Oh wow, looks like a bulletin board, nice"

Reflection

The most important thing we learnt is that user testing is really important!

If we had to do it again, I think a heuristic evaluation of our initial designs would have been useful, to get some very early feedback on our design (GR2) before even doing the paper prototypes. We spoke to some classmates about our design, but a more in-depth review would have pointed out some obvious problems that we could have then avoided in the paper prototypes. Since this would have been relatively low-cost and easy to do, we would definitely incorporate it in the future.

We would also have done more rounds of testing with paper prototypes. We ended up throwing away a lot of code after doing a round of informal user testing on our initial design (before GR4), and a 3rd round of paper prototype testing would have brought up these issues before we began coding, saving us lots of time. At the same time, some issues became much more obvious once we actually wrote some code (for example, the paper prototype was very big, which hid the fact that screen real-estate was actually very limited in our initial design). Thus writing code and testing that was useful, and it was good that we weren't afraid to throw away code and restart during the early stages of codingIf we had to do it again, I think a heuristic evaluation of our initial designs would have been useful, to get some very early feedback on our design (GR2) before even doing the paper prototypes. We spoke to some classmates about our design, but a more in-depth review would have pointed out some obvious problems that we could have then avoided in the paper prototypes. We would also have done more rounds of testing with paper prototypes. We ended up throwing away a lot of code after doing a round of informal user testing on our initial design (before GR4), and a 3rd round of paper prototype testing would have brought up these issues before we began coding, saving us lots of time.

Reviewing the heuristic evaluations and deciding what advice was important and what we should ignore was also important. Some of the advice we got in our heuristic evaluation was very useful, but other advice was misguided (perhaps because the back-end functionality wasn't working yet, so users had difficulty evaluating some parts of the application). If we had blindly followed all the advice we got, I think we would've ended up scrapping a lot of key features that were important to our design but not clear in implementation as of GR4. Having a relatively advanced prototype as of GR4 helped a lot in this respect, though, because testers could get a better sense of our application.

Another aspect of the design process that was very important was the initial project proposal and analysis, background research, etc. Speaking to users and figuring out what they would actually want was really important and helpful in focusing our application and deciding what features were important. This also let us design our initial prototype with these features in mind, and get feedback on what to include and what to scrap. For example, the bulletin board was initially  

Finally, starting early and planning ahead was really important. GR4 and GR5 required *a lot* of work, so starting on them early was extremely helpful.