...
Another comment in heuristic evaluation was that users wanted to be able to open multiple pieces of paper at once, as depicted in Figure 4. In response to this feedback, we decided that if a user clicks a task or bucket from a drop-down menu, it would open a new paper. This makes the action more obvious, and lets users view multiple tasks at once, if desired. After noticing that some users were confused about the difference between Bucket Papers and Task Papers, we attempted to make this more clear. We put an image of a bucket on each Bucket paper, and a check-box on Task papers. We added a link back to the Bucket paper from each Task paper, making it easier for a user to both remember what Bucket the task is in, and get back to that Bucket. These changes were all based on the heuristic evaluations.
Figure 5: Zoom-in List of Tasks on a 'Bucket Paper'
...
There are a couple of remaining implementation problems which may affect the usability of our interface. Unfortunately, we never quite got to work on the "Alert" functionality. Having all of the affordances of an alert system without an actual alert system is obviously a usability drawback of our interface, but it highlights some important not-yet implemented functionality that will improve user experience in the future. In addition, our system does not handle real-time collaboration properly yet - whenever a client makes a change it gets written to the database on the server immediately, however clients only pull changes every couple of minutes. This means that one user may over-write another user's change if they are editing at the same time. This problem never came up during testing, though, since we only tested our system with one user at a time.
Evaluation
- Describe how you conducted your user test.
- Describe how you found your users and how representative they are of your target user population (but don't identify your users by name).
- Describe how the users were briefed
- and what tasks they performed;
- if you did a demo for them as part of your briefing, justify that decision.
- List the usability problems you found, and discuss how you might solve them.
User tests on the final BucketList website were conducted with 5 different users, from varying demographics.
Representative Users
We made sure that each of the users who participated in final user testing use to-do lists on a regular basis and have a need to keep track of group projects, thus they all fit perfectly into our target user population. Since BucketList targets such a wide range of users, we also made sure our test subjects came from varying demographics with respect to age, location, gender, education, and technological expertise, so as not to limit ourselves to other MIT students who think too similarly to us. Many users in our evaluation were MIT student (by far the easiest to find), but we also included professionals and high-school students (found by speaking with friends and family) who we believe would benefit from BucketList.
Briefing
The following short briefing was given to each test subject before we began the evaluation:
"BucketList is a shared task-management application, allowing users to easily create group to-do lists.
Once you have an account, you can create "Buckets" - categories to put your tasks in. You can then add tasks to your existing buckets. This application is different than most task-management applications you've probably used in the past, because you can also share buckets with your friends. So, for example, you might create a "Final Group Assignment" bucket, which you will share with the rest of your group. You can then assign each person in the group individual tasks, like "do research" or "outline presentation", and monitor your progress, as a group, towards completing the project. You can also write and share notes about each task, to make sure you're all on the same page, and you can even send alerts to make sure your friends don't miss important notes.
We are going to describe a scenario for you, and ask you to use BucketList to help you accomplish your goals. This shouldn't take more than 10-15 minutes. Do you have any questions before we start?"
We decided not to do a demo as part of the briefing, as we really wanted to see if users would be able to figure out how to accomplish key tasks, so we didn't want to show them first. Our application is simple enough that users can figure it out themselves, without needing to see others use it first.
Tasks
Instead of giving users specific tasks to accomplish, we decided to give them a scenario with less guidance, and observer how they would use BucketList to help organize themselves. This allowed us to see which features are more important, and which features users don't attempt to use - something that would not be obvious if we specifically prompted them to use each feature. We structured the scenario in such a way that it would require the use of the entire system.
The following instructions were given to users:
Please imagine that you are a junior in college, taking 4 classes, and editor of the student newspaper. You already have a BucketList account with some ongoing projects.
User tests on the final BucketList website were conducted with 5 different users, from varying demographics.
Representative Users
We made sure that each of the users who participated in final user testing use to-do lists on a regular basis and have a need to keep track of group projects, thus they all fit perfectly into our target user population. Since BucketList targets such a wide range of users, we also made sure our test subjects came from varying demographics with respect to age, location, gender, education, and technological expertise, so as not to limit ourselves to other MIT students who think too similarly to us. Many users in our evaluation were MIT student (by far the easiest to find), but we also included professionals and high-school students (found by speaking with friends and family) who we believe would benefit from BucketList.
Briefing
The following short briefing was given to each test subject before we began the evaluation:
"BucketList is a shared task-management application, allowing users to easily create group to-do lists.
Once you have an account, you can create "Buckets" - categories to put your tasks in. You can then add tasks to your existing buckets. This application is different than most task-management applications you've probably used in the past, because you can also share buckets with your friends. So, for example, you might create a "Final Group Assignment" bucket, which you will share with the rest of your group. You can then assign each person in the group individual tasks, like "do research" or "outline presentation", and monitor your progress, as a group, towards completing the project. You can also write and share notes about each task, to make sure you're all on the same page, and you can even send alerts to make sure your friends don't miss important notes.
We are going to describe a scenario for you, and ask you to use BucketList to help you accomplish your goals. This shouldn't take more than 10-15 minutes. Do you have any questions before we start?"
We decided not to do a demo as part of the briefing, as we really wanted to see if users would be able to figure out how to accomplish key tasks, so we didn't want to show them first. Our application is simple enough that users can figure it out themselves, without needing to see others use it first.
Tasks
Instead of giving users specific tasks to accomplish, we decided to give them a scenario with less guidance, and observer how they would use BucketList to help organize themselves. This allowed us to see which features are more important, and which features users don't attempt to use - something that would not be obvious if we specifically prompted them to use each feature. We structured the scenario in such a way that it would require the use of the entire system.
The following instructions were given to users:
Please imagine that you are a junior in college, taking 4 classes, and editor of the student newspaper. You already have a BucketList account with Buckets for 'Newspaper' and 'French Literature Project'. There are some notes on your board reminding you about the presentation for French Literature.
- You were just assigned a final project in your Computer Systems class. You need to write a report for next Friday, and give a presentation the following Monday.
- Your teacher reminds you that the presentation cannot be longer than 5 minutes, and must include a visual aid.
- You just submitted the proposal for the group final project in your French
- You were just assigned a final project in your Computer Systems class. You need to write a report for next Friday, and give a presentation the following Monday.
- Your teacher reminds you that the presentation cannot be longer than 5 minutes, and must include a visual aid.
- You just submitted the proposal for the group final project in your French Literature class.
- You remember that Tracey needs to return the book your group borrowed to the library.
- You
- You decide that there should be a newspaper article about MIT 150 this week, and assign Jason and Matt to write it. It needs to be completed by Wednesday night.
- You realize that Pat went home for the week, and reassign the article his article on
student in 6.813, and your group decided to switch the focus of your project, so you have to redo GR1. You were also just assigned GR2. And in your busy schedule, you somehow also just got a UROP.
Task 1 - Add note to GR1, reminding your group to update the wiki.
Task 2 - Remove Bayo's note from the bulletin, then place your new note in the center.
Task 3 - You've completed GR1! Now add a task called GR2 to your 6.813 bucket, and add the appropriate collaborators.
Task 4 - Add a UROP bucket.
...
- decide Fatima will take over your responsibility to prepare the presentation.
- You check what articles are scheduled to be written for this Thursday's newspaper publication.
- You decide that there should be a newspaper article about MIT 150 this week, and assign Jason and Matt to write it. It needs to be completed by Wednesday night.
- You plan what you will do tomorrow, and then go to sleep.
Usability Issues
Reflection
- Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations.
...