...
Finally, we allow the user to trade for midnights on their watch list from the watch list page because that is why the watch list page exists.
...
Implementation
There were a few important design decisions we made on the implementation level.
Design Decision I: Store All Bid/Asks, But Display Only the Best
We mentioned this earlier. Our original design showed all the bids, along with the usernames of the people who posted them. Now, we chose to display only the best. This made our design easier to implement and better from a usability standpoint. The constant content size allowed for the reuse of the table interface that we'd use for the schedule page. Furthermore, the 'click-to-offer' pop-up interface was much simplified by having a fixed-length table. The usability aspect was discussed earlier; the only extra point we'll make is that anonymity is important on a fair market.
Design Decision II: Display Prices in Tables Similar to How Midnights are Displayed
Although we changed the way MidnightExchange ran (mentioned above) to more accurately reflect a stock exchange, we chose to display the information in a more user-friendly manner than the stock exchange (i.e. in the same way the schedule page displayed assigned midnights). Again, this was easier to implement and had positive usability repercussions. For the majority of users, the display bears more resemblance to the real world, and is thus easier to learn.
Design Decision III: No Ajax
We couldn't figure out how to use Ajax in time, so we didn't. Thus, while the page refreshes when the user himself puts a bid on the market, it doesn't refresh when other users put bids on the market. This is a potentially catastrophic usability issue, since using old market data can very easily lead to inconsistent pricing and misinformed decisions. Clearly, there was no incentive for us to avoid Ajax except that we couldn't figure it out.
Design Decision IV: Yes, PHP and mySQL
We could figure out how to use PHP and mySQL in time, so we did. This meant that we could maintain user data on-line. This is obviously very important if the interface is actually used in real life, since a midnight exchange whose transactions are ephemeral is useless. The implementation was a little tricky, but definitely worth it.
...
Evaluation
We conducted the user test at 8:00 PM on 11 May, 2011 at the fraternity house for which the interface is built. The target user population, brothers of the fraternity, was the same as the that of the environment in which the user tests took place. Our tasks involved putting a bid on the market for an assigned midnight; thus, only users whose usernames appeared on the schedule could be used. We added every user on the schedule to a list (no repetition), and chose three randomly. We asked them if they'd like to be part of the user test, and had a one-hundred percent success rate. The first user was a senior double majoring in math and management. We deem him an expert user, as he had extensive work in finance (internships at top-tier banks). The second user was a senior majoring in political science, and the third a sophomore majoring in 6-1. Neither had financial background, and so we deem them average users. The first user had the least trouble understanding and remembering the definitions of bid/ask.
Process
We followed the following set of steps with each user test.
- The facilitator read the briefing to the user, then gave the user as much time as necessary to familiarize himself with the interface.
- The facilitator read each task sequentially. The facilitator would wait for the user to complete the task before proceeding, providing guidance as requested.
- The observer recorded any vocal feedback provided by the user.
Roles
- Timothy Peng was the facilitator.
- Stephanie Chan and Yang Yang were observers.
Briefing
Hello there! Thanks for agreeing to user test our interface. Our interface attempts to provide stock market-esque functionality to the midnight process. Please go to this URL. Ready? stepcie.scripts.mit.edu/index.php. You will see that, after logging in with your certificate, you'll be able to see what midnights you've been assigned, as well as the schedule, the current market, as well as a watch list. Please note that these are the two definitions that we'll be using in our website. A bid means that you'd be willing for someone to do the labor for you: so, if you had Monday waitings and couldn't make it, you'd want to put a bid out. An ask is how much you'd be willing to sell your labor for; so, if you didn't have anything to do on Thursday and enjoy waitings, you could put an ask out on it. Now, we're going to give you a few tasks to do. Take some time now and familiarize yourself with the interface, you can click and do whatever, then let me know when you're ready. Also, any issues you may have with the interface are not your fault, but the interface's. As you do the tasks, please be as vocal as possible. So speak like you're talking to yourself out loud and we aren't here to hear it.
Demo
There was no demo. We just gave the user as much time as necessary to familiarize himself with the interface. We thought this would be a good idea since it was the most likely thing to happen in the deployment of the website.
Tasks
- See what midnights you've been assigned for the week.
- Put a bid on the market for one of the midnights you've been assigned.
- Put an ask on the market for one of the midnights you've not been assigned.
- Verify that the bid and ask have successfully been entered, and if so, check to see if you have the current best bid and ask.
- Add all Thursday midnights to your watch list, then remove some.
Usability Problems Found
Severity | Description of Problem | Possible Solutions |
|---|---|---|
Major | Batch Removal of Midnights from Watch List. The user found that there was no support for removing more than one midnight at a time from the watch list at a time. | Use checkboxes instead of 'x's. Then have a "clear all" button at the bottom. |
Major | No Log Out. The user wished to, but couldn't figure out how to, log out. | Include a logout button that will log the user out. |
Minor | Quotes Page Aesthetics. The user felt that the quotes page was very information-dense, and suggested color coding bids and asks. | Color code bids and asks. |
Good | Very Nice. The user felt that the interface was very nice. | None. |
Major | Confusion Over Buy/Sell. Despite verbal guidance, the user was still confused by the bid/ask terminology. This happened to two users. | This is a problem we have been trying to deal with ever since the beginning. We don't have a solution yest, other than some sort of help section. It should be a one-time problem for any user, though. |
Major | *Unintuitive Enter Press. * The user pressed the enter key to complete a bid, but instead, the bid was cancelled. We were able to replicate this event, though not with a 100% success rate. | Something fishy going on in the back-end. Need to explicitly say what happens when the enter key is pressed. |
Major | Button Relocation. Sometimes the buttons on the bid ticket pop-up would appear outside of the boundaries of the pop-up. The user was able to realize this and complete the task, but still remarked upon it. | The way we chose to change the content of the pop-up led to this error. While the buttons could move down, they didn't move back up. We need to fix the back-end to compensate. |
Minor | Text Field Relocation. In certain windows, the text box for entering prices on the bid ticket pop-up would appear on the next line, confusing the user. | Increase the bid ticket size. |
Minor | Watch List Ordering. The user noticed that after deleting an item from the watch list, the order of the items in the watch list would change. | We can't figure out the reason for this, but a possible solution is to re-sort the list after deletion of an item. |
Good | Table Formatting. The user liked the table formatting, and also that his username was bolded. | None |
Cosmetic | Table Formatting. The user thought it'd be nice to change the color of the column corresponding to the current day of the week. | Change the back-end to do this, though we feel it is unnecessary. |
Major | No FAQ. The user looked for a FAQ while exploring the website and didn't find one. | Take advantage of the fact that the user might actually want a FAQ and put one in. This can help alleviate the bid/ask distinguishment problem. |
...
Reflection
Our interface was designed for the brothers of ZBT, a very specific audience that is more or less an experienced user in the realm of trading midnights. Yet we found ourselves constantly criticizing the design for being too specific in terms of language. Fundamentally, we were trying to merge the midnight trading concept with that of financial trading, which turned out to be more of a challenge than we had initially anticipated.
Some questions we found asking often was “Would all brothers know what a bid/ask is?”, or “Does buy market and buy limit sound intuitive for brothers with no financial exposure?” As a compromise, we ended up keeping some trading terminology while removing others. Even then, some brothers who did have trading experience confused the notion of buying and selling labor rather than midnights.
...
We tested our interface by appealing to our friends to test it for us. This definitely was a good choice, since they were very diverse and provided useful feedback. Keeping an open mind was one of the greatest contributors to this interface's success.
...
Acknowledgments
Special thanks to Sean Chang and Mason Tang for their input.