Skullweb - User Testing

Design

Overview

The main purpose of SkullWeb is to be an in-house website for the brothers of Phi Kappa Sigma. SkullWeb was designed to offer two main components for its users, house jobs and room reservations. The main design decision we adopted for the task of presenting both of these components was the use of tabs. After the initial sign in of a user they would be brought to the House Job page. At the top they could then click on the Room Reservation tab. This allowed for a quick and fundamentally easy movement between components.

Another main feature that we changed was the overall color scheme of the website. Initially the site used a gray, white, and black lettered color display. However, the heuristic evaluations presented multiple problems with this including, diminished sight, blending of colors, and an overall uninspired look. We originally went with these design colors because the site was not meant to have flashy colors but was meant to be an effective working and observing environment. In our final design we decided to keep these goals in mind but to allow for a more engaging color scheme. The site now follows the colors of the fraternity Phi Kappa Sigma, black and old gold.

Login Page and User Profile

The Login Page simply consists of the Phi Kappa Sigma crest and a login form, making the page as simple as possible.

Clicking on the user's name in the main navigation tab bar takes you to the edit profile page, where the user can change his account information. All fields except for the last two password fields are pre-filled since these are unlikely to change and to speed up the editing process.

House Job Page

Within the House Job tab we chose to present the house jobs as a list for each week of a month. To navigate through the weeks a simple "Previous, This, Next Week" link system was implemented. A list of the house jobs was the most efficient way to present all jobs to the user. There are usually around 10 to 15 jobs a week, and to present this on a calendar would have created a cluttered interface, while one job at a time would have been very inefficient to see all jobs for a week. Within this list any job that belonged to the user would have a specific symbol and color to depict that it was completed or not.

To help obey Fitt's Law, the whole row that pertained to a user job could be clicked on the change the status of that job. Along with this change in status was also a change in color (green for complete and red for incomplete) and a change in symbol. The final design change we had for this tab was the new placement of the symbol on a user house job.

Originally positioned on the right side of the described job, the symbol has now been placed on the left side of the user's name. As described in one of our heuristic evaluations this now provides a consistent position for all symbols, possibly creating an easer user experience. 

Room Reservation Page

The Room Reservation tab has greatly changed from our paper prototype design. Instead of having a persistent form, used to reserve a room, the form is now only visible after the user has clicked on the "Rerserve a Room" button.

Another design decision that was made based on feedback from the heuristic evaluations was the position of the list of reserved rooms and the form described above. The final design now has the list above the form, which creates a more efficient user experience. Users are now able to log on and quickly view the upcoming events without first encountering an empty form and being required to scroll down to view the list.

The final design changes that this tab had were adding a few functionalities to the reserved rooms list. As a user hovered over an event the row would now change color providing the affordance that it could be selected. Similar to the user house job row, each reserved room row obeyed Fitt's Law and could be clicked anywhere on to display all information.

Administration Page

The final page of SkullWeb is not a page that the majority of users will see or interact with, it is the Administration Page. For this reason, the Admin tab was designed with efficiency and control as the main goals. While still provding a clean and basic design, this tab includes many more links and options that the previous pages did not present.

Although built with an administrator in mind, the tab is straightforward and uses real world language to appeal to any and all users. The tab allows a user to add, delete, or edit any feature in the site including: user creation and deletion, room reservation editing, house job assigning, and status of each job.

Implementation

Skullweb was implemented using a combination of Ruby on Rails, HTML, jQuery/JavaScript, and additional Rails plugins. Rails served as the framework for the site, providing a strongly interconnected back and front-end.

Our work on the back-end consisted of implementing the functionality behind the house job management and room reservation systems. This consisted of writing Ruby code to dynamically change the UI based on the database and to perform data validation on user-submitted data. We used a Sqlite3 database and designed a relational schema to properly manage and persistently store the site data. We implemented a fully-functional user login and session authentication system using Authlogic and an extensive Admin Panel using Typus. The design decision to use a Rails framework was motivated by its ease of use, as well as the clearly-delineated framework it provided, which helped us mentally and physically separate our work on the user interface and the back-end. JavaScript and jQuery were used for AJAX server calls.

Work on the front-end user interface chiefly consisted of changing the HTML structure and CSS layout of the site. Rails allowed us to separate common site-wide elements to a single layout, helping us divide the user interface into separate components, useful for designing and debugging. This also spurred our decision to move the links to the user profile and to logout to the main navigation tab bar, in order to maintain visual consistency and to save space. We spent a significant amount of time working on the CSS, individually modifying elements on the site in order to improve site-wide visual consistency and clarity. We made extensive use of jQuery and JavaScript for animation and small visual touches to improve the feel of the site (e.g "Reserve a Room" button). We also spent some time in image editing programs to work on custom images for our site, such as the newly-added banner crest.

During implementation, we tested and retested our site, checking for usability problems and bugs, following the iterative pattern of design. Overall, we felt that using Rails helped us abstract our design into separate parts, helping us focus on different individual parts with ease.

Evaluation

The user tests were conducted by giving the uesrs a brief overview of the system by explaining to them its intended goals. We then asked the users to navigate the interface in order to familiarize themselves with it before we assigned them specific tasks. The tasks we assigned were very simlilar to the ones we used in our paper prototyping. Since our interface has a limited number of use cases, we wanted to optimize the design for these common scenarios instead of having an overly-complicated UI.

We selected three different users from our fraternity to test the system. Since our system is designed specifically for the brothers of PKS, it didn't make sense to test it on anyone else. We did, however, decided to pick users carefully based on their computer expertise and experience with managing house jobs using other methods. We chose two users that had never managed house jobs before and one who had previously done all management by hand. One of the users who had never managed house jobs before was a programmer while the other didn't use his computer for much other than email. The user who previously managed house jobs was a power users, but not a programmer. We thought picking users of varying skill levels and domain experience would help us identify different types of problems.

We briefed each user by describing the basic fucntions of the interface. Since the users already know what house jobs are and how room reservations work, we didn't need to do any type of demonstration. Instead, we hoped that the interface would be intuitive enough for them to figure out on their own. In fact, this is a necessary requirement for our UI, since it would be infeasible to hold a training session or demonstration each year as new brothers need to use the system.

We first had them login as regular users with predefined usernames and passwords. We asked them to find their house jobs, complete them, and tells us what their past house jobs were. From there we asked them to identify other users jobs and had them switch to the room reservation tab. In this tab, we gave them a predefined event time to schedule for, had them manage an event time conflict, and then schedule an event of their own. Once completed, we asked the users to manage their account to change their password and logout. We had the user take a 30-second break while we changed their admin status in the database. Upon completion, we had them log back in with their new password. We told them that they were now administrators and asked them to manage several settings in the admin panel. Since we hoped that most things would be intuitive, we didn't give them any specific guidance; instead, we observed what they tried to do. After letting them explore the panel on their own, we asked them to use the filters and search (if they hadn't done so already). Once we were satisfied that they had explored all the interfaces features, we had them log out and give us their overall impression.

Usability Issues

User 1 (programmer)

1. Confirmation of password change appears in a confusing location (visibility, usability)

This user expected the notification that the password had been changed to appear in a different location. He had used many web applications before and was used to certain stylistic conventions. However, this problem isn't serious and can be easily fixed.

User 2 (power user, house job administration experience)

1. The admin panel has unnecessary filters (learnability)

This user had managed house jobs by hand previously and was confused by the number of filters that were present on the admin panel. He thought they were unnecessary and wanted some of them to be removed. This can also be fixed by modifying the typus template that we used to produce the administration panel. Right now it provides standard options based on the database structure, and we can customize certain pages.

User 3 (novice user)

1. Saving and event with errors clears submission form (usability)

This error was noticed by all users of the system. It was something that we didn't catch initially since we had been entering events with correct details when testing. However, errors like we instructed our users to make will be fairly common, so this is a major design improvement that we need to make. 

Reflection

Over the course of the iterative design process, we were able to focus on improving the overall design of the site, by focusing on the individual components. At the start of the project, we had a much larger project in mind, but we decided to tone it down to be able to work on making the most integral components of the site as good as possible. Prototyping our design in several different ways helped us to form a better picture of the site in our heads before we even sat down to code, which significantly cut down on the actual coding time. The computer prototype in particular allowed us to be able to focus a lot of our attention on the back-end of the site, and to make several small iterative changes to the prototype to improve the final product. User testing and heuristic evaluation really helped us focus on improving particular parts of the user interface, also cutting down time working on the front-end. Furthermore, because of the time saved, we were able to mock up, design, and completely implement a brand new feature to the site (the Admin Panel).

Our overall site layout really improved as the result of iterative design and prototyping it, both on paper and on the computer. We worked a lot on making it as clean and functional as possible, mocking it up on paper at first. When we had it implemented, we made iterative changes to it by using Firebug to change both the HTML and CSS of the site in real-time, allowing us to decide which layout we liked the best. Over the course of changes, we also consulted other brothers as to which design they considered the cleanest and simplest. This helped us avoid designs that we really liked, but were at times confusing to actual future users.

Overall, the iterative design process really helped us to make the right decisions as to the look, feel, and functionality of SkullWeb, simultaneously helping us avoid confusing and complicated designs. It helped us keep the site incredibly simple with a very strong feature set.