User Analysis

We have identified three demographics who could benefit from using a mobile device to look up products while shopping: technological innovators, power users, and casual users. In this section, we discuss their goals and challenges, and explain how our proposed application could benefit them. Finally, to substantiate our claims, we interviewed three subjects about how mobile product identification software could assist them in their lives. Our results are intriguing and suggest that our proposed mobile phone application can have a significant impact.

Technological Innovators

Overview: A compelling demographic are technological innovators and researchers who are likely to be early adopters of technology. People in this demographic typically use sites like Amazon.com and Newegg.com to find products and read their reviews. These users have grown accustomed to quick access to reviews and product information while shopping. In other words, they desire efficiency. Consequently, we believe this demographic will find a mobile lookup application useful since they want to replicate the online experience while in stores. 

User Study: We located one research subject who was currently looking to purchase a laptop. We asked him how he makes purchasing decisions. He responded that he typically uses a variety of metrics such as price, appearance, functionality, and reviews. In order to find reputable reviews while at stores, he uses web-services such as Amazon to type in queries for products he is on the verge of purchasing. Despite its slow speed, he prefers to use a text-based query input method. He argued that picture-based product lookup applications from Amazon or Google do not work well because he found them difficult to use. He often wanted to specify multiple objects. In other words, he found the user interface inadequate for his needs because he was unable to easily specify the products he wanted to lookup. More explicitly, he pointed out that product ratings are relative and that the most important measurement is a relative cue. He wanted to be able to compare two products, not just see absolute ratings. 

Proficient Users

Overview: Another demographic are the power users of technology. They adopt working technology into their lives, but do not necessarily use it to its full potential. While such users are aware of online services like Amazon.com, they are comfortable shopping without them. Consequently, in order to foster rapid adoption of technology into their lives, products must have high learnability. Once they learn how to use the software, they will continue to use it.

User Study: We brought one research subject to a furniture store and asked her to buy a sofa. Since this subject identified herself as a proficient but not savvy user of technology, she immediately located a salesperson and asked questions about the couch. The thought of using her iPhone to lookup information did not occur to her until we asked her about it. Upon explicit prompting, she stated she would normally lookup products on her personal computer at home before going to the store. Upon this reminder, she also stated that she wanted to go home to lookup sofas before making this purchase. She did not want to use an iPhone application because she often found them both a) rude to use in a store, and, more importantly, b) worried that they would be biased. In this vein, she felt that if she looked it up on Amazon, they would make their own products look better and preferred to shop using an independent source. 

Causal Users

Overview: The final demographic we consider are users who only use a casual amount of technology in their life. These users are typically characterized by senior citizens. While they rarely use technology, when they do they need it to work reliably and minimize mistakes. Indeed, these users need safety and simplicity. They want to just see information about one product. 

User Study: After finding a more senior research subject (i.e., postdoc), we brought them to a coat store and asked them to determine which coat to purchase. Again, this user went straight to the salesperson. When asked about computerized lookup methods, they stated they normally do not use them because they do not want to cumbersomely type in product information. When we explained the idea of our project, they agreed it would be very useful, as long as it worked well. They thought they would use it because it only required a few clicks to work.

Summary

Our preliminary experiments indicate that while a mobile product look service would be useful, most user interfaces are slow and awkward. We believe that by developing an intuitive user interface for mobile image annotation and recognition that we can build a useful product lookup service.

Task Analysis

Inspired by our experiences after interviewing subjects, we identified three primary tasks for a mobile phone application: quickly identify products of interest, efficiently compare them on price and reviews, and clearly show results to the user. In this section, we discuss these tasks for each user demographic.

How do users lookup products?

Goal: The primary task for users is to lookup products using a mobile phone application. Our user studies reveal that all demographics desire an efficient interface for identifying products in a store.

Subtasks: To accomplish this, users must first take a photo of the object they wish to identify. Next, they annotate the object by drawing a box around the object of interest. After deploying modern computer vision algorithms, the product is identified from a database. The user is then shown a screen with the product information.

Preconditions: User is in a store.

Question 1: What is the best way for a user to annotate an object? Bounding box, ellipse, freehand, tapping, enclosing hull?

Question 2: What is the best way to display results to the user? Popup or new screen?

How do users compare products?

Goal: Users (especially experts) want a system to quickly compare products with a side-by-side chart. 

Subtasks: Users again take a photos of the objects they want product information for. In order to specify which objects to compare, they must annotate multiple objects. After recognizing both objects, a comparison screen is shown.

Preconditions: User is in a store and has two products he is considering.

Question 3: What is the best way for a user to annotate multiple objects? 

Question 4: What is the best way to display comparisons to users on a mobile screen? Column wise or row wise?

How do users get unbiased results?

Goal: Users need to trust that their information is coming from an unbiased source. 

Subtasks: After our application recognizes the products, the user should be able to choose which sources are trustworthy for this particular product.

Preconditions: User is in a store, but may have had bad experiences with reviews before.

Question 5: How do you instill confidence in users that the product results are unbiased?

Question 6: How do you allow users to easily change which sources we report on?