Voice Composer GR6 - User Testing

Final Design

This is just a bunch of random screenshots; we're missing a lot of stuff.

"Describe the final design of your interface. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, and user testing)."

Implementation

"Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface."

Evaluation

Our users were friends from our living groups who had varying levels of interest in music. While they're all MIT course 6 students and so might not be generally representative, MIT students seem to have a pretty wide range of musical inclination and so weren't a particularly bad sampling of our user population, though they might have been more familiar with the inner workings of programming than most (don't think any of them had programmed on Android before, though). Our app aims to be usable by people with varying levels of music experience and who are probably somewhat younger, creative and tech-savvy.

We conducted user tasks by telling the user the purpose of the app (record songs without having to figure out the musical notation and when you're on the go) and the target audience (should be usable by people without a lot of musical experience).

We mentioned that they should think out loud, and that we were testing the app's usability, not their intelligence.

We had two editing interfaces that we can't decide between. We tested this by having different users test different editing interfaces first, give us their comments on it, then switch and give us their comments on the other one.

We tried to stay silent during observation, only giving tasks, then at the end asking questions about certain things the user did that they didn't explain, or things they may not have discovered.

Tasks

1. Record something
2. Edit
3. Add a harmony line
4. Export

Transcripts

These should be summarized for final submission, but they're here in full for you guys to look at.

User 1

Android User
A capella singer
Starting with old edit interface, then getting commentary on new

1. Record Something

• Records without looking at phone, trusting it to work
• "Is it gonna play back as voice or what?"
• He sings a fast song kind of quietly and the recording is um, is quite different from what he said
• Uses undo to try again.
• "twinkle twinkle..." Sings with the words
• It does better but gets some noise and merges many double notes
• "It has a problem with notes on the same pitch, but I would probably just articulate more if I wanted it to work. That might be hard to actually do."

2. Edit a note

• "How does one edit notes?"
• "Wah!
• Drags pitch
• "Wait, did that even move?" Arrows might be blocking note
• plays to confirm
• "Wait, stop!" Seems to have a bit of difficulty finding stop, but it's the first time he used play after all.
• "I wanted this note!" and moves it to the right place, but clicks back.
• "Oh check box! I didn't see that. I pressed the back button thinking it was okay."
• "Can I split notes?"
• Tries various buttons to see if he can find split
• Trying to use arrows that have x's over them, probably mostly looking at the note he wants to edit and stroking based on that
• Tries left/right stroking.
• "that made it shorter..."
• "Gah, not clear that that's possible"
• accidentally clicked a note trying to move cursor
• "I like the swipe up and down to get the right pitch, cuz you can hear what you want."

3. Record a harmony or something

• "Let's do it in a round."
• accidentally started recording at end rather than start
• accidentally pressed play to record, then surprised no notes show up
• gets right button
• this time looks at screen to record, sometimes. Maybe trying to line it up but gives up
• seems to figure out how tracks work just fine
• "at the beginning I was trying to track it... but it's completely off."
• "now it will sound completely weird"
• presses play
• it does
• "it's hard to record harmony without knowing the original tune."
• "Now that's modern music!"
• "I'd definitely want to hear what I'm recording against, but I don't know how easy that is"
• "you could definitely recommend using headphones, but it's really hard to record more than one track without that."

4. Export song

• "Okay I'm gonna save it as."
• "Uh, Modern twinkle twinkle. save."
• back
• "Tap and hold for options. Ok, let's export it to midi!"
• "whoah I can send it to (all these things)!"
• "Where does wifi go? Does it just go, randomly somewhere over the network? Oh, so it looks like wifi direct. Ok I don't know what wifi direct is."
• "I'll email it to myself! Look I'm in alan's contact list."
• "Now let's ask myself if I got a midi! Well, not yet."
• App crashed awhile after export.

Did you notice you can change instruments?

• "No. Oh, I didn't try this [settings button]. Aha! All of these guys look familiar"

What did you think of note editing interface?

• swiping all directions
• "I like the up and down, that's for sure. The moving the note left to right seems sort of difficult. For one it seems to move in big chunks. The x's mean you can't move it any further, right? Obviously you can't move a note to zero. I guess you have a maximum note length? That doesn't seem so reasonable. Or it just doesn't, wait, I guess it's not clear to me whether that's a max note length or, oh, I see, it goes until it butts the next note in the same voice. That makes mores sense. Yeah. I guess it's hard to deal with, like say I wanted to move a note forward a little bit in time, that would be kinda difficult with this interface. At the same time it is also useful to stretch and shrink notes, I think it's hard to get all of that in there."

Try (DD's) interface. I tell him my phone takes longer to record things.

• Means to hit record
• "Oh that was play."
• "hellooooooo helllooooooooooo!"
• seems to record okay, gets random noise an octave above.
• "awo!"
• has a bit of a hard time pressing the note.
• "it doesn't have this whole save making changes to a note, or stop making changes to a note, you just click away. I see."
• "It's a liiiitle strange. You have to click directly on a note and then drag it to move it, whereas in alan's you can click anywhere to drag it. What's the most confusing about this up and down part is that I have to drag it, but it doesn't follow my finger exactly. Okay let's try left and right."
• "Okay so that moves it left and right, oh I like that."
• "and that makes it bigger, okay"
• "I like the left and right. I definitely like the left and right better. You can do that without changing the width of the note, and you can change the width. That's cool, sweet."
• "It does seem to be easier to do a lot of different notes."
• "I definitely noticed on Alan's you can just click undo if you don't want your change... But since you have undo up here, that's probably..."

Compare them.

• I think that.. hmm. (thinks for quite awhile)

• (Playing with alan's) "is this actually centered on the note? hmm I think I would kind of like Alan's better except with left and right functionality more like yours. You know with move [in all directions] in the middle" (lots of gesticulation)

• "it's true getting from note to note is much faster in this one (DD's)"

User 2

Using DD's interface first.
iPhone user
Casual composer, doesn't often finish songs

1. Record a song

• Clicks on a bunch a buttons
• Hit record
• "okay alright...." 
• Sings "twinkle twinkle", except with doooooos, looking at it.
• Phone picks up pitches, kind of.

2. Edit

• taps a note. Trying different things on it.
• "So what I wanted to do was split a note in half. I guess that might be part of your phone thing but yeah."
• tries to tap end arrows with no effect
• deletes note
• "I was trying to click note between those notes (a row of 3 min length notes) and then I tried to tap away and then I clicked another note and it was hard to get away because all these things appeared and all I wanted was that note."
• "I tried to play, but I meant play from start." (uses to start)
• "I don't know if to start without a just play from start makes sense. I guess you could use it for record."
• plays back and stops to edit things. Uses dragging left to right a lot. Change pitch very carefully by tapping on note.
• "Is there like a go to next note button?"
• "It's hard to drag the pitch of small notes."
• Notices from red edge that she needs to shorten next note to lengthen this.
• "It's really hard to click small notes."
• "oops. Stoop!" Has hard time finding/hitting stop. Taps everywhere.
• "I was totally editing a note and I pressed play and it jumped to a nother place, what was up with that?" (cursor was in another place. Maybe cursor should follow editing)
• "wait the cursor isn't on my note?"
• doesn't notice you can stroke anywhere in the vertical
• "oh shoot I deleted. Undo. Yeaaah"
• "Darn I want another note. Oh look another note!" Take little extra note and drags it over
• "I really like how you can just drag a note around"
• "It turns out it's really easy to hit the trashcan cuz sometimes you have notes right over the trashcan you wanna hit. Esp if you have a descending line."
• "I think my song gets faster. (as in the lengths get shorter at end.) I really wish I could extend a note and have all notes move over, or have a mass move button, rather than moving them all one at a time."
• "I keep meaning to play from start but it just plays from whereever."
• finishes
• "Yeah! alright."

3. Record another track

• "...I don't know any harmonies to twinkle little star."
• (presses play rather than record again)
• (Singing) "pleeeeease record my song!" 
• After several tries of clicking play, realizes it's not record.
• Realizes record doesn't do playback
• "Awh! I don't get to hear my song now? how am I supposed to harmonize with myself!"
• "Damn I can't click small notes!!"
• "Agh I just dragged my note to the side and it disappeared."
• bug. Various terrible things happen.
• she re-records
• Noise in the octave (she sings one note but it alternates between octaves.
• "I'm not sure how it got those ones"
• "wait can I no longer zoom in?" (she's at max zoom"

4. Export the song

• "Too bad, I'm naming it."
• "the star tried to twiiiiiiiiinkle. done. save. Yeah!"
• settings: "What's this? Oh cool!"
• "Wait how do I get outta here? ahhhh!" (not an android user)
• giggles while playing track in helicopter
• "wait what was I supposed to do now?" (re-instructed)
• "tap and hold for more options."
• chooses export.
• it crashes.

What do you think of this editing interface?

• "I really like how you can just move notes around. The trashcan kept getting in the way. I think it'd be nice to have a next/prev note function instead of playing the whole thing cuz I know where the pitches are sposed to be to hear what's wrong. And it's extremely hard to click small notes, holy shit."

Try Alan's interface

• Katfang tries to click play button to record again, despite already making this mistake and it being disabled.
• "Oh my god. it's different."
• "oops I pressed back instead of check"
• "Oh my god I have to click small notes again. Did I do it? I can't tell!" (arrows covering note, seems to be a problem with early notes)
• "I gave up trying to remember what the right note was while editing. I want to go one note at a time, cuz I don't actually know what note (pitch) I want, but I know the interval, but there's no next note and you have to go out every time."
• Still tries to tap arrows to change note even though she knows it's swipe
• About to use back
• "Oh shoot no. Check."
• "I want to cut out the beginning part, where there are no notes, cuz I'm like what the hell am I doing k I'll start now."
• "I'm still trying to tap these arrows even though I know they don't do anything unless I swipe over them." (shortening and lengthening)
• keeps trying to press back instead of confirm. 
• "Stop stoooop!" tapping everywhere. "Oh stop."
• "Moving notes is bizarre and awkward." (messing with arrows to scoot notes over)
• "Nope": tries to play from wrong place agin.
• (trying to get note lengths right) "One thing that kind of bugs me is that I can't tell how long notes are relative to each other. Like a quarter note or something. Like if one is twice as long as another. I can only tell how long a note is within 5 seconds."
• "Also I have no music sense so I keep having to replay." (loses the note she wants while editing because it plays)
• "what is this note supposed to be??" (tries many times to get pitch)
• "Darn this was the actual note it recorded and that was just a messup note. Darn! Darn! I'm so angry!"

Impressions on this interface

• "My biggest thing was that I couldn't move notes. I just want to take this chunk of notes and be like you move that way! There's this big gap and I don't want it. I didn't randomly delete notes like the other one.
• I had a really hard time remembering the back button vs confirm, I kept clickng back note."
• "This interface especially could have a next and prev note. I dunno if that's good, if that's too many buttons. I kept having to click a note and pressing back and playing and clicking a note and pressing back and playing. You know cuz intervals. This would totally be fixed with a next/prev button."
• "turns out it's really hard to type potter. Oh well he'll just have to be pretty."

Compare interfaces.

• "I liked this (DD's) interface better, because I could move notes and I could move between notes more easily. LIke on the other one I'd have to click the back button, or the save button. But accidentally deleted a bunch of notes, but there's an undo button I guess."
• "I really wish I could add/split notes. On twinkle twinkle all the tones that there are two of them there was only one of them."
• phone crashes after export again.

Could you tell what's disabled?

• She was trying to press disabled buttons often, like play when she wanted record on an empty song.
• "I can't tell what's disabled." (alan's interface) "This one is... better." (DD's) "I can't tell if they're the ones I want to press or not press, but they're different."
• (Another person, watching): "My expectation is colored is enabled and non colored is disabled. Looks like Only record is enabled."
• "It might be more reasonable if record were white, itunes"

Discovered usability issues

Reflection

"Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations."