Dinner and a little usability testing

On monday four friends were invited along for an informal usability test of remoteless. The pay was dinner made by me and some wine left over from my mom. Pictures were mentioned several times, but non were taken…so the picture clipped from the recipe will have to do.

Before we go on I would like to thank Karo, Ivar, Tom Erik, Tor-Arne and Ola for their time and insight.

What we learned

First of all we can confirm that even a small and informal usability test like we did can be very useful. This is especially true in combination with all the feedback we have gotten from our beta-testers. The beta-tester have given feedback on usability, but one can’t change an app so it fits everyones personal tastes. The usability gave us an ability to separate the “I  would have liked it better this way” from the true usability issues.

We uncovered two such usability issues:

• when selecting a song it was expected to start, and the play button was not found
• in the known connection screen it was hard to understand that the big button represented a possible connection and that it had to be pressed

On the positive side non of the testers had problems adding a song to favorites when asked to “make it easy to play later”, and finding a song by searching was easy peasy. Overall they were happy with the app and the dinner, even though the sauce looked kind of strange.

Playing a song

Beta testers have told us and test subjects showed us that they expected a song to play immediately after being selected. I agree that would be nice, but functionality like adding the song to favorites or finding an artist by searching for his/her hit-single would then not be possible. As soon as the song is played on spotify remoteless do not know the exact identity of the song, so these things need to be available before the song is played.

When the initial reaction of the song not playing was over, the test subjects still found it hard to play a song. I think the major mistake was the change in where the user had to look. They would tap, tap, tap mostly  in the upper area of the screen and did not expect to look around for a play button (ours was placed along the bottom). Play is such a core function I don’t think users expect to spend any time figuring it out.

In the new solution play track appears in the same main area as the song selection takes place. Hopefully this will help users in their quest to play a song.

Known connections screen

Before the test started we had connected the computer and remoteless together. The connection therefore appeared in the list over known connections, but the test subjects needed to select this connection to be able to go ahead with their tasks.

All of the them had problems at this stage, they looked at the screen with one big button with the computer’s name on it and kept on looking. They did manage it in the end, but this is not the first impression we would like for the app.
Below I list a set of factors I think contributed to this issue, it could be one alone or a combination:

1. The computer’s name was not known to them.
2. The button was not very iPhonish and didn’t really invite an action.
3. There was only one connection available  making it hard to understand they had to make a choice.

In the new solution we solve 2 and 3 by making it look like an iPhone list, “forcing” the user to make a selection, even when there is only one connection to choose from. Factor 1 we can’t do much about, but as owner of both computer and iPhone this should not be to hard…

Planning

Even with a small app like remoteless it is impossible to test every feature and every possible sequence a user will go through these features. One has to prioritize and we choose;

• Feature 1:  Search and play a song
• Feature 2: Connecting the iPhone to the computer running spotify
• Feature 3: Add to favorites

The reasoning behind these choices were that search and play will by far be the most used feature. Connecting will not be used as much, but we all know how important first impression are. Most likely the two first features wouldn’t take that long to complete, so we needed something more. We landed on “add to favorites” since this could let us retest searching for a song, and we ourself find it useful.

It is harder than one would think making good task for the test subjects, or at least I think so. Especially when you look at how simple they ended up. The task description has to make sure the test subjects uses the features you want to test, but without giving away clues they would not have in a real situation.

I did not want to use the words search or browse since that was what the button said. The same thing goes for favorites, how could we test if the use of the word “favorite” was right. In a real situation a person might think of saving or bookmarking, and I didn’t want to force any specific word. I solved this by writing “make it simple for yourself to play it later.”

Underneath are the tasks as they were presented to the test subjects (originally in norwegian). Task 1 tests feature 1 and feature 2 since the iPhone was not already connected to the computer and task 2 test feature 3.

Task 1

• Pick a song you like
• Play it for us by using Remoteless on the iPhone

Task 2

• Pick another song
• Find the song
• Do NOT play it now, but make it simple for yourself to play it later

Doing

After dinner we invited a participant one at the time into another room with Anders and I. There we had already installed the spotify helper on the computer present and installed remoteless on the iPhone. Since these were our friends the tone was casual, but we still made sure to say “We are testing the product and not you”. We also made sure to explain that they needed to read through the whole text of the task and that we needed them to think out loud.

Conclusion

All in all, this usability test was a great and useful event. Prior to the test we had our doubts and our opinions on what parts that worked and what parts that did not work. By doing this test we suddenly had an empirical based list of stuff that actually was bad, and stuff that maybe wasn’t perfect, but worked exactly as everybody expected. After all the subjects had finished testing, the picture of what had to be done, was perfectly clear. For all future projects I will conduct this kind of testing. It’s easy, it’s revealing, it’s in some sense eyeopening, and the results are incredible useful. We just identified and removed our app’s two weakest points! It would take years to get to this understanding by ourself trough books and such.