What are Usability Tests?
As an app is being designed, teams must verify that it is easy to use and fulfills the purpose of the original idea —hence why it’s essential to test with users.
Usability tests are performed to get valuable user feedback so that corrections and improvements may be made1. Tests are performed in labs or spaces designed specifically for this, where a user accomplishes tasks previously established by a moderator in charge of leading the test while a group of observers takes notes.
What These Tests are Not
Usability tests should not be confused with focus groups, which have a completely different objective in getting design opinions without taking into account utility.
When It’s Time to Test
Ideally, tests should be performed before launching an app, and better yet, before programming the code. Testing in the early stages, and frequently, can help with new ideas before publishing, when there is still time to implement changes. The advantage of this is that it saves time and money and makes possible the inclusion of changes that can have a long-lasting impact on the project.
Although generally considered a one-time event, tests can be performed as often as necessary, even after an app has been launched. Tests allow us to continuously gather information and make adjustments accordingly.
Mobile Testing
When a usability test is performed with a mobile device, users don’t necessarily have to test a completely finished app. As mentioned, it’s advisable to do testing in the early stages of the design and development process, meaning that a prototype is often sufficient.
Mobile apps are related to a certain context of use, something difficult to replicate unless a field test is done. It’s important to consider that using the app in a room is probably not the most realistic scenario, but it’s clearly better than not performing a test at all. In such cases, use conditions should be simulated as closely as possible. For example, testing should be done with an average internet connection speed, not high-speed wireless.
The phone should also resemble reality as closely as possible. Ideally, that means a test should be performed directly on a user’s mobile device. If that can’t be arranged, a device that is as similar as possible should be used. Before testing, the user should be provided a few guidelines as to how the device works so that the smartphone’s characteristics don’t interfere and the conclusions reached are related exclusively to the app. Likewise, if the app tested is, for example, for Android, the user should be a frequent user of that operating system.
When testing, you also need to pay attention to one element specific to mobile devices: touch gestures. For this, filming devices may be installed that, besides recording where and what the user taps, can also register what gestures are used to complete the actions.
Guerrilla Testing
Guerrilla tests are an agile and economic alternative to traditional usability tests, which can prove costly with the inclusion of specialized professionals, numerous users and expensive workspaces. With a more informal format and fewer users, guerilla tests can be quite efficient and are especially useful when a project’s deadline is approaching.
A guerrilla test consists of gathering a determined number of users to try the app. The main objective is watching how they behave and getting information to correct mistakes.
How It Works
To perform the test, one member of the team should be designated as the leader responsible for the test. For this role, a great amount of patience is essential, as is the ability to listen.
Ideally, at least one other person also takes part in the guerilla test to accompany the leader and observe. The observer will be in charge of taking notes on the usability problems that users encounter, without active participation.
The complete development of the test can be divided into three very distinct stages: planning, execution and analysis.
Planning
At this stage, the foundations are laid for the test before meeting with users. The goal is determined —such as testing the entire app or only some parts— and the guidelines needed to make the most of the test are defined.
At this stage of the process, participants are chosen. Ideally, five to eight participants should be selected—the amount of people needed to detect the most common usability issues of the app. Having more participants does not ensure that more problems will be found.
Besides test users, the place where the test will be done has to be determined. Preferably, it should be a quiet place with no external distractions or interruptions.
Which of the two possible tests participants will carry out must be decided. The first alternative consists of simply showing the users the app and observing them to determine if they understand what it does and how it works. The second kind of test involves assigning them a specific task and observing them as they perform that task with the help of the app.
Based on what has been decided, a few questions can be defined to help obtain information about user interactions. These questions should be neutral and adapted to their levels of knowledge. It’s best to break the ice with easy questions to avoid intimidating participants at the start.
Once all the decisions mentioned above have been made, a small pilot test can be performed with team members to get everything prepped.
Execution
Usability tests are performed with one participant at a time. Before beginning the test, do a brief introduction to explain the test and how long it will last. The volunteer should be invited to participate actively throughout the process and be encouraged to express all of his or her opinions, concerns and issues, even when they’re not critical. It’s important that the participant feels that his or her opinions are valuable and useful for the project, and that they will not affect the feelings of the team.
During the execution of the test, the moderator should be more aware of the user’s behavior than of what he or she actually says. It’s not uncommon for participants to unintentionally lie or try to please the moderator to avoid coming off poorly.
When the goal of the test is to perform a series of tasks with the app, the moderator should ask the user to perform them without any assistance. Behavior can’t be biased or influenced. Orientation is only allowed in situations when it is absolutely necessary and as an opportunity to ask users what they think the expected result of an action may be.
The moderator might not be the only person that asks questions during the test. Participants may have doubts, and it should be clear to them that their questions will be answered once the test is over.
When this stage is finished, it’s nice to give each participant a small reward for having completed the test. Even though it can be mentioned beforehand, it’s recommended not to give them this final retribution before the test so as not to condition their answers.
Analysis
Once the tests are over, and shortly thereafter, the moderator and observers should meet to comment on the results of the test and review all notes taken.
This is the stage where problems are identified and solutions are proposed immediately. Once issues are rectified, another test can be performed to determine how effective the solutions are and identify any potential new conflictive solutions that may have appeared.
Other Ways of Gathering Information
Guerrilla tests are not the only way of getting information from users. There are other ways that are faster —in fact, some require just a few seconds.
Dogfooding
There is an urban legend that says that because of a TV advertisement for dog food, a Microsoft executive sent an email to employees telling them to taste their own food —in other words, use their own products.
Since then, dogfooding has become a fast and easy way to test one’s own software. It’s as simple as using the app you’re working on to gain a profound understanding of how it works.
Over time, dogfooding can become a standard practice for detecting technical and conceptual mistakes. These kinds of problems are often difficult to detect in design or code programming and easier to pinpoint when abstracted and experienced in a more relaxed environment.
Yes, dogfooding is easy and comfortable, but it doesn’t replace user testing. More than anything, it’s useful for gaining confidence in your own work.
The Five-Second Test
The first impression of a design provides a lot of valuable information, especially with respect to content hierarchies. The five-second test is, essentially, explaining a concrete situation to a user and showing him or her the app’s design for only five seconds immediately after. Next, the participant is asked to speak about everything he or she remembers seeing.
User reactions after a quick glance help to establish what is more eye-catching. It also helps one to determine whether users’ impressions are in line with what the team has originally conceived.
There are a number of tools available online for these types of tests, such as the website FiveSecondTest.