Four Common User Testing Mistakes

To the outsider, the idea of user testing might seem like a simple process that requires little preparation. Give the product to the user; let them play with it, push its buttons, and try to break it. If it doesn’t break, great! It’s ready to go to market. While this might be quite revealing and fun to watch, user testing requires much more planning and preparation.

It is a common misconception to think that it is sufficient to throw your product to the lions (users) and just pick up the pieces afterwards to see what works and what doesn’t. There are also a number of common mistakes made during the process itself.

Tasks lacking structure – Throughout the user testing session, each participant must carry out a series of tasks to replicate how the product would be used by users in the real-world. Alternately, the tasks may be specifically selected so the design team can determine the success rate for a certain action or group of actions. In order to gain robust findings that are representative of the task itself, as opposed to any other factor(s), it is essential that the tasks are structured. By structured we mean that each task must be prepared for the user and they must be provided a clear statement of what they are trying to achieve. You must let the user know what the aim is, but this does not mean they should be told how to do the task.

Using leading questions – Questions should never lead the user to a specific response; they must be open and impartial. This means you should not ask test users things like ‘did you like X feature?’, and ‘Would you describe this product as exciting?’. Loaded questions are equally damaging to the usefulness of findings, so asking participants things like ‘Did you fail the task because it was too hard?’, and ‘Do you usually have problems with online payment procedures?’ is a definite no-no.

Intervening – The user should not be lead through the tasks; they should be left to make their way through the tasks with the minimum involvement from the testers as possible. Intervention isn’t necessarily specific to directing the test users; stopping them, speaking to them in the middle of a task, telling them specific aspects of the task before they have begun, and any other disruption can have a detrimental effect on the results.

Talking rather than observing – The main activity for testers in user testing sessions is observation. Findings are dependent on how the user goes about their tasks by themselves, and not with distractions, directions, or instructions from somebody else. Talking will affect the test user. The sessions are not interviews, so there is no need for constant questioning. Cover the questions at the beginning before each task starts, and try not to talk to the user. However, if you can gain further insight by asking why they used a certain approach or probing the user in other ways this can provide the designers with useful qualitative data.

 Header Image: Author/Copyright holder: Usability Geek. Copyright terms and licence: All rights reserved. Img