Validating Ideas Fast With Remote User Research

In his UXBristol 2016 workshop, Adam Babajee-Pycroft shared what he’s learnt about validating ideas quickly, having executed hundreds of hours of remote research on a variety of projects.

In this session summary we capture his key insights about the process and advice to other UX researchers looking to try this method, including how to write tasks for self-moderated testing, and how to find the right participants.

“Remote user research is doing exactly the same user research we have always done, but taking advantage of the technology available to do it remotely.”
 

Adam was initially skeptical about remote user testing, but when circumstances forced him to try it, he became a convert.

Some of the advantages of remote user testing include:

  • It is quicker to arrange
  • You can run multiple tests at the same time
  • You can remove the social pressure/temptation to intervene when someone is struggling with a task, which means you get the unedited truth!
  • You can recruit participants from anywhere in the world
  • You can see users in their natural habitat, but remove the discomfort of being observed

Some of the disadvantages include:

  • It is harder to build up relationships with test participants (but video could help to bridge that gap)
  • You can’t see a test subject’s wider context
  • Connection speeds vary

 
Adam described some of the tools you can use to administer remote user testing, including survey tools, such as Google Forms and Survey Monkey, and interview tools, such as Skype, which are often free and widely available. He also outlined the key features of some of the more quantitive tools available, including Optimal Workshop, Notable and User Zoom. These offer facilities such as card sorting, tree testing and click testing, allow you to upload a design and ask users to annotate, and allow you to carry out memory and mood tests with remote users.

Adam discussed some of the key things to consider when judging different services on the market. One of the key considerations is speed: most services allow you to get results in 1-2 hours using self moderated remote user testing. However, this is usually only the case for basic demographics. If you need a specific user group, results will not be as fast. Many of these platforms allow you to do your own recruitment, under different subscription models.

Finally, Adam recommended considering what users can or have to install in order to participate, as this may be a barrier. Many of the tools offer native Android and iOS apps.
 

Adam provided some key tips for creating effective remote user tests, including:

  • Decide what you’re testing – set some objectives and write the tasks against those objectives.
  • Be succinct – it is is really important that tasks can fit on a post it note. Users need to be able to remember and understand the task in its entirety
  • Avoid ambiguity – you won’t be there to explain it!
  • Don’t use leading questions
  • Establish a rhythm – if anything is going to behave differently in a task, make it clear, as if anything unexpected happens the participant may assume something is wrong with the test and may give up

He also highlighted some common issues, in particular stressing that some of the specialist platforms don’t limit the number of tests a participant can do, which means participants can become too familiar with the process and not representative of normal users.
 

Exercise

Worksop participants were asked to prepare a set of tasks for a self moderated remote user test lasting no more than five minutes.

Each table was assigned a website which they could ask a user to test on any device to hand. Adam selected a random person from each table to act as the user to carry out the test. Each table had the opportunity to revise their tasks and test them again on a different user before showcasing the best tasks to the rest of the group.

 

Take Aways

  1. Try it at least once to overcome preconceptions
  2. Use the same budgets to do more rounds of research – don’t use this as an excuse to spend less
  3. Ensure instructions are easily understood
  4. Use the right tools for the job
  5. Be rigorous in your analysis
  6. Use free trials to demonstrate the benefits and get management buy in
  7. Hold yourself to the same standards of proof that you would require for traditional research methods
  8. Show your working out – most of the self moderated solutions offer video evidence which you can show to the people you’re trying to influence
  9. Remember your research objectives

About Adam Babajee-Pycroft

Adam Babajee-Pycroft is UX Director and Co-Founder at Natural Interaction. Fuelled by coffee and heavy metal he’s spent the last decade delivering successful User Experience projects across Automotive, Financial Services and eCommerce. Adam enjoys Lean UX projects, where we validate that we’re building the right thing whilst getting it done.

Follow Adam on Twitter: @adam_ux

Posted 15th July 2016 in: Event by Kirsty Pitkin