26 March 2013

UX: which test is best?

Tags:UX, Strategy, Innovation

The concept of user testing has been around for decades but for a long time there was only one way to do it. Thanks to advances in technology, we can now choose from a whole range of methods when it comes to testing websites, because a one-size fits all approach just doesn't work. Let's take a look at some of the options on hand:

  • Moderated usability testing
  • Un-moderated, remote usability testing
  • Eye-tracking
  • A/B testing (split testing)

Moderated usability testing

In this type of testing, people are observed as they attempt to complete specific tasks on a site (or prototype) in a controlled environment. A facilitator is present to ask follow-up questions that allow in-depth exploration of improvement areas and discussion of alternative design concepts.

When should you use it?

This approach is best suited to multifaceted sites or complex tasks that don't have a structured sequence of steps, and when looking to explore potential solutions during the design process or test the waters during the product design cycle.

What are the benefits?

  • Helps unearth new requirements through follow-up questions.
  • Allows in-depth exploration of user expectations, perceptions and feedback, as well as user suggestions for site improvements.
  • Enables other activities such as interviews, participative design and eye tracking to be conducted within the same session.

Un-moderated, remote usability testing

During remote testing, people are asked to attempt to complete specific tasks on a site (or prototype) in their own time and in the comfort of their home. Remote testing can be conducted with or without a facilitator being present; however, for large-scale testing un-moderated testing is usually conducted to increase the reach of representative users in terms of numbers and geography - this is one of its key benefits.

When should you use it?

We think remote testing is best used when you have a lot of things to test - preferences, strengths, issues and so on. It's also perfect for sites that need to access a large number of users or users in remote locations.

What are the benefits?

  • Gives you a solid foundation of quantitative data.
  • Can identify priorities for subsequent research such as workshops or interviews.
  • Gives you access to a much larger sampling of users across a wider geographic area.


Eye-tracking involves special equipment that tracks the eye movements of a user and can trace the path their eyes (and attention) take when scanning a webpage. This is usually used in conjunction with moderated usability testing. When combined with verbal user feedback from moderated testing, this sort of tracking can give a lot of context around what design elements are most appealing to users.

When should you use it?

Eye-tracking is recommended when you want indisputable, objective and compelling visuals describing user behaviour when viewing the key, graphically designed pages of your website, such as your homepage and key landing pages.

What are the benefits?

  • Compelling visual findings such as eye-tracking gaze paths and heat maps that are easy for non-usability experts to understand.
  • Offers unique insights about first glance, search patterns and failed search.
  • Can help shape design refinements that would otherwise be based on instinct.

A/B testing (split testing)

A/B testing is exactly what it sounds like - comparing two different but similar designs to see which one fulfils your goals. This can be anything from click-through rate to conversions to dwell time - it depends entirely on your specific needs and requirements. In its most basic form, half of the visitors to a site page are shown design version A while the other half are shown design version B.

When should you use it?

We recommend A/B testing for clients who are looking to measure the effectiveness of a design or messaging strategy rather than information architecture.

What are the benefits?

  • Measures actual user behaviour in real-world conditions
  • Captures more data than other testing through the sheer volume of website visitors
  • Gives objective data that can help you make design decisions.

Back to the question of the day - which test is best?

Drumroll please... there is no answer. Each test has its own pros and cons so it's more about choosing the method that's most suited to your website and your goals. Factors such as budget, timeframe, project life-cycle and access to representative users will determine what test is best for you. Of course, that's what our UX team is there for.

Give us a shout to find out which test is best for you

Leave Us a Comment

Other stuff you'll love

All news articles

Contact us

We look forward to hearing from you

We take your privacy seriously
Back to the top of this page