FACTSHEET: How to Buy Eye Tracking for Market Research

December 13, 2010 9:13 am

FACTSHEET: How to Buy Eye Tracking for Market Research

What could possibly go wrong? (Why all eye tracking is not equal)

1. Participant recruitment

It is difficult to overestimate the importance of careful and thorough selection procedures when recruiting for an eye tracking study. In a conventional depth interview or focus group a close, but not ideal fit of participant to the recruitment criteria can often be overlooked. However, the subconscious nature of eye movements makes it very hard for a participant to ‘hide’ behind scenarios and imagined motivations, you need the real thing. It might surprise you to learn the range of behavioural and attitudinal criteria we typically recruit against. Experience has taught us that shared behaviours are far more important than shared demo- and geo-graphics.

2. Research environment & technical quality

Some providers offer remote testing methods for screen based quantitative eye tracking, where users are not required to come in to a lab. There are obvious cost savings associated with this as users can be in different locations. However, several grave issues arise with this approach;

  1. These ‘do it yourself tech’ methods attract panels of very informed and motivated participants which rarely represent target audiences.
  2. The quality of calibration set-up (and the resulting eye tracking data) is dependent on the participant ‘doing-it-themselves’ on their own, non-specialist machines, introducing considerable variances between testing scenarios and data collection methods.
  3. Eye movements occur far more rapidly than you might initially assume. Modern commercial eye tracking units cope with this by sampling at up to 300 times per second. Low quality webcam capture uses a far, far slower sampling rate, and is subject to the stability of the participant’s home office computer. For this reason swathes of vital data is lost and gaze mapping becomes inaccurate and unreliable.
  4. Outputs are limited to heatmap visualisations and, as no skilled moderator is present to probe their behaviour, analysis is therefore left to inference.

In proposals for testing large numbers of participants (over 30-40), be sure to closely scrutinize the testing set-up and recruitment. And if you are going to use large volume remote testing make sure you take account of the above limitations in your analysis.

3. Interpretation and analysis

Heatmaps are flawed – but only in the wrong hands. Inference is the watch word here, and buyers should guard against it. Heatmaps can be immensely useful visual demonstrators of a finding, but should never form the full basis for analysis. Often in eye tracking, supplier outputs leave the buyer with a big deck of heatmaps to interpret on their own. The researcher who undertook the testing is always best placed to interpret heatmaps accurately. Without having first-hand experience of the testing it’s impossible to weed out artefacts of the task, environment or experimental set-up from the true findings or put them in context. Just as you would never draw conclusions from a single depth interview or focus group, so too heatmaps should be respected.

4. Outsourcing

Few large agencies have their own in house eye tracking hardware and software to conduct eye tracking with and so it’s outsourced. This raises the usual issues of communicating the brief and requirements successfully. As mentioned previously, eye tracking is often commoditised, your agency might be using a third party specialist eye tracking supplier. The suppliers may know the right (eye-tracking specific) questions to ask the end client about their aims but the agency won’t necessarily. Ensure your research hits the right spot by asking to involve the eye tracking practitioners directly in all conversations relating to methodology. Similarly an agency researcher can arm themselves with the right information for writing briefs by engaging meaningfully with an eye tracking supplier at the earliest proposal stages.

But don’t forget that it can be of very real benefit to involve a neutral third party; it has the power to settle arguments or concerns and bring fresh thinking to all stakeholders.

5. Vested interests

Market research agencies who may be providing other services (for example supportive surveys or focus groups) alongside eye tracking are under pressure for the outputs from all research methods employed to complement one another, or at least all appear to support key recommendations. Buyers are similarly under pressure from internal stakeholders for results to support past marketing spend, where considerable investment has already taken place, or future spend where key decisions have already been made. It is important in these cases to think carefully ahead of commissioning a project how the potential outcomes might influence such internal constraints or politics. In such situations it might be that the research can be tailored more closely around informing the next discrete business decision, or settling an argument that has been impeding progress, as opposed to more ‘blue sky’ wider lens research.

To Conclude;

When considering purchasing eye tracking it is important to be mindful of the ‘who’s’, ‘where’s’, ‘what’s’ and ‘how’s’. The methodology used in eye tracking is often overshadowed in pre-purchase decisions by the novelty of a powerful technology. In this factsheet and accompanying article we have illustrated why it is even more crucial to consider eye tracking as a methodology, not just considering it a bolt-on, and have outlined why eye tracking may have disappointed you in the past. With this knowledge we hope to have rekindled enthusiasm for the method and enabled interested researchers to gain maximum value from their next experience of eye tracking.