Research panels in user research – what do you need to consider?

August 2, 2018 5:01 pm

Research panels in user research – what do you need to consider?



In our last article, we considered when it’s appropriate to reuse your research participants. If you’re wanting to run a number of rounds of research or gather insight over a prolonged period of time, then reusing participants is probably appropriate, and a research panel could be an efficient way to gather your participants. However, as something less common in user research, you might be unsure whether this is the right method for you, so in this article, we provide some of the advantages and disadvantages to recruiting a research panel to help you decide.

What is a research panel?

Research panels are made up of people who have been screened into profiles suitable for your research and agreed to take part over an agreed period.

In the past panels have been more common in market research for surveys and in the moment diary studies. However as agile development becomes more common, and testing is required to take place at speed, research panels could allow you to quickly access a suitable group of participants.

But how do you know whether a research panel is the right approach for you? This next section will detail the pros and cons to setting up a research panel to help you decide.

Benefits to research panels

  • Convenience – once your panel has been set up, you’ll be able to quickly and easily access pre-profiled participants to take part in your study. Whether this is remote research, surveys, or usability testing, panels allow you to increase efficiency and flexibility in setting up a research study.
  • Continuous engagement means more opportunities to test – having quick access to participants means as designs or concepts change you can respond by running research with panel participants. You could interview some participants to gain initial insights, then use the same participants when you have a prototype ready to refine the concept.
  • Cost saving – although the upfront cost to create the panel might be greater than a single round of recruitment, you then have a group of participants prepared to take part in future research meaning long-term costs are reduced.
  • Shows a ‘we care’ attitude – customers love to feel like they are contributing towards product development, so involving users throughout development is a good way to encourage customer buy-in.

Disadvantages to research panels

  • Ensuring you have a representative sample – although it’s likely to require some initial hard work when recruiting, it’s important to make sure your panel are diverse and reflect your user base. Also, be mindful that people willing to be on your panel may have a biased interest in the company whether this is good or bad, so ensure you always recruit some neutral or non-engaged users as well to keep your sample representative.
  • Practice bias – there’s a risk users will become knowledgeable about your product and familiar with the research methods. For this reason, you should be wary using research panels if you need fresh user insight, and if you plan to repeat the same methodology such as surveys.
  • Multiple panel participation – don’t always trust that panel respondents are only signed up to your panel. Just like with any research, some users will sign up to every panel for some extra cash, so make sure you keep an eye on your database for any ‘professional’ testers.
  • Consistent feedback – once panel members have expressed a particular attitude or opinion, they are likely to try and stick to it. This means users are likely to be upset if the feature they wanted doesn’t make it to the designs, therefore impacting their overall feelings, and misrepresenting the general population.
  • Frequent panel management – your panel is likely to depreciate in numbers over time, so regular management and replenishment will be required to maintain numbers.

Types of project that may require a research panel

We’ve mentioned that panels can make it quicker and more efficient to get feedback over a prolonged period of time. But here’s a couple of examples of how you could use them:

  • Longitudinal studies for a product that requires extended use, e.g. health apps
  • During the lifecycle of a new product from concept to delivery
  • When using different types of research to triangulate insights
  • Benchmarking studies, to regularly check your top 5 tasks score well against competitors

Takeaway

A research panel could be a useful resource for UX researchers at a number of levels, but it’s important to contemplate the purpose of your research, the methodologies you will be using and the pros and cons of panel research before deciding if this is the best method for you.