3 minutes reading time
If you’re looking for guidance on how to recruit research participants, the internet is your oyster. However, best practice advice on whether you should reuse participants is rare. We have many years of recruiting research participants at CDS and are happy to share what we’ve learnt.
Should we reuse participants?
Reusing participants is something that often gets brought up with clients. “Should we invite the same users in for each round of research so that they can see our product improving” or “should we make sure we recruit a fresh set of eyes?”
There are a lot of questions to ask when considering whether to reuse participants or recruit a fresh sample, and the outcome is likely to depend on the methodology you are using or the type of product you are testing. So first, let’s have a look at the general principles that apply to re-using research participants.
When you should not reuse your research participants:
If the participant has participated in a research session within the last six months.
You want to avoid developing ‘professional participants’ who anticipate the problems they think you are expecting them to find and lose the fresh perspective needed. Nielson Norman Group and MRS recommend that participants not be used more than twice a year. Waiting six months before inviting a participant in again prevents them from becoming research masters.
For iterative tests focusing on the ease of use or first exposure to the system.
If a user has previously engaged with an earlier version of the site, system, or app, they will likely remember things from the session. These participants will not approach the new designs with a fresh perspective. Avoid reusing participants where this is the case.
Research on a similar topic.
Even if the system itself is different, sometimes users may recall their previous experiences if the research is on a similar topic. For example, if users have previously completed research for home insurance, we would avoid inviting them in for another session relating to any insurance.
If the participant has previously been excluded from a study.
Whether this is during a session or the analysis, participants sometimes need to be excluded due to their dishonesty during the recruitment process, limited feedback during the session, or inappropriate responses. It is always important to record which users had to be excluded to avoid making the mistake of inviting them again.
If the participant has previously been unreliable.
We’ve all had times when our circumstances change unexpectedly, and we must cancel our plans. However, if a participant makes a habit of cancelling their sessions last minute, it may be best to avoid using them in the future.
When you can consider reusing your research participants:
When researching an internal system.
In certain cases, you may be limited to a particular user group. For example, to evaluate an internal system, your user base is restricted to people who work for the company and use that system. When you can’t physically find alternative users, you will have to go back to the same participants to gain feedback on new designs.
When your research is targeted towards a restricted target audience.
You may want to see how your product addresses the needs of a particular target audience. For example, when testing for accessibility your audience is already restricted, so when coupled with limited time and resources, it may be difficult to avoid reusing participants. However, where possible, we would still recommend avoiding anyone recruited within the last six months.
For studies on the same system that do not focus on ease of learning.
If you aim to see how a system works over extended use or how iterative designs can improve it over time, you might want to reuse the same participants. However, to ensure learnability is not affecting the research findings, continue to include a mix of new participants in your sample.
Based on these general principles, we recommend avoiding reusing participants when research takes place more than every six months, or for systems requiring fresh user insight. However, as we’ve highlighted, there are exceptions where reusing participants is appropriate, such as if you are dealing with a restricted user base or wish to carry out a longitudinal study. Ultimately, consider these guidelines on a project-by-project basis to decide whether to reuse participants or recruit new participants for each round of research.
At CDS, we have an expert team of participant recruiters and outreach specialists who support our internal research teams by finding the users they need for research projects. If you are looking to recruit participants for user research, please get in touch with our friendly team.
Author
