If Carlsberg did market research recruitment….

February 24, 2012 5:32 pm

If Carlsberg did market research recruitment….

Of course not all recruitment for website usability is equal, and it may be having an effect on the validity of your research outputs. We’re pretty passionate about quality recruitment at SimpleUsability. Here’s a few big reasons why;

  1. Speaking to the correct audience is an utterly fundamental prerequisite for meaningful research.
  2. If you have a limited budget every participant counts, you can’t afford off-spec recruits or no shows.
  3. We believe poor recruitment practices are widespread in the market research industry, but that new tools and techniques can help.
  4. Our methodology, eye tracking, is so truthful it’s nigh impossible to fake. We need the real deal, nothing less will do.

Our methodology highlights bad recruitment

It is difficult to overestimate the importance of careful, thorough and honest selection procedures when recruiting for an eye tracking study. In a conventional depth interview or focus group a close, if not ideal fit can often go unnoticed. However, because our eyes move largely outside of our conscious control it’s very hard for a participant to simulate imagined motivations in scenarios they wouldn’t naturally find themselves in. If they’re not right their eyes simply won’t scan and react to a site in the same way a genuine users’ would and this will be apparent from the post-task retrospective interview.

Read more about how we ask why in eye tracking sessions with a retrospective interview.

This is why we conduct all our recruitment in-house

*At SimpleUsability the researcher is also the moderator for your research, this is not always the case in a research agency model where a professional moderator may be employed.

In house, under control

You’ll notice two things about SimpleUsability that sets it apart from the typical Research Agency;

  • There are fewer steps in the recruitment chain overall.
  • All of them happen in-house and under our control.

There’s more than just a business economics case for this, it’s about the recruiters’ proximity to your research. Our researcher works particularly closely with their colleague the recruiter, they share an office, and so can bounce ideas back and forth, check suitability etc. This ultimately helps maintain focus on the key aims of your project, ensuring recruitment comes in on spec and on time. Our recruiter is also present on the day of testing, unlike many professional recruiters they aren’t simply ‘signed off’ at this point, promoting higher quality recruitment overall.

Minimising Chinese Whispers

In a traditional agency set up the information is often second or even third hand by the time it reaches the recruiter, meaning that any nuances and emphases of the brief have been lost, or worse, distorted. At SimpleUsability both the probability of miscommunication is lower and any issues are more likely to be identified sooner and remedied before the expensive phase of testing takes place.

Removing fraud with online recruitment

A lot of market research recruitment, particularly for qualitative groups and interviews continues to be done in-street or by telephone based recruiters. Asking questions verbally, a recruiter can often impart tone and persuasion or bend around a screener to ‘help’ someone fulfil certain criteria. After all they’re financially incentivised to meet strict and ambitious quotas, often within tight deadlines.

By contrast at SimpleUsability we use online forums, social media and a variety of online resources to invite the public to join our panel for the chance to earn £40. New recruits answer an in-depth profiling questionnaire, administered online. We then hand pick respondents from this panel for each project based on individual suitability. There are several benefits to this approach versus a traditional in-street screener;

  • Questions are asked neutrally, without inter-recruiter variation, nothing is skipped.
  • Respondents don’t know which answers result in selection so no incentive to tell us untruths.
  • The breadth of questions asked means that users rarely know the exact topic or the sponsor of the research so they don’t arrive ‘prepared’.
  • It ensures computer literate attendees.

The breadth of questions in our database allows us to know a fair amount about the respondent’s lifestyles, including gauges of disposable income, online purchase activity etc. Occasionally an additional screener is required to recruit users with specific experiences and behaviours.

We only use each participant once per research style

We find that users learn a considerable amount about their own behaviour when they complete an eye tracking session with us. They become aware of the eye tracking technology and how we use it to the point it would influence future behaviour in a second session. This is why we only ever use the same person once for the same kind of research method, e.g. they will only take part in one eye tracked website evaluation but may subsequently be invited for store based shopping research.

We don’t like ‘groupies’

We work hard to screen out so-called professional research attendees because they are often overly reflective, opinionated or unrealistic in their creative suggestions, adversely affecting your research outcomes. Past attendance also results in primed behaviours, giving overly reflective answers, as opposed to unbiased reactions to questioning. We routinely screen our database for multiple registrations using the same email, phone numbers, addresses for de-duplication to check respondents aren’t creating aliases. After seeing how we test one to one, users are very unlikely to try to attend again under false identities.

We only put people in boxes when it’s useful

Traditional social grading systems that classify individuals on the basis of occupation and education (A,B,C1,C2,D,E) have prevailed in market research for the last 50 years. However our experience has taught us that they rarely translate into meaningful identifiers of target customers. Modern consumers come in all shapes and sizes and job title rarely predicts how someone chooses to spend their disposable income. We believe it’s far more important to identify a shared motivation and behavioural trait for each audience of interest. We take a wider lens view, collecting large amounts of behavioural and attitudinal data that better describe an individuals’ lifestyle and consumer activity so that you can be sure of meeting your true audience.

Personality comes in many flavours ….

Many market research attendees have often been selected on the basis that they are ‘social, articulate and creative’. But being ‘chatty’ is less crucial for an eye tracking session than a focus group, where your only data outputs are respondents comments. We also have their eye tracking evidence, this in turn gives us a powerful stimulus to show to, and elicit meaningful responses from quieter respondents. We recruit for computer literacy, not personality or chattiness. We then reach all personality types, your audience is equally represented and ‘creative’ suggestions don’t become pie in the sky.

We have a high attendance rate

We’re quite proud of our attendance record at SimpleUsability and rarely experience no-shows. Here’s a few reasons why;

  • Our incentives are at the more competitive end of what is typically paid for a focus group attendee, and for less time out of their day.
  • We’ve developed an attentive system of pre-session contact with participants so that we learn about any potential issues whilst we can still recruit a replacement.
  • Contact also generates a strong social motivation to attend.
  • It’s a less stressful prospect to attend a one-to-one chat and browse of the internet than it is to talk in front of a roomful of your peers.

Striving for Carlsberg status

So you see recruitment brings with it unique challenges, and definitely isn’t a part of your research where quality can be overlooked. Over the years we have honed and streamlined our recruitment processes over and above just bringing them in-house. We are constantly developing means of combating the traditional recruitment downfalls and we review the effectiveness of every recruitment project. And if you were wondering how it feels to be in the participant’s shoes then just have a look below at what some of our past recruits have had to say about us.

We’re always looking for ways to make it better and better for both clients and participants. So if you have a tricky brief you’d like us to take a look at, or would like to know more about the issues raised here please contact us and we’ll be happy to help.

Participant Testimonials

The research was very easy, it involved looking at a website and at the end of the session I was asked for my thoughts on certain aspects of the pages… what was good about the page, what could be improved etc. All in all, a friendly bunch of people who made me feel at ease and paid me well for simply giving my opinion. Oh and free parking too…what could be better?!

A very positive experience all round with your company. I signed up online, which I recall was quick and simple. Subsequently I have had several communications from you advising of possible opportunities for which I might be interested.

Very helpful information sent about the day with instructions and directions and also a car parking space. A very warm welcome on arrival, with a cup of coffee and clear instructions about what was to happen. The market researcher was very personable but professional. The exercise was interesting and I felt confident with what I was doing. Clearly the pre-screening made sure that I was a suitable candidate for this particular project. The time flew by. Clearly I would not have volunteered my time without the promise of the £40, but I thoroughly enjoyed the hour with SimpleUsability.

Attending the market research, I was made very comfortable, waiting in the foyer until my appointment time arrived. I was greeted once again by another person, who took me to the room being used. She was very polite and my experience was made very easy with her relaxed and helpful manner. I enjoyed the experience, and was well rewarded for my time.

My experience was very positive, a nice venue with a decent coffee to drink whilst waiting and free safe parking. The interview itself was very straight forward and clear instructions as to what was required. Yourself [hostess/recruiter] and your colleague [research moderator] were both polite and friendly and the whole experience was quite pleasant.

I recently attended a market research session at the offices of SimpleUsability. The market researchers were very friendly, explained what was required of me and completely put me at ease. Throughout the session I was using a website and the market researcher sat at the side of me and talked me through what I needed to do. I thoroughly enjoyed the session and would definitely recommend it to other people.

I enjoyed the experience, I thought it was fun and interesting. The setting was comfortable and I was made to feel relaxed and at ease with clear instructions on what to do – the experience flew by!

Staff were friendly on arrival and throughout the session. The staff gave easy to follow instructions and made you feel very relaxed whilst doing the session. They fully explained every step of the session clearly and explained any tasks which they wanted you to carry out.