Combining quantitative online surveys with qualitative methods for more effective user research
There is an ongoing debate around whether quantitative and qualitative methods are better. Quantitative research is good for finding out ‘How many’ or ‘How often,’ as it measures the incidence of findings in a given sample size. The findings are quantifiable and help make informed decisions about the impact of future developments. However quantitative research does not allow us to probe and understand why. Qualitative research on the other hand, uses a wide range of methods to gain insight into underlying user needs and behaviour which is not available in quantitative research. This can provide a sound base for further decision making. However, the in-depth approach of qualitative research limits us to small sample sizes meaning results are not quantifiable.
So does anyone fancy a quant/qual sandwich? Combining quantitative methods with qualitative methods is a good way to overcome the drawbacks and obtain richer, meatier, findings to highlight key themes and steer business decisions. This article will discuss effective ways to combine the two methods – using quantitative online surveys with qualitative user research.
Online surveys familiar example of quantitative research and an effective tool for collecting quantitative data as:
- They allow researchers to reach the target audience across a wider geographical range.
- They allow us to reach larger sample sizes, as they can be done remotely.
- They allow us to ask closed questions that generate statistical data that can be collected and analysed quickly.
Here are some ways in which quantitative and qualitative research can be combined in order to obtain ‘meatier’ findings:
1. Quantitative research can be followed by qualitative research to explore the findings further. For example, an initial survey allows you to find out how many customers thought a website met their expectations and draws out key themes. Qualitative research can then gain insight into why areas of the website were particularly user friendly or did not meet their expectations.
2. Alternatively, it could work on the flipside, qualitative research can be followed by quantitative research. This time in-depth user interviews are conducted to find out some key findings about a particular interface and then an online survey could be conducted to gain a wider sample size and generate quantifiable findings. We have used this approach at SimpleUsability on multiple occasions as it helped us to include a larger sample size and access a wider target audience, which may otherwise have been difficult to reach.
“We needed similar results from spending time on a one to one basis with customers, but on a larger scale and across multiple locations. Conducting one to one interviews first with a qualitative sample size, and then adapting the survey to allow more customers to give different types of feedback, gave us the quantity of quality insight that the team needed to move forward. ” – Fiona Harwood, UX Director
“Conducting an online survey allowed us to reach a large group of people in different parts of the world, in a short space of time. This quantitative feedback supported the qualitative comments gathered from our user testing.” – Katharine Johnson, UX Practitioner
3. Another strength of using quantitative data in combination with qualitative research is that we can use surveys to build on detail from longitudinal studies and are able to create powerful visualisations, such as graphs to highlight key trends. We’ve found from our experience that adding qualitative feedback from user testing as well as visualisations provides the project team with the evidence they require to enable key business changes.
“Visualisations created from quantitative data provide a powerful illustration of findings, which can be shared easily within a team. Backing up the numbers with qualitative insight adds a human element to the graphs as well as strengthening the evidence and informing change.” – Amy Martindale, Lead UX Practitioner
4. Often quantitative and qualitative methods are used in parallel and can be a powerful way to benchmark against previous research. At SimpleUsability we have previously used a quantitative survey to deliver quantitative feedback and provide comparative results with a previous survey.
“In a recent project, we included questions from a previous study along with new questions. This allowed us to compare changes in customer attitudes year-on-year as well as gathering new information about customer website use and expectations.” – Lucy Buykx, Senior UX Practitioner
Our work at SimpleUsability, using quantitative and qualitative methods in combination results in richer findings, bigger sample sizes and quantifiable results, which can steer development and future improvements. It is important to first understand the project goals and research objectives when combining quantitative and qualitative methods and consider which way round is the most beneficial for what you are trying to find out.