When should I test? How to fit research into your design process
Are you concerned about conducting research too far in advance that it might make the insight become invalid? Or gathering insight prior to launch but then not having the resource to implement the changes?
These are the kinds of concerns we hear regularly from clients when user research is considered. The budget will only allow for one round of research, so the timing of that research becomes a worry.
But the reality is, there is no bad time to test. All too often, research is reactive rather than pro-active. Once a problem occurs, alarm bells ring and one big piece of research is commissioned. However, in order to adapt to changes in behaviour and pre-empt problems, we recommend breaking research down and gathering customer insight little and often. Below we look at the entire design process from discovery through to post-launch and suggest how research could be implemented at each stage of the process, as well as addressing some key concerns.
This is usually the point at which you’ve decided you want to understand the user’s needs. You may have a product in mind and want to understand the user needs it could address. Or you may have a product that’s live and want to understand more about how it’s used and could be improved. Some of the key things you might want to find out at this stage are:
- How does your current or proposed product or service solve the needs of its users?
- Learn about your users and identify groups amongst them
- Validate assumptions you have about users and their needs
To do this there are a series of research methods you can use. These can be broken down into research with users and desk research which require varying levels of time and budget:
Research with users
- In-depth and contextual interviews – requires no development, and allows you to understand user wants and needs
- Competitor usability testing – understand the user experience on competitor sites to raise opportunities and problems to avoid
- Personas – based on research, personas allow you to identify the profiles of your users and represent their needs, motivations and frustrations
- User journey mapping – involves identifying user goals and visualising the process users go through to reach this goal. Again, requires no development or functionality
- Competitor analysis – identifying what your competitors are offering, how your current product compares, or understanding gaps in the market
But unfortunately, investing in research early in the design process is still a concern for some clients. A common question we receive is:
What if the research is too far in advance of launch that findings become obsolete?
There’s really no such thing as research being too far in advance. Only by testing early will you be able to validate your concept and understand the changes that need to be made. These will then help to shape your program of work and understand where additional research should be gathered to support development. Even at launch, you might have come on a long way since that initial discovery research, but you wouldn’t have got to where you are without it.
By this point, you have a clear set of goals and any assumptions have been validated. With a concept in mind and some evidence to support the user needs, stage 2 involves considering content, getting some lo-fidelity designs in front of users, and reiterating those designs until refined.
Some of the research methods you may use at this stage include:
- Card sorting and tree testing – focus on information architecture, and allows users to input into the organisation and structure of your content
- Designing and testing prototypes of increasing fidelity – involves the iterative development of prototypes (from lo-fidelity wireframes, to fully functioning prototypes), to refine designs
- Targeted research focussing on single features – full development of a single feature in order to observe user interaction and refine accordingly
By this stage, the prospect of moving forward can seem quite daunting. You have lots of elements that need addressing but you’re unsure how to prioritise, and how much to chew off at once. A concern we often hear is:
What if there’s too much to change before the next sprint? How do I prioritise what we design and test first?
Although daunting, the key at this stage is to plan ahead. Consider what needs addressing and order these issues based on how likely they are to impact the user experience. From this, you’ll have a good idea what to focus on first. If you’re still unsure, discuss your ideas with your team, or even with us – based on the results of research or an expert review, we can help you to prioritise what to work on.
As the final stage before your new product goes live, this is when you should be testing the full user journey to uncover any last-minute tweaks or bug fixes.
At this stage your research methods may include:
- Full user journey testing – in-depth usability sessions allow you to assess the full flow, and identify sticking points
- Eye tracking – used in usability sessions to observe natural interaction with the interface, helping you to understand how users navigate the page and make journey choices
This should identify some of the high priority fixes that may need to take place before launch, but also some less severe issues which can be built into a backlog and addressed post-launch. With that in mind a key concern at this stage is:
What if I don’t have the development resources to make recommended changes before launch?
If research has been taking place throughout the development process, then, in theory, there should only be minor changes needed at this stage. But we know things don’t always run as smoothly as anticipated, so in these situations, it’s a matter of prioritising the work. After delivering a project at SimpleUsability we often work alongside clients to make recommendations and help to prioritise how and when these recommendations should be implemented. This way you can get those essential problems fixed and begin building a backlog of issues to work through post-launch.
Now your product is live, but there’s likely to be some teething issues that need addressing, and you might want to start working through that backlog. Your product should be a working progress, so conducting research little and often can be useful to check everything is working as it should and test any new changes that might be going live.
For this reason, the most appropriate research methods at this stage are:
- Regular usability testing – to check the end to end user journey is still working effectively, and explore use with different user groups and functionality
- Eye tracking – to observe natural interaction with the interface, understand how it’s navigated, and what is missed
- Expert review – evaluating your interface to understand likely problems encountered by users
- Competitor analysis and benchmarking – identifying what your competitors are offering, and understanding how your current product compares
With regular use of these methods, your product should be constantly improving. However, post-launch research often raises concerns relating to cost when the live product seems to be functioning perfectly fine. For example:
I know my site is working now, so why should I test if it’s not broken?
‘If it ain’t broke, don’t fix it’. Unfortunately, this is something we usually hear when research is believed to be wasteful or unnecessary. In reality, no one should be waiting until something is broken to fix it. With competition and innovation at its prime, you should constantly be pushing to make your product better. For example, competitors may be implementing changes, and the industry may be shifting, so you don’t want to be at the back of the pack.
We can’t ignore internal concerns and constraints, but getting your peers on board and conducting research little and often is an essential way to stay in touch with users and improve your product. It’s important that research is not just reactive when a problem needs to be addressed as this will require a lot of hard work and will be expensive to fix. Instead, research should be a routine part of your design strategy, implemented across the discovery, iteration, pre-launch, and post-launch stages. It doesn’t have to be days of testing with a fully functioning website, just little and often should provide the feedback you need for success!