Unmoderated remote research tools: 5 things to consider before choosing

Team PFR

17 Sep 2019 — 4 min read

If you keep an eye on our website or LinkedIn account, you may notice that we often talk about the relevance of remote research and unmoderated online tasks. Not only do we offer a specialised recruitment service for unmoderated research and testing, but we also want to ensure you are getting the most out of this research option whether you work with us or not.

Today we’re talking about five important things you need to consider when planning your unmoderated remote research project and, specifically, when choosing the perfect tool to run your sessions or task. According to the this article by the Norman Nielsen Group, “unmoderated sessions require a more complex platform than moderated remote sessions, for which online-meeting software such as GoToMeeting is enough. For unmoderated user testing, you need an application that plays the role of a session facilitator: it guides the participants through the session and records what happens.”

Choosing the right tool to run your remote sessions is essential, as it will determine how the participants, for example, view and feed back on your prototypes and share quality insights. Keep an eye out for the following.

👉 Customise your tasks

Check that you are able to write and provide your own tasks directly to your users, but also that the process of delivering said tasks to the participants during the session is user-friendly and straightforward.

Although most remote research platforms can provide sample tasks, these should ideally only be used as inspiration for your sessions, as they are often too generic. Also make sure that the platform you have chosen doesn’t limit the amount of tasks you can share with the participants: having a time limit is normal in most scenarios, but a good platform will allow you to write as many tasks as you want.

👉 Record video and audio during the session

If being able to see a recording of the session is essential to help you achieve your goals, then this is functionality you must look for ahead of the research or testing. However, don’t forget the importance of being able to listen to the recording if you are running a think-out-loud (TOL) task: “screen recordings are helpful, but, in the absence of an audio recording, it is easy to miss why certain behaviour occurred. […] The audio recording of participants verbalising their actions is essential,” the nngroup.com writes.

Be aware that, if testing with people who are not familiar with the tool you have selected, some users may not feel comfortable with the idea of being filmed or have their screen or audio recorded and that could make the recruitment more challenging.

👉 Test different devices and products

Make sure that the tool you are planning to use allows you to test with multiple devices – web and mobile on desktop, smartphones, tablets – and research into the specifics of what each service offers.

Something else to look for is the ability to upload your own prototypes or sketches to the platform, seamlessly integrating them into the structure of the session.

👉 Access results on the same day…

… or as quickly as possible depending on how niche your audience is and how many participants you need. It’s also worth checking how the platform provides the results: is it just a data dump or can you actually get an organised report with different graphs and charts? Can you see results being updated in real time and feeding into a nicely designed dashboard, which can be accessed by your entire team using different logins?

Before we start thinking about the results, however, we need to consider how you will find the participants, which brings us to the following question.

👉 Does the platform provide user recruitment?

While some tools only provide you with a link to submit to your own participants, other platforms offer extras like user recruitment services. There is also a third scenario: working with a user recruitment agency like People for Research to find the right participants for your sessions.

Using your own customers

Pros: participants are already engaged with your brand, so involving them in the project should be fairly easy. This is also the cheapest recruitment option, as you only need to provide the participants with incentives.

Cons: if you run regular testing or research with this panel, they are likely to be too knowledgeable when it comes to research or your brand. Also, if they are your customers, it’s likely they will go into the sessions with positive or negative assumptions resulting from previous interactions with your company.

Using the platform’s user recruitment services

Pros: users are familiar with the software and methodology and can participate in your session as soon as they have time.

Cons: once again, you risk running sessions with “serial participants”. Most platforms will not tell you how many times users have taken part in research/testing previously, which could be damaging for your project. Also, check the targeting options offered by the platform to make sure they match your needs.

Working with an agency like People for Research

Pros: a specialist recruiter like PFR will tailor the process to your needs and deliver the participants you require according to your brief. If needed, some recruitment agencies can also run feasibility studies ahead of the recruitment and feed back on your screening process. If you are considering working with a recruitment partner, make sure that you select one (like PFR) that has experience in recruiting participants for unmoderated remote research/testing and works according to the Market Research Society’s code of conduct.

Cons: recruitment is likely to be a bit more expensive and lengthy compared to using the platform’s user recruitment services, but there is a guarantee that you will get the users you need – and everyone knows that great participants translate into quality insights.