Ipsos x PFR: recruiting coffee lovers and hosting in-person product testing

It's not every day that we get to both recruit participants for product testing and also host the sessions in our own viewing facility, User Viewing. So, when global market research organisation Ipsos challenged us to find and host 14 Bristol-based coffee lovers to test a new coffee machine, we had to accept.

This project happened back in October 2023 and, with the client being based in the USA, it required some help from the team at Make Human, who took on the responsibility of running the testing sessions at our facility.

The mission

The PFR team was tasked with recruiting a group of diverse people based in Bristol or surrounding areas to test a new kitchen appliance made by a well-known brand in the coffee game.

The client wanted to get insights from 14 people who were regular coffee drinkers – consuming it several times a week –, with a mix of preferences: from people who preferred black coffee to people who liked it better with milk, not forgetting people who liked iced coffee seasonally, and more. Ipsos also wanted to see a good mix of genders and ethnicity, where possible, as well as a mix of income levels. Some non-negotiables included the participants being involved in decision-making in their household, with some already owning a coffee machine and others planning to buy a new one.

The plan was to host 60-minute in-person sessions where the participants would be asked to test a coffee machine prototype. The session included two devices: one for physical testing with pods and one for aesthetic review purposes.

The PFR approach

Although this was a relatively easy audience to target, recruiting people for in-person research in a post-COVID world has proven to be a challenge at times. Participants with an awareness of user research are now used to hearing about remote research by default, so ensuring people are aware they are applying to take part in in-person research and double checking their availability is a must.

As usual, we followed our tried and tested two-stage screening process, first checking the suitability of our pool of participants via an online questionnaire and then confirming their information and availability on the phone, as well as probing deeper into their profile and coffee consumption tendencies and preferences.

To avoid leading the participants simply by asking if they liked coffee or not, we built our screener around their daily habits and routines, including consumption of hot drinks on a weekly basis. Other suitability requirements included checking both personal and household income and purchasing habits.

While the participant recruitment was happening, our viewing facility team was in constant communication with the client to ensure everything was going according to planned.

Preparing to host the sessions

As part of the project, PFR was tasked with handling the logistics of receiving the devices, setting up the room according to the client's instructions and then send the devices back to the client in the United States.

In order to prepare the facility for the research, our studio team shared some questions with the client specifically targeting physical product testing. This is the information we collected:

• What type of research is the client running?
• How many participants will be taking part?
• How many observers will be in attendance?
• Do the researchers require live streaming?
• What can the participants expect during the session?
• What is being tested?
• If testing a device, what connection does the device require?
• What kind of room set-up would they need?
• Do the researchers require any screens for presenting, whiteboard, flipboard, clipboards or other accessories?
• Any additional requirements such as printing documents?

As we approached the research dates, our recruitment team followed our three-stage confirmation process, ensuring the participants were aware of all the details they needed to be able to attend the research successfully.

Results

We managed to fully recruit all the participants required by the client and the testing sessions were a hit with both the researchers and the users providing feedback on the devices. Although our recruitment team had to deal with three participants dropping out last minute, we were able to successfully replace them, ensuring the researchers collected enough feedback during the testing.

Looking at the group of users we sourced from our community, all the participants drank coffee at least six times a week, owned a coffee machine, and were planning to buy a new coffee machine in the next year. In terms of demographic diversity, we had a good mix of ages, ethnicities, income and industries.

The client managed to collect some great feedback from this group of participants. According to Damian Rees from Make Human, "we evaluated the appliance's user journey, from unboxing and initial setup to navigating the instruction manual and handling maintenance.Researching with a physical product was a refreshing change from a recent trend of remote digital studies."

"Having the client team on-site for real-time feedback amplified the value of our research. There are so many insights to extract," he added.

Damian shared some bits of feedback provided by the participants that he was confident were going to help improve the product's experience:

• Button feedback is essential. When someone presses a button, they need visual, auditory or tactile feedback to know their action has been received.
• Instruction manuals must provide support for visual and verbal learners, with written step-by-step guidance, as well as illustration and video.
• Users blame themselves. People who struggle to use tech tend to blame themselves for not being tech-savvy instead of the way it’s been designed. "This has always fascinated and frustrated me in equal measure," Damian added.
• First impressions last. It’s critical that the first use of anything, machine or digital experience, has to result in a successful outcome for users. If it doesn’t, it sets up the whole experience for failure and rejection of the experience.

According to Damian, it was "a fascinating two days of research into how people interacted with a prototype of an innovative new kitchen appliance. [User researcher] Rebecca Gill and I worked together to design and moderate research at People for Research's studios and they were amazing."