The prominence of doing research digitally has brought about an influx of data and an increased knowledge of people across the globe. This carries with it the assumption that the more information we have about an audience, the more informed we will be about how to effectively reach them.
However, using a million data points to identify the right participants to conduct research with does have its drawbacks, and we tend to believe that the pros of overly-detailed screeners outweigh the cons.
Work with local, on-the-ground partners to sense-check the key defining criteria of your screener for every market you're working in.
Nikki Lavoie, EVP of Global Experience Strategy lists a few common screener types and simple solutions that set up your multi-country study for success and you and your team to make better decisions
1. The overly scrupulous screener
Why is this a problem? You’ve revised your screener 6 times and it’s now down to only 17 pages. Documents of this size and length require lots of time and attention and several extra rounds of pre-qualifying participants, unnecessarily prolonging the preparations for your study. If you’re working with an external partner for recruitment, they likely haven’t estimated spending this much time screening and confirming participants. From a respondent perspective, a long screener means a long screening process. Respondents might be tempted to lie – they’ve already dedicated 10 minutes to answer questions and they really want to be invited to the research. Who cares if they don’t actually have a cat? They used to, so they could probably knowledgeably answer questions about being a cat owner, right?
The solution: Simplifying your screener as much as possible: earmark some time to take a colleague (preferably one from another department) through the entire screener—out loud—as if you were trying to recruit them for your study. Take notes of how long it takes, which questions are uncomfortable, and which questions don’t really add much to your understanding of whether or not they fit the profile you’re after. If you still end up with a complicated or long screener (it happens!), schedule extra time for the recruitment phase to make sure that you find your niche respondents. If you’re working with external suppliers, let them know in advance so they can gauge their timing and pricing accordingly. Recruitment is an unquestionably important piece of any study, and no one wants it to be rushed or underestimated.
2. The searching for the needle-in-the-haystack screener
Why is this a problem? When recruitment criteria are extremely specific, you are likely creating the behavior you’d like to see, and not exploring it naturally. Unless you’re looking for users who exemplify those already identified in a quantitative segmentation study (which has its own set of considerations), qualitative research should assume that the users you’re looking for exhibit some of the traits you think are important. A common example of this is using typing tools based off of segmentation. While typing tools can be extremely useful ways of classifying someone into a particular segment, when talking about a qualitative recruit, those tools can introduce a level of detail that impedes the recruitment process.
The solution: Take some time to sit down and identify all of the criteria that you deem essential to identifying your users, then cut that list in half and be flexible about the remaining items. It also helps if you think in broader terms that allow researchers to talk to participants and determine if they’re a good fit or not, for example: “I’m looking for recent grads who have started a new job but not in the field they studied at university, and are considering whether they want to stay the course or keep looking for a new job.”
3. The mono-market screener that is rolled out globally
Why is this a problem? A screener that was developed in one market and then used to find participants in other locations needs to be adjusted, and not just in terms of language. Cultural differences, nuances and several other factors can make a screener completely inadequate or inappropriate across markets. For example, when seeking users who consume social media, the consumption types and platforms will not be the same in the US as say, Greece or Turkey, and thus measuring potential participants by the same metrics will generate both inaccurate qualitative data, and less fruitful conversations.
The solution: work with local, on-the-ground partners to sense-check the key defining criteria of your screener for every market you’re working in. This may even include subregions of the same market, where urban and rural differences may be vast, and thus seeking “a general mix” isn’t possible and could skew results.
There’s a lot of great work being done out there to combat problematic screeners and make your next research project participant-first and successful across several markets.