
I ask questions if there is anything that is even slightly ambiguous. Badly designed screeners can have a disastrous effect on the quality of the recruitment. We can’t design a questionnaire if we don’t understand the sample that is required. It is also important that any ‘jargon’ is explained. We aren’t experts on everything, and whilst Google is a handy tool, it is appreciated when niche terminology is explained!
Question design should acknowledge human complexity and avoid restrictive answer choices. If someone can’t find the tick box which applies to them they’ll end up ticking something else. Adding ‘Don’t know’ and ‘Other – write in’ as an option caters for everyone.
Make sure questions are clear, unambiguous and any technical terms are defined.
When creating questions it can sometimes be prudent to build some flex in, just in case it is needed. We wouldn’t take on a project unless we thought we could book it but sometimes it’s useful to plan ahead. A brief might require people who earn over £60k. We could just ask ‘Please tell us how much you earn a) Less than £60,000 or b) £60,000 or over’. Whilst recruiting we could be struggling with this criteria and the client says we can lower the income level to £50,000. This is not helpful if we haven’t got that info to hand. Therefore, setting a question with income bands would mean we can use the new flex given. This can be applied to other questions too, for example when asking about frequency of something.
A shorter screener reduces dropout rates and improves response quality. I’ve had screeners with more than 35 competitor brands just to hide the one brand the client was interested in. It’s just wasting the respondent’s time and they are more likely to make a mistake or drop out. We sometimes get complaints from people claiming we’re not actually recruiting people for research but using data from the screeners. These complaints are usually from people who have filled in many screeners and not yet been selected to take part. It can be frustrating, and we don’t want them to stop applying for research so trying to keep the screeners to the point is important.
Open ended imagination questions are also useful for seeing how creative or literate our respondents are. This is sometimes requested for certain projects. On a recent screener we asked what advice they would give their younger self if they could go back in time. Memorable answers included “Eat less sugar, join a contortion class and drink water.”; “Use more eye cream” and “Avoid a girl called Jane at all costs and never ever play a gig in Watford”
Finally it’s important to test the questionnaire, as well as picking up on typos, you can check the flow of the questions and make sure any logic in place works.
“Thanks again, for another excellent job. You are my go-to agency when it comes to consumer recruitment, and we really appreciate how good you are at questionnaire design, and the thoroughness of the recruitment. Thank you!”
Kath Rhodes, Qual Street