Asking better research questions

‘Over the next year, how often do you think you would use our product if we changed to this new packaging?’ 

A reasonable question for a researcher to ask, perhaps. The answer would of course be gold dust for predicting sales. So why not ask as many people as possible? 

The answer, if true, would be extremely useful. Unfortunately, the answer you receive will almost certainly be useless. Why? Because you asked the wrong question.

Some of the biggest marketing mistakes come down to the wrong questions being asked in research. An array of biases can cloud the researcher’s conclusions if questions aren’t framed in ways that elicit accurate responses. The truth is, respondent powers of recollection, prediction and self-assessment are often over-estimated. In the example above, any response given here would be pure fantasy, and not grounded in any reality. What someone says they will do is often quite different from what they really do, and it’s a researcher’s job to identify the difference.

To bypass this say-do gap, researchers need to understand how to apply a behavioural lens to the process of asking questions. Consider the following when creating your research questions to ensure you are getting valid responses which lead to genuinely useful outcomes.

Behaviour before attitudes

Simplicity and directness can be crucial to eliciting accurate responses. Self-reported behaviour is often unreliable, especially when it is recalling attitudes rather than actions. Replicating real-world choice in a question will encourage the respondent to consider context, and provide useful responses that relate to how they actually behave.

Disguise your questions

To avoid acquiescence bias, your true motives should often be hidden from the respondent in the way that you ask your questions. Some people may feel pressured to give exaggerated responses to align with norms for example. Use questions that aren’t relevant to your objective to give them an opportunity to provide positive self-affirming responses, leaving room for more honesty and accuracy in the areas that you actually want to find out about. 

Break behaviour down into smaller questions

The job of the researcher is to make it as easy as possible for the respondent to reliably recall information about their behaviour. This can often involved breaking down larger questions into smaller ones. For example, rather than asking ‘Have you bought a cordless vacuum cleaner in the last six months?’, break it down into the following: 

Instead:

Which, if any, of these have you bought in the last six months?

• A washing machine
• A tumble dryer
• A vacuum cleaner
• A microwave

If you bought a vacuum cleaner, was it: 

• Corded
• Cordless
• Robotic etc.

This is not only more granular, but more provides wider context, and avoids bias.

Use appropriate recall periods

Before asking a question, ask yourself one first. ‘Can I really expect this person to remember that?’.

Realistic recall periods are crucial for accuracy. For reference, here are some timeframes to consider.

• One day: Newspaper readership
• One week to one month: Food consumption
• One to two weeks: Bank transaction
• 12 to 18 months: Holiday
• One to two years: Car purchase

Stretch beyond these, and your data will be less grounded in reality.  

Reduce Social Desirability Bias

People will often want to appear other than they are. Certain wordings of questions will prompt the respondent to be less honest, and put up a guard in an attempt to make themselves appear more socially desirable. 

Questions can be written to eliminate SDB. Instead of asking ‘Have you read the latest edition of the Highway Code?’, we could ask ‘Have you had time yet to read the latest edition of the Highway Code?’. This removes the pressure, and an implication of obligation which may lead the respondent to lie or exaggerate. 

Ask better questions with a behavioural approach.

These tips are but a small fraction of the considerations that researchers must make to ensure that they are collecting balanced, useful information. For more information on how to craft better research questions, and how to close the say-do gap as a result, get in touch with Trinity McQueen today.