A row flared up this week as the Vote Leave Group accused the Confederation of British Industry (CBI) and YouGov of “wholly unrepresentative” research that amounted to a “serious violation” of the rules.
Some may write this off as campaigning tactics, while others will argue the CBI and YouGov genuinely have a case to answer. Regardless of which side of the debate you sit on, the case brings to the fore both the opportunities and perils of using research to inform and shape debates.
The core of the complaint is that the CBI carefully selected the interviewees from its own member base, allegedly ignoring many of its smaller members. As a result, Vote Leave argues, it was wrong to claim the figures were representative of British businesses. They also believe that both the CBI and YouGov breached best practice by failing to disclose information about the poll on their websites.
In short, the accusation here is that the research was engineered to provide the outcome its owners wanted.
This is a timely moment to reinforce a number of vital points around the use of research to shape debates:
- Objectives – what are you trying to achieve? Depending on the answer to that question, a quantitative survey may work best but qualitative interviews, focus groups, and other techniques and data gathering tools could also be viable options.
- Validity – are you actually measuring what you think you are measuring? Be absolutely clear at the outset about what your research represents and, critically, what it doesn’t. If you’re using a research agency, work with them to establish this.
- Reliability – if you replicated the same research three or four times, would you get the same/a similar answer each time?
- Clarity – when it comes to sharing the findings publicly, ensure that the field notes to specify samples and methodology in any public facing materials are crystal clear. And make sure spokespeople are as exact when they speak.
- Scrutiny – an evidence-led approach to emotive issues can be hugely powerful. But expect to have every element of it scrutinised closely.
Research is a critical part of an evidence-led campaign to shape and lead debates, including thought leadership. If what you have to share shines new light on an area of interest, or suggests a whole new approach to a discipline (as genuine thought leadership should), then by definition you are sticking your head above the parapet and daring to be different. Just as this will hopefully earn you plaudits, it can also create critics. However, if your evidence can speak to the points above then you can be confident and can stand by the findings.