It’s the day after the Same Sex Marriage referendum which has been passed by a huge majority, 62% voted in favour of the change to the constitution and 38% voted against. So is this a great victory for the pollsters, who all predicted a majority victory? No it certainly isn’t.

In fact all the pollsters (including RED C) using standard polling techniques were out by at least 7% in their prediction of the proportion of people who would vote yes. In every case they over predicted the Yes vote and under predicted the No vote.

 

Final Poll Results (Excluding D/K) Yes No Sample Dates
RED C*
(RED C did not re-allocate D/Ks, but suggested they would all vote NO)		 69% 31% 1009 11-13 May
Millward Brown 69% 31% 994 2-15 May
Behaviour & Attitudes 71% 29% 927 1-11 May
Ipsos MORI 70% 30% 1200 13-14 May

 

However, using a Wisdom of Crowd approach in an effort to get behind issues seen both in the run up to this referendum and learnt from past referendums, RED C did successfully predicted the result. This approach, which we used in an effort to uncover shy NO voters, meant we were able to predict a 62% Yes to 38% No result completely accurately.

Slide6

In the past we had seen that most of those telling us they were undecided at referenda, ended up voting No. If you are not convinced of the arguments to vote to change the constitution, keeping the status quo and voting No was a far more likely outcome for your vote.

This referendum also raised the possibility that there were “Shy No” voters?
The polls in the recent UK election once again showed us it is important to evaluate the possibility that some voters are avoiding telling us exactly how they feel, particularly if they are somewhat concerned about admitting to voting for one side or another. In this campaign the No side have not been strongly represented, with no political party on the side of the No camp, and the media firmly in the Yes camp. So it wouldn’t be a surprise if voters were “shy” about telling us they plan to vote No. This is why in our view it is safer to assume that those claiming to be undecided are more likely to vote No.
In past polls we have also seen that many voters claiming that they will vote Yes do still have reservations about some of the claims being made by the No camp. To get behind this for our final poll, we tried a different exercise, by testing a Wisdom of the Crowd approach to polling.

As well as asking how our representative sample of voters will vote themselves, we also asked them what they thought the final result might be. This aims to take into account the conversations that voters are having with family and friends about the topic, and how they see people voting on Election Day. Once the 1% of don’t knows were excluded, this analysis predicted the final result with 100% accuracy! Suggesting that there was indeed a “shy No” voter effect in the standard poll analysis.
So what of the standard polling? Despite the error across most polling companies using only standard techniques, unlike in the recent UK general elections no one is calling for an investigation into polling here in Ireland. The main reason is that the polls generally told the story that the referendum would be passed, and so no one is complaining.

But lets put this into context. The only reason the pollsters have got away with it on this occasion is because the error didn’t have a material effect on the result. If the vote had been lot closer a 7% error could well have made the difference between the vote being won or lost. In fact the UK polling companies worse party prediction was just 4% away from the actual result, so the reality is that they predicted the UK election significantly better than polling companies here predicted the result of this referendum.

Clear aspects of voting that influence poll predictions need to be further taken into account by pollsters trying to get an accurate result. Likelihood to vote remains an important feature. Many of the polls final results showed high levels of undecided voters. But many of these voters were never going to get to the polling booth. The reallocation of undecided voters, simply by suggesting they will vote the same as those who are decided is again shown to be a blunt analysis. In this case almost all undecided voters in the pre-referendum polls who were likely to vote, appear to have ended up voting No. Simply showing the result excluding undecided doesn’t take account of this kind of movement. Finally, ignoring the possibility that shy voters exist, has once again been shown to be a mistake, and one that the industry needs to consider carefully as we move towards another general election.

The learnings for us as pollsters are that we need to constantly question and get behind the topline claimed behaviour of voters, to ensure we are doing everything we can to deliver clear insight into how they plan to behave. Predicting elections is not an easy business, and on another day we may not have been so fortunate that our analysis to get behind shy voters worked so well. But the most important thing is that we continue to try different techniques to deliver the best insight we can.

Download our final poll report below:-
SBP-May-2015-Poll-Report-Same-Sex-Marriage2