Monday, May 05, 2014

Turning poll results into estimated election outcomes

I devised a means of taking public opinion surveys and calculating what they would mean if they accurately reflected election outcomes in 1993.  I have therefore had the opportunity to compare what I projected about an election outcome and the actual results over the course of many elections.

I can say with certainty that this is more an art than a science although an algorithm is the basis for the calculation. The polls I used at first were scientific in that they were random samples based on telephone surveys (many polls today don't qualify as truly random). However, there is no guarantee that the electorate will follow mathematical rules when they actually vote. For example, the seat forecasts (including mine) all underestimated how well the CAQ would do in the recent Quebec election. I have had some very accurate results and some with significant errors. Along the way I have learned enough to know who is good at this and who is not.

A host of younger (I am now a senior citizen) bloggers and political scientists have got into this game. The best known in Canada is Eric Grenier of threehundredeight.com but there are many others including Bryan Breguet, Kyle Hutton of Blunt Objects, Canadian Election Watch, Dan Arnold of Calgary Grit, Barry Kay of the Laurier Institute for the Study of Public Opinion and Policy (LISPOP for short) and Earl Washburn of Canadian Election Atlas. Overall, I would say the work of all of them is quite good.

Not so Forum research. Yes, they have had some good outcomes but others that are downright bizarre, including their current seat projection based on a poll that has the Ontario PCs ahead by five points in public opinion, but trailing the Liberals in expected seats.


I am not the first to notice there are problems with Forum's methods, but it is their polling methodology that typically is the focus of attention.

One good thing about Forum is that they typically put the results of their surveys on the internet. However, consider the following two poll results and seat projections from Forum:


Forum would have us believe that almost identical polls taken less than four months apart would produce two quite different outcomes for the Liberals and PCs even though their vote shares are identical. It simply isn't believable. A second and in my mind more serious issue is that even though the NDP in these polls has a vote share less than 1% different from its 2011 vote percentage in both cases, the seat projection suggests the NDP would lose 8 seats from its 2011 total. This would occur despite the fact that most of its competitive races are run against the Liberals, who are down significantly from their 36.5% vote share in 2011 in both Forum surveys. Again it isn't believable. Newspapers use Forum and that dominates secondary and social media coverage, but my recommendation would be to pay attention to the seat forecasts of 308 or LISPOP or any of the others before taking Forum seriously.

For what it is worth my own seat projection using Forum's latest numbers is PC 49 seats, Liberal 36 seats, NDP 22 seats (a majority is 54 seats). Almost never would a party trailing by five points finish ahead of its rival in an election. It is true that the Liberal vote is more efficient as Forum says, but there limits to vote efficiency.

TC thinks the campaign dynamics will be what matters in Ontario, how important the alleged scandals turn out to be, and how much fear Tim Hudak inspires in the centre and left. More on Ontario in the future.