How did the polls fare in Election 2015?

On the whole the polls did well in Election 2015.

Below is a table displaying the difference between the final polls from different firms and the actual preliminary vote count. I subtracted the poll number from actual vote shares. Negative values mean the poll underestimated party support while black represents an overestimate. Angus Reid provided polling numbers representing both eligible voters and those they deemed to be likely voters.

Very few numbers are outside the margin of error specified by the pollsters (note that margin of error does not really apply to online pollsters).

What is striking is that the two most accurate surveys were both conducted on the day before the October 19 election, which strongly suggests that late movement to the Liberals was a significant factor in the outcome. Also note that the likely voter model of Angus Reid performed less well than their eligible voter model. This may partly reflect the higher turnout we saw on election day.

The numbers above deal only with the national scene. However, seat outcomes depend much more on regional numbers, which being based on smaller regional samples are prone to greater error. However, I think the table below suggests that even in the regions the polls did perform well.

Note that Nanos has an advantage here. It would have a slightly smaller total error because it reports on one fewer region (the Prairies rather than treating Manitoba-Saskatchewan as one region and Alberta as another). Nevertheless it is clear Nanos performed well.  I have also omitted the Bloc from this table. The numbers here of necessity are all absolute values (the difference between the regional vote shares and the poll numbers) as it was the only practical way to aggregate them. This is why all are positive values and there is no red as in the table above.

Mainstreet Technologies finished behind Forum and Nanos I think mainly because the poll was finished earlier. However, it had the largest sample size of any pollster at 5,546, partly accounting for a strong overall performance.

My conclusion is that the polls did well this time despite the obvious challenges confronting them. On those challenges, this article by Donna Dasko in the Globe and Mail is a must read for anyone wanting to understand the dilemmas confronting pollsters today.

They did get it right this time but the possibility of a future fiasco as we was in British Columbia in 2013 and Alberta in 2012 remains.

What did not do well were the seat projection models like my own. This is a subject for a future post.