Polling Errors in the 2019 Canadian Election - Leger Marketing Achieved the Best Results

NATIONAL POLLS

Most analysts who made calculations on the accuracy of polling in the 2019 election published their data soon after the result was made known. All the tables I saw compared polling only to the national results. I made a similar calculation that can be seen in the first table below, but I went further comparing the regional/ provincial poll numbers to election results (in a table further down).

First the national polls and how they fared. The polls are ranked by total absolute error. The red numbers indicate polling that fell below the actual results while the numbers in black indicate polling above the actual results.

A couple of observations: every pollster underestimated Conservative support and over-estimated the NDP. It is likely that the latter is an indication of a significant degree of tactical voting for the Liberals on the part of NDP supporters who, despite how they actually voted, had no hesitation expressing their true feelings to pollsters. One region where Conservative strength was under-estimated was Alberta (see below).  In most cases the polls over-estimated the People's Party of Canada (PPC), a party I suspect may now disappear. On the whole the national polls were were fairly accurate.

It is not indicated in the table, but the national polling in this election from companies that rely on online polling was more accurate than the telephone based pollsters, including those that used the computerized Interactive Voice Response (IVR) technique. However, some of the phone/IVR pollsters did better with their regional polling numbers.

REGIONAL POLLING

It is a given that the errors in regional polling are much higher as the sub-sample sizes in provinces and regions are smaller (in some cases quite small). Despite the errors and what look like high totals of absolute error I found the performance of the polls regionally was relatively good with the exception of the Conservatives (see the line at the bottom of the table labelled 'Average error all regions'). In the table below I grouped the polls using shading ranging from white to darker gray to distinguish three groupings: those that attained the best regional/provincial results, a middle group and those that were weaker. They are ranked by Total Error.

A few observations (partly based on data not shown here). It is notable that Leger (on online pollster) performed best in both national and regional polling accuracy.

One region where polling error was high was Alberta.  The polls significantly underestimated Conservative popular support. However, the error was of no importance with respect to seats. The CPC won all but one seat and would have done just as well in seats with many fewer votes. The error appears to have been rooted in turnout, which increased in all the constituencies in Alberta outside Calgary and Edmonton (it dropped in some ridings in the two cities). The most conservative region of Canada where the net outcome in all ridings was all a foregone conclusion turned out in greater numbers than in 2015. This is the region of maximum anti-Trudeau feeling, so perhaps we shouldn't be surprised.

I wrote previously of how different the outcome would have been if votes were distributed by proportional representation. Even with a pure PR system the Conservatives would still have won 25 of 34 Alberta constituencies rather than their actual 33 (the party won nearly sixty-nine percent of the province-wide vote). Rural Alberta Conservative support was a classic example of a wasted vote in a first-past-the-post system.