IRPP Analysis of 2004 Canadian Election

The Institute for Research on Public Policy’s monthly magazine Policy Options September 2004 issue was substantially devoted to analysis of the 2004 federal election (you can find it in the back issues section).

Included is an excellent analysis of the polls by political scientist Geoffrey Hale. There was a megaton of media and other comment at the time of the election about how the polls missed the outcome. Prof. Hale, however, argues that the polls were by and large accurate and I agree:

This article examines polling data for each region during the campaign, and compares it with actual outcomes by region. It concludes that although most pollsters missed the extent of last minute vote swings in Ontario, regional vote projections for three of four major polling firms that publishing regular polls during the campaign were well within national margins of error.

Another article by Maurice Pinard makes the following observation about the alleged polling errors.

How can such forecasting errors be explained? The answer is astonishingly simple: Voting intentions changed at the very end of the campaign. The recently published results of the Canadian Election Study, conducted by university professors, indicated that, at least outside Quebec, an important surge in support for the Liberals occurred during the last few days of the campaign, especially during the days after the media’s pollsters had left the field. This coincided with a corresponding decline of both Conservative and NDP support.

Pinard says “outside Quebec”. More precisely what happened was that public opinion changed in the last few days in Ontario shifting from both the Conservatives and the NDP to the Liberals.

The articles can be found along with others, here: http://www.irpp.org/po/index.htm

What my forecast model says about this

I have compared the regional polls to the final results myself. The only strong and consistent pattern regionally is in Ontario where all the pre-election polls overstated Conservative and NDP support and understated Liberal support, likely because of last minute shifts. On average the closing polls in Ontario understated the Liberals by 6 points, while overestimating Conservative support by 3 points and the NDP by 2 points.

The results of a last minute poll conducted by Environics for the CBC lend support to this thesis. When you connect to the CBC’s web page, scroll down and look at the party preference numbers in Etobicoke-Lakeshore. It shows the Liberals winning over the Conservatives 49% to 31% almost identical to the actual margin of 50.2% to 30.6%.

However, using my forecast model, when I project the final Ipsos-Reid poll Ontario results, which reported the Liberals ahead of the Conservatives 38-34, I find the Liberals would only have won this riding by two points. When I put in the final election margin in Ontario of 45-31 into the model, the Etobicoke-Lakeshore outcome becomes 45-33, much closer to the actual result. If the pollsters were really getting it wrong in this election, then Environics CBC should have been wrong in Etobicoke-Lakeshore and clearly they weren’t.

The patterns in other regions don’t clearly show a shift to the Liberals although there may have been some small movement. The margin of error and different findings by different pollsters make it impossible for me to come to a definitive conclusion.