Forecast Model Errors

Every election I test the accuracy of my forecasting model by using the actual vote shares as inputs. In other words this exercise treats the provincial vote shares as if they were the perfect opinion poll. This isolates the model’s error from any polling errors. I don’t use the vote shares for each province. I group the Atlantic Provinces into one region and make Manitoba/Saskatchewan as another region, exactly as most national opinion polls do.

The results I measure in detail in two ways. First, did it accurately predict which party would win individual constituencies? Out of the 308 ridings the model accurately predicts the outcome in 279 ridings, and incorrectly in the remaining 29 for 90.6% accuracy.

Secondly I also compare each party’s predicted share with its actual result, turning the pluses and minuses into absolute numbers, and then I average the errors.

The average error by party was:

Liberal – 3.4%
Conservative – 3.2%
NDP – 2.2%
BQ – 2.6%
Green – 0.8%
Other – 1.4%

On the whole this compares favourably with earlier elections. The average error per seat in 2004 was slightly higher. With the two elections closer together this is perhaps not surprising. However, before the election I had thought seat predictions might be less accurate. I was wrong about that.