Reporting on Polls

It is TC's view that the reporting of public opinion polls by media outlets has deteriorated recently, athough it has long been flawed.

Two cases in point.

Take, for example, this headline in the Toronto Star, a newspaper that has increasingly been displaying the worst habits of the tabloid NY Post/Toronto Sun variety:  Majority of Canadians support return of death penalty, poll finds.

What is striking is that the headline that appears on the website of the polling firm in question, Angus Reid, actually reads: Canadians Hold Conflicting Views on Death Penalty

While the lead paragraph in the Star speaks of Canadians "warming to the idea of a return of capital punishment", it never outlines the details of the second question asked in the Reid survey: "All things considered, which of these two approaches would you prefer as punishment for convicted murderers in Canada?" The options are "life imprisonment without parole" or "the death penalty".  By a margin of 50% to 38% life imprisonment is preferred.  The question that returned a majority for the death penalty was: "As you may know, Canada eliminated the death penalty for murder in July 1976. All things considered, would you support or oppose reinstating the death penalty for murder in Canada?"

It is important to recognize that capturing real public opinion is elusive. Question wording has a significant impact on the numbers that a given query will produce, but it seems clear that, given a choice, Canadians prefer life imprisonment as a punishment for murder.

The Reid headline is more accurate. The Star acknowledges the point about respondents preferring life imprisonment, but the words are deep in the story and oblique: "Given the choice of supporting the death penalty or life imprisonment, 50 per cent chose the latter, the survey found." The impression is clearly left that Canadians are more enthusiastic about the death penalty than life imprisonment. But it ain't so.

A quick Google search also reveals a 2010 Ekos poll that contradicts the Reid question that did produce a majority favouring return of the death penalty (a plurality of 46% to 40% disagreed with the idea of restoring the death penalty).  Information about other surveys with results on the same topic should also have been included if the Star was doing its job.

A more traditional flaw one sees oft repeated is the media outlet/pollster that reports a new result without reference to the work of any other polling firms. Thus we have the following National Post headline about a Forum Research poll: Liberals reach post-election high, but Tories rule the polls 

Just one problem, they are referring only to Forum Research polls. The last Nanos poll, taken before the Forum Poll, had the Liberals at 27.6%, slightly higher than Forum's 26%. Unfortunately, one sees this kind of comparison frequently.  The real problem is the careless journalism.