Several polling firms whose results are being widely reported in Canada during this election conduct their surveys via the internet. Now new research, "rigorous work from researchers at the forefront of academic evaluations of survey data", suggests that such surveys are unreliable. Gary Langer, who provides public opinion polling, analysis and consulting services to ABC News after having served as the network's polling director, says:
"Past research has thrown doubt on the ability of so-called “opt-in online panels” to produce results that accurately reflect the views of the broader population. The new study not only reinforces that evidence, it also calls into question whether such data are reliable for two other key purposes, evaluating changes over time and differences among groups."
Among online surveys being reported in Canada concerning the election are party preference results from Angus Reid and Léger Marketing, and issue and leadership preferences from Ipsos. (It should be noted that, to date, the Ipsos party preference numbers come from telephone surveys.) Why are Canadian media uncritically reporting these online results when significant doubts about this type of research are being raised south of the border and have been in the past?
The new research was about the U.S. Census, not about party politics, but it is clear that it applies to party preference polls. Langer notes: "The paper was written by Josh Pasek, now an assistant professor of communication studies at the University of Michigan, and his Ph.D. adviser, Prof. Jon Krosnick, of Stanford University."
He goes on to point out that ABC News does not accept online data as valid:
"We ruled out reporting opt-in online panels at ABC News more than a decade ago. In 2008 David Yeager, then another student of Krosnick’s, along with Krosnick and several of their colleagues, wrote a groundbreaking paper questioning the accuracy of opt-in online data. And a year ago the American Association for Public Opinion Research issued a report saying such panels should not be used to represent population values, should not be described as representative and should not claim a margin of sampling error."
Online surveys are now routinely used for market research "because it’s cheap and fast. But it’s also problematic, because the nature of opt-in panels violates the most basic principles of probability sampling."
All polling needs to be treated cautiously and skeptically but it is clear from this Langer's commentary and the new research that we should view telephone polling as more trustworthy than the online variety.
Subscribe To
Posts
Comments
