Posts
First, there are legitimate unanswered questions about online polls. There is no accepted methodology. Partly because of that there is effectively zero transparency and accountability on the part of online polling companies. For example, on their website at the bottom of a recent poll the Reid group says this:
Methodology: From September 20 to September 21, 2011, Angus Reid Public Opinion conducted an online survey among 1,668 randomly selected Canadian adults who are Angus Reid Forum panellists. The margin of error—which measures sampling variability—is +/- 2.4%, 19 times out of 20. The results have been statistically weighted according to the most current education, age, gender and region Census data to ensure a sample representative of the entire adult population of Canada. Discrepancies in or between totals are due to rounding.What it doesn't tell you is how large the panel is from which the sample is selected. When it says 'randomly selected' it goes on to say 'who are Angus Reid Forum panelists'. However, the phrase 'randomly selected' on its own implies 'selected from the whole population of Canada'. The subsequent reference to the panel also infers, without saying so explicitly, that they actually come from the Angus Reid panel.
In Canada if it is within a province, one assumes it is a subset of a national panel and perhaps panel size within the province could be an issue. But we don't know the overall size or demographics of the national panel - gender, income, language - let alone the provincial sub-sample. We don't know what methods were used to recruit it. Surely if we are to accept the panel itself as representative we would need to be assured that it was selected at random. Is this case?
We don't know how the panel is contacted or how the surveys are conducted to ensure their integrity. If these are to become commonplace, the polling industry should set some standards that provide some reassurance that there is some validity to all this.
TC doesn't agree with the Ipsos critique of IVR. These are ultimately phone surveys, an established proven methodology. The only difference is an electronic voice and answers given on a telephone touchpad. We already know such polls have obtained good results. See what New York Times polling blogger Nate Silver (who formerly blogged at fivethirtyeight.com) said about Survey USA, which uses this methodology:
... SurveyUSA is a very strong polling firm; no company has done more to contradict the notion that a "robopollster" need be inferior. Although it's not my place to make any endorsements, it would certainly make the life of electoral forecasters easier if SurveyUSA were to get more business.
