Friday, July 08, 2016

What role has the U.S. media played in the rise of Donald Trump?

The role of the media in the rise of Donald Trump has been the topic of a fair bit discussion on political web pages. Rarely, however, is the actual impact of the media on politics subject to serious analysis. Helping fill the gap is a new study from Harvard's Shorenstein Center on Media, Politics and Public Policy. It recently released a study of media coverage of the pre-primary period in the current presidential election campaign on the part of major media outlets including broadcast networks NBC, CBS and FOX as well as the newspapers the Los Angeles Times, USA Today, the Wall Street Journal, the New York Times and the Washington Post. CNN is a notable omission from this list.

The findings, to say the least, are striking. It is important first to understand that a media tilt one way or another can have a paradoxical impact on political choices. There is no one-to-one relationship where favourable media coverage and attitudes (or the reverse) necessarily creates the expected consequential impact. However, it remains true that the media can profoundly influence beliefs in some circumstances in what seems to be the obvious way.

The Shorenstein study deserves to be taken seriously. It justifies its focus on the pre-primary period on the grounds that political success prior to the primaries is "the best predictor of which candidate will win the presidential nomination", better than Iowa and New Hampshire.

It employed a careful methodology that I have quoted in detail at the bottom of this post.

There was no more important finding than the following: "during the year 2015, major news outlets covered Donald Trump in a way that was unusual given his low initial polling numbers—a high volume of media coverage preceded Trump’s rise in the polls. Trump’s coverage was positive in tone—he received far more “good press” than “bad press.” The volume and tone of the coverage helped propel Trump to the top of Republican polls."

Moreover the media, essentially seduced by the carnival barker character of Trump's news value, could not look away, and helped enable his success:
"So what explains the news media’s early fascination with Trump? The answer is that journalists were behaving in their normal way. Although journalists play a political brokering role in presidential primaries, their decisions are driven by news values rather than political values.  Journalists are attracted to the new, the unusual, the sensational—the type of story material that will catch and hold an audience’s attention. Trump fit that need as no other candidate in recent memory. Trump is arguably the first bona fide media-created presidential nominee. Although he subsequently tapped a political nerve, journalists fueled his launch."
The free media was worth millions to Trump. A graphic from the study:

Source: Media Tenor. Based on amount of positive and neutral news coverage in eight news outlets—CBS, Fox, the Los Angeles Times, NBC, The New York Times, USA Today, The Wall Street Journal, and The Washington Post— for the period January 1-December 31, 2015.

So the media was pro-Trump but what of Hillary Clinton. The media tended to ignore the Democratic race because at first it simply assumed Hillary Clinton was going to win the nomination. Nevertheless, they still had some things to say:
"For her part, Clinton might have wished that the Democratic race received even less attention than it did, given that her coverage was the least favorable of the leading contenders, Democratic and Republican. Month after month (see figure below)... her coverage was more negative than positive. There was only one month in the whole of 2015 where the tone of her coverage was not in the red and, even then, it barely touched positive territory. During the first half of the year, excluding neutral references, it averaged three to one negative statements over positive statements. Her coverage in the second half of the year was more favorable, but still damning. The ratio for that period was more than three to two negative over positive.... 
Whereas media coverage helped build up Trump, it helped tear down Clinton. Trump’s positive coverage was the equivalent of millions of dollars in ad-buys in his favor, whereas Clinton’s negative coverage can be equated to millions of dollars in attack ads, with her on the receiving end. 
Source: Media Tenor, January 1-December 31, 2015.

Not surprisingly, the media at first ignored Bernie Sanders. However, the Sanders campaign, somewhat like Howard Dean's endless summer anti-war effort in 2003, was able to generate enormous crowds at events that did get his message out. His followers then donated millions of dollars online, making him impossible to ignore. And then the media started to cover him:
"Strictly in terms of tonal balance—good news vs. bad news—Sanders was the most favorably reported candidate—Republican or Democratic—during the invisible primary....
Sanders’ issue positions also netted him positive coverage. Although they accounted for only about 7 percent of his coverage, they were a source of “good news.” News statements about Sanders’ stands on income inequality, the minimum wage, student debt, and trade agreements were more than three-to-one positive over negative.... That ratio far exceeded those of other top candidates, Republican or Democratic....
Sanders’ media coverage during the pre-primary period was a sore spot with his followers, who complained the media was biased against his candidacy. In relative terms at least, their complaint lacks substance. Among candidates in recent decades who entered the campaign with no money, no organization, and no national following, Sanders fared better than nearly all of them. Sanders’ initial low poll numbers marked him as less newsworthy than Clinton but, as he gained strength, the news tilted in his favor."
Despite negative coverage for Clinton and positive coverage for her opponents she has both defeated Sanders in the race for the Democratic nomination, and leads Trump by 5 to 6 points in the national polling averages. Polling wizard Nate Silver currently gives her a 77% chance of winning in November.

There is more to politics than favourable media coverage, much more. Clinton's poll numbers tell us that we are seeing what appears to be a paradoxical outcome from a year-long process that in the end has defied the direction of media coverage. Clinton's strength derives from multiple sources including longstanding relationships, which I won't to into here. However, we should not forget that mass media can be better at telling the public what to think about, than what to think. This whole area is much more complex than is generally assumed.

The study has received little attention. One would not expect mass media to place too much emphasis on studies documenting their failures, but the Shorenstein effort deserved more attention than it has received. I do think Shorenstein made a mistake when they incorporated Fox News and excluded CNN.  Fox isn't really a news organization per se. It is likely it made the results more anti-Clinton and pro-Trump than otherwise. However, I doubt it would have changed the balance of the overall findings.

A major story in the news this week has been coverage of the fact that Hillary Clinton will not face criminal prosecution for her handling of emails as Secretary of State. The build-up to this conclusion has meant nonetheless plenty of negative coverage for her (including during the period of the Shorenstein study).

There is a legitimate question here about media coverage: "Was this conclusion that no charges were justified easily predictable?" One the best analysts of issues of this kind is Josh Marshall of the widely-read blog Talking Points Memo. After the FBI made its announcement he penned an analysis that argued:
"What is most notable about this news from a political and news perspective is that this outcome was entirely predictable, indeed almost inevitable, based on the facts that were publicly known about the case.
Let me say that again. There was always the chance that there were dramatically different or new facts the FBI had that had never been made public or intimated in any way. Possible but extremely unlikely. Given what we knew, criminal charges weren't even in the realm of reasonable consideration. You could find this out with just a little bit of reporting, speaking to former federal prosecutors, legal experts, really anyone knowledgable about the relevant law and past practice."

So how was it that the media missed this?  I think the there is a persistent media bias I like to call the "good story bias". Essentially the media always want a good story. That influences them in many ways including how they treat any case that potentially affects the possibility that a political figure has committed a crime, major or minor. They want to believe it is true because it is a much better story than "nothing to see here folks". But if they are seriously committed to the truth, they should be looking that the possibility of the opposite is true in major cases such as this one. Josh Marshall sees the Clinton email story as an example of massive media failure:
"All this said, this was 99.9% predictable and 100% obvious. It's a mammoth press failure that for various reasons this reality was concealed from the public."
Methodology of Shorenstein Study

The key paragraph is below.

"The data were provided by Media Tenor, a firm that specializes in collecting and coding news content. Media Tenor’s coding of print and television news stories is conducted by trained staff members who visually evaluate the content. Computer-based coding is less reliable and is not used in Media Tenor’s research. Coding of individual actors (e.g., presidential candidates) is done on a comprehensive basis, capturing all statements of more than five lines (print) or five seconds (TV) of coverage for a given actor. Coders identify relevant themes (topics) for all actors in a given report and evaluate tone (positive or negative) on a six-point scale. These tonality ratings are then combined to classify each report for each actor as being negative, positive, or having no clear tone. Coding quality is maintained through comprehensive spot checks and inter-coder cross checks to maintain a minimum 85 percent inter-coder reliability rate.

Monday, March 21, 2016

Manitoba Election: After 16 years the NDP can expect to leave office

A version of this has been cross-posted at iPolitics.

Manitoba will elect a new government when voters go to the polls on April 19. Barring dramatic and unexpected developments during the campaign, Manitobans will elect a PC government lead by Brian Pallister, a one-time federal Canadian Alliance and Conservative MP.

Should he win, Pallister would displace an NDP regime that has governed since 1999, when Gary Doer defeated incumbent PC Gary Filmon. Recent polls have given the Progressive Conservatives a large lead. However, average Conservative support is registering just a little higher than the nearly 44 percent the party achieved in the October, 2011 election.

The big difference has been a striking increase in Liberal support at the expense of the NDP: the big lead comes from an even split in support between the two parties. Polls currently suggest they might end up with as many as 40 of 57 constituencies.

The NDP government of Greg Selinger has been in political trouble since 2013, when it reversed a campaign promise from 2011 not to increase the sales tax in order to generate revenue to finance infrastructure investment needed to respond to a series of unprecedented floods. The tax hike (from 7 percent to 8) was followed a year later by an extraordinary rebellion in party ranks that forced Selinger to defend his leadership at a party convention in 2015 - where he barely scraped by.

Liberal gains should perhaps not be surprising given the strong showing of the federal Liberals in Manitoba last October (the party captured 7 of Manitoba's 14 seats). One might think that a key advantage for the Liberals is the popularity of the new federal Liberal government of Justin Trudeau, and no doubt he has helped the Liberal brand name.  However, the federal and provincial parties have an icy relationship and the federal party apparently does not think much of Manitoba Liberal leader Rana Bokhari.

Winnipeg Free Press reporter Mia Rabson offered this account in February:

"There’s not a lot of enthusiasm for the leader," said one federal Liberal source speaking on the condition of anonymity. "It’s a different organization than us. They are just completely different."

Some of Bokhari’s policies, such as privatizing liquor sales, are too far to the right for federal Grits, and the fact she hired former Tories solidified the feeling she doesn’t have a lot in common with Team Trudeau. (Provincial Liberal Director of Communications Mike) Brown used to work in the regional ministerial office of former Conservative MP Vic Toews. To say there was no love lost between Toews and federal Liberals would be an understatement, and there are many who simply don’t trust anyone who worked for him.

That has prevented federal Liberal riding associations from handing over their lengthy electoral lists of supporters, volunteers and donors to provincial candidates.

Historically the Manitoba Liberals have traditionally leaned to the right of their federal counterparts so it appears nothing has changed. Still the surge in Liberal support may have the party thinking in terms of 1988, a year when the Manitoba Liberals, after years of electoral drought, suddenly found both federal and provincial success. In the 1988 Manitoba election under leader Sharon Carstairs the party captured two-thirds of the constituencies in the City of Winnipeg just as an incumbent NDP government was failing. If the province stopped at Winnipeg's Perimeter Highway there would have been a large Liberal majority government that year. However, the Liberals won just one seat outside Winnipeg, allowing PC Gary Filmon to form a minority government that he would convert to a long-term majority two years later.

For the Manitoba PCs division of the electorate between two opposition parties has been a key to electoral success. For the NDP success has come at the expense of the Liberals. The reciprocal character of NDP/Liberal support over time is illustrated in the graphic.


There are other differences: this is Rana Bokhari's first election as Liberal leader while 1988 was the second for Sharon Carstairs. She had already achieved at least modest success in the 1986 Manitoba election including winning a seat in the legislature. Bokhari may not repeat the 1988 success but some gains are likely.

The NDP won the 2011 election in no small part due to some highly successful attacks on the previous PC government's record in office. The dissatisfaction levels they face this time make repeating that effort seemingly impossible. They would be better off going after the provincial Liberals to strengthen the party's position over the longer run.

And the Pallister Tories may be planting the seeds of their own destruction. How many Manitoba voters have noticed the caveat in the PC platform about the sales tax: it won't be reduced immediately but rather as the PC website declares: 'PCs will roll back PST to seven per cent within first mandate'. In other words it will be a four-year wait, and getting there could well involve highly unpopular spending cuts.

The slow growth of recent years has put all governments in Canada under acute fiscal pressures that have yet to let up. There will be no special dispensation for Brian Pallister. After railing for years agains the fiscal record of the NDP he now cautiously says he will work towards balancing the budget, promising only to slow the rate of increase in spending undertaken by the Selinger government.  The sales tax cut must wait.

One paradox of this election is that the Selinger government is not getting the credit a government might expect to receive for the relatively strong growth in the Manitoba economy - perhaps because New Democrats have a reputation as wobbly economic managers. Pallister is going to need this growth to continue and strengthen, if his optimistic plan is to have any hope of succeeding. It appears certain now that he will get his chance.

Governments don't get to stay in office forever, as we saw last year in Alberta. A similar reality is about to confront the Manitoba NDP.  However, if history is any guide, just as the federal Liberals achieved a restoration to power on Parliament Hill after a decade in the wilderness, we might expect to see something similar down the road in Manitoba.

Sunday, February 28, 2016

How do you account for the rise and popularity of Donald Trump?

That was the question posed to me in an email from my younger brother Dave. 

A good place to start an explanation is the off year election in Nebraska in 2014. This solidly safe conservative Republican midwestern state appeared to do something normally associated with liberal Democrats.  The 2014 mid-term election result in Nebraska saw the state overwhelmingly re-elect their Republican Governor and Senator and deliver two out of three Congressional districts to the Republicans (one Omaha based district went barely Democrat). However, Nebraskans also voted by a margin of 59 to 41 percent in a referendum to raise the minimum wage from $7.25 an hour to $8.00. By varying margins four other conservative Republican states did the same. Back in Washington at the same time the Congressional Republicans were rejecting efforts by President Obama to raise the federal minimum wage.

Much of the Republican base is blue collar white working class, but have generally elected Republicans who only paid attention to the economic interests of their billionaire donors and country club suburban voters. With the blue collar base (primarily in the southern, plains and border states) they cultivated its social conservatism and racism. So it should not be surprising that given the chance many of these same voters would directly support a higher minimum wage even as their Washington representatives voted the opposite way. This speaks directly to the internal tensions and contradictions that have long been at the heart of the Republican Party.

Part of Trump's appeal is an economically populist pitch. For example, this quote is from the conservative columnist Byron Yorke of the National Review who, like the rest of his ilk, is appalled by Trump:
In a nearly one-hour speech, Trump railed against pharmaceutical companies. He railed against oil companies. And insurance companies. And defense contractors. And he set himself against a political system that he said allows big-money corporate "bloodsuckers" to control the government with campaign contributions.
"Whether it's the insurance companies, or the drug companies, or the oil companies, it's all the same thing," Trump said. "We're never going to get our country back if we keep doing this."
Trump promised to allow the government to negotiate drug prices — a common position among Democrats but rarely heard at nominally Republican events. He said he would not raise military spending, arguing that the nation's defenses can be improved without increasing its already huge Pentagon budget. He promised tough sanctions on American companies that move jobs overseas.
Trump has attacked one dearly held elite Republican belief after another, including international trade agreements such as the Trans-Pacific Partnership, the so-called TPP, which has been signed by Canada as well as the United States, although neither country has yet formally ratified the deal. Another Trump quote, this time from liberal New York columnist Jonathan Chait, concerning Trump's professed affinity with Bernie Sanders, who also opposes the TPP:
"The one thing we very much agree on is trade. We both agree that we are getting ripped off by China, by Japan, by Mexico, everyone we do business with," Trump said.
However, at the core of Trump's appeal is his outspoken pitch to the intolerance that is characteristic of the Republican party's lower income base - his anti-Muslim and anti-immigration rhetoric, capped by his idiotic proposal to build a wall along the Mexican border that Mexico will allegedly pay for. Trump's bombastic speaking style befits this longtime TV huckster. The important thing about Trump's words is that they have typically gone far beyond the code words and euphemisms about race and immigration that most Republicans have utilized.

Appeals to racism have been a key part of Republican strategy since Nixon concocted his "Southern strategy" in 1968 with former Dixiecrat Strom Thurmond. But the coalition is coming apart at the seams. This is well described in a Nation article by William Greider who notes:
At the heart of this intramural conflict is the fact that society has changed dramatically in recent decades, but the GOP has refused to change with it. Americans are rapidly shifting toward more tolerant understandings of personal behavior and social values, but the Republican Party sticks with retrograde social taboos and hard-edged prejudices about race, gender, sexual freedom, immigration, and religion.
He quotes from a recent article by "Scott Lilly, a liberal Democrat who for many years was the sagacious staff director of the House Appropriations Committee". Lilly wrote the following:
Ever since Kevin Phillips published The Emerging Republican Majority 46 years ago, there has been a serious flaw in the contract that holds the party together. Today’s GOP is made up of two groups that are polar opposites among White Americans. The traditional wing of the party is not simply pro-business but represents a large majority of the country’s corporate leadership, financiers, and investors. Some call these the “country club” Republicans, and while they are a small fraction of the voting population, they have enormous resources to influence the voting behavior of those not privileged to hold a country club membership.
At the opposite end of the economic and educational spectrum of White America is a group that was welcomed into the Republican fold by Richard Nixon in 1968 and 1972. Their numbers make the Republicans a formidable national party. The problem is that this latter group has almost nothing in common with the country club wing of the party. Country clubbers don’t care about prayer in the public schools, gun rights, stopping birth control, abortion, and immigration (i.e. restricting their own ability to hire illegal aliens).
On the other hand, the largely poor, rural, church-going whites who have swelled the party’s electoral turnout don’t care about marginal tax rates, capital formation, or government subsidies to large corporations, such as those provided through the Export-Import Bank. If they ever fully understood that their more prosperous party brethren were contemplating deep cuts in Medicare and Medicaid to pay for those policies, they would be in open rebellion.
In that last sentence we see the connection to the vote on the minimum wage in Nebraska.

The demographic reality that made the "Southern strategy" effective is rapidly changing. The core racism and intolerance of Republican voters explain Trump's appeal, which is made that much more attractive to the base by his economic populism and nationalism. Trump has brilliantly synthesized an appeal that combines a pitch to the Republican rural and blue collar voters' racial and religious intolerance, while simultaneously offering solace to their economic frustrations triggered by stagnating income amid growing inequality.

Many of the southern Evangelical Christians assiduously courted by Trump might have been thought to be offended by stories of his prior support for liberal positions on issues such as abortion, gun control and legalizing drugs. That has not proven to be the case, and part of the reason is the weakening of religion among these Christians who have joined others in reducing their church attendance. Consider the following:
It has long been accepted wisdom that less-educated, working-class white Americans are the nation’s most faithful churchgoers. However, a study released Sunday at the American Sociological Association’s annual convention dispels that widely-held perception. 
Over the past four decades, monthly church attendance by moderately educated whites – defined as those with high school diplomas and maybe some college – has declined to 37 percent from 50 percent, according to the study co-authored by sociologists W. Bradford Wilcox of the University of Virginia and Andrew Cherlin of Johns Hopkins University. 
Church attendance by the least educated whites – defined as those lacking high school diplomas – fell to 23 percent from 38 percent.
It appears as the blogger Digby wrote "that a lot of these white conservative working class types identify as evangelical as much for tribal reasons as religious commitment." This simply makes them open to his economic nationalist message combined with the intolerance.  They easily overlook his 'liberal' past.

While those on the centre and left have been appalled by Trump, so too have shrewd Republican analysts and strategists who are terrified of what he might do to the party's prospects. While Trump appears to be highly intelligent and tactically agile, he has actually conducted a campaign based on winging it, and is wholly unprepared to be President. As Elizabeth Drew has pointed out:
One has the sense that Trump hasn’t made a special effort to bone up on specific issues, that he gets his information from television talk shows and Time. He counts on bluster to propel him. And he adjusts. After the Republican debate preceding the South Carolina primary descended into an out-of-control screaming match, with candidates calling each other liars—John Kasich, who has insisted on a positive campaign, didn’t partake—Trump toned himself down; he also swore off swearing, because it wouldn’t go down well, he was told, with the good people of South Carolina, which has an even higher proportion of Evangelical voters than Iowa. If Trump is the nominee, I wouldn’t be at all surprised if he toned down further, studied up on some issues, and projected an earnestness that would have reporters rhapsodizing about the “New Trump.” What’s astonishing is that someone so intellectually and temperamentally unsuited for the presidency has gotten so close to it.
The question this raises is: how close? Some time ago Rachel Maddow, the MSNBC talk show host who is horrified by Trump, was saying that even if he looked like he would probably lose the election if nominated, she remained anxious because, as she kept repeating, "you never know" what might happen during a campaign to produce an unanticipated outcome.

On the face of it Trump is likely to lose.  As Nate Silver has  pointed out, he is the most unpopular of all the presidential candidates, when you take the opinions of those who view him favourably minus those who have an unfavourable view. But it is not just opinions of the moment that we should consider. Trump and the Republican Party appeal primarily to white voters in a country where demographics are quickly making it more diverse.

In 2012 Obama was re-elected despite getting the "smallest share of white voters of any presidential candidate in history." Blogger Digby quoted from an article by Ron Brownstein that is behind a paywall:
...The key ques­tion fa­cing the GOP is wheth­er Obama’s 2012 per­form­ance rep­res­ents a struc­tur­al Demo­crat­ic de­cline among whites that could deep­en even fur­ther in the years ahead — or a floor from which the next Demo­crat­ic nom­in­ee is likely to im­prove.
In re­cent months, a chor­us of con­ser­vat­ive ana­lysts has bet on the first op­tion. They in­sist that Re­pub­lic­ans, by im­prov­ing both turnout and already-gap­ing mar­gins among whites, can re­cap­ture the White House in 2016 without re­for­mu­lat­ing their agenda to at­tract more minor­ity voters — most prom­in­ently by passing im­mig­ra­tion-re­form le­gis­la­tion that in­cludes a path­way to cit­izen­ship for those here il­leg­ally.
On the oth­er side is an ar­ray of Re­pub­lic­an strategists who view minor­ity out­reach and im­mig­ra­tion re­form as crit­ic­al to restor­ing the party’s com­pet­it­ive­ness — and con­sider it sui­cid­al for the GOP to bet its fu­ture on the pro­spect that it can squeeze even lar­ger ad­vant­ages out of the di­min­ish­ing pool of white voters. Karl Rove, the chief strategist for George W. Bush’s two pres­id­en­tial vic­tor­ies, has noted that re­ly­ing en­tirely on whites would soon re­quire Re­pub­lic­ans to reg­u­larly match the tower­ing ad­vant­age Re­agan re­cor­ded among them when he lost only a single state in his 1984 reelec­tion. “It’s un­reas­on­able to ex­pect Re­pub­lic­ans to routinely pull num­bers that last oc­curred in a 49-state sweep,” Rove said at the As­pen Ideas Fest­iv­al this sum­mer.
No presidential candidate has so clearly articulated his rationale for a whites-only strategy as Donald Trump. Rachel Maddow is right - you never know what unusual event might happen mid-election to propel a Trump into the White House. That qualification aside if Trump wins the nomination the Republican Party is headed for its worst electoral result since 1964. It won't be nearly as bad as 1964 given the gerrymandering of districts in the House of Representatives and the increasingly ideological polarization of the United States as illustratated in this Pew Research graphic:

However, compared to any election since then it is likely to produce worst possible outcome for the Republicans.

And Trump's success to date is due in no small part to his weak opponents.  Evangelical Christian Ted Cruz has a self-defined limit to his appeal while none of the establishment or so-called moderate Republicans including Marco Rubio and Jeb Bush have demonstrated strong political skills. When an unlikely candidate does unexpectedly well, a closer examination of the contest can reveal simply the absence of real competition.

Since their last truly terrible electoral defeat in 1964, more recent Republican losses have been much narrower and can be explained away. Meanwhile the forces in the Republican party opposed to Trump have grown increasingly desperate. The New York Times reported on February 27, the day of the Democratic primary in South Carolina that appeared to confirm that Hillary Clinton would be the party's nominee, they found real despair among the upper echelons of the Republicans:
In dozens of interviews, elected officials, political strategists and donors described a frantic, last-ditch campaign to block Mr. Trump — and the agonizing reasons that many of them have become convinced it will fail. Behind the scenes, a desperate mission to save the party sputtered and stalled at every turn.
The bleak mood of these strategists is entirely justified. Trump's triumph is internal to the Republican Party; it does not extend beyond it.

Wednesday, October 21, 2015

How did the polls fare in Election 2015?

On the whole the polls did well in Election 2015.

Below is a table displaying the difference between the final polls from different firms and the actual preliminary vote count. I subtracted the poll number from actual vote shares. Negative values mean the poll underestimated party support while black represents an overestimate. Angus Reid provided polling numbers representing both eligible voters and those they deemed to be likely voters.



Very few numbers are outside the margin of error specified by the pollsters (note that margin of error does not really apply to online pollsters).

What is striking is that the two most accurate surveys were both conducted on the day before the October 19 election, which strongly suggests that late movement to the Liberals was a significant factor in the outcome. Also note that the likely voter model of Angus Reid performed less well than their eligible voter model. This may partly reflect the higher turnout we saw on election day.

The numbers above deal only with the national scene. However, seat outcomes depend much more on regional numbers, which being based on smaller regional samples are prone to greater error. However, I think the table below suggests that even in the regions the polls did perform well.

Note that Nanos has an advantage here. It would have a slightly smaller total error because it reports on one fewer region (the Prairies rather than treating Manitoba-Saskatchewan as one region and Alberta as another). Nevertheless it is clear Nanos performed well.  I have also omitted the Bloc from this table. The numbers here of necessity are all absolute values (the difference between the regional vote shares and the poll numbers) as it was the only practical way to aggregate them. This is why all are positive values and there is no red as in the table above.

Mainstreet Technologies finished behind Forum and Nanos I think mainly because the poll was finished earlier. However, it had the largest sample size of any pollster at 5,546, partly accounting for a strong overall performance.

My conclusion is that the polls did well this time despite the obvious challenges confronting them. On those challenges, this article by Donna Dasko in the Globe and Mail is a must read for anyone wanting to understand the dilemmas confronting pollsters today.

They did get it right this time but the possibility of a future fiasco as we was in British Columbia in 2013 and Alberta in 2012 remains.

What did not do well were the seat projection models like my own. This is a subject for a future post.





Sunday, October 18, 2015

A Liberal minority is in prospect

We highly personalize politics now and many will attribute a Liberal government, most likely a minority, to Justin Trudeau.  However, his leadership numbers, while they have gone up substantially over the course of the campaign, have trailed party preference. This is neither surprising nor new. It has happened before, notably in 1993 when Jean Chrétien went from yesterday's man to PM over a two month campaign. This result as in 1993 appears signficantly driven by negative considerations on the part of voters. Regardless, Trudeau will become prime minister

Personalities were front and centre yesterday when Stephen Harper attended an event in Toronto organized and sponsored by the Ford brothers, Rob and Doug. Normally the ever astute Harper would avoid relying on the polarizing Ford brothers, but times for him are desperate. He hoped for some boost from the Fords that might pull out a few extra blue collar, low education, low information Tory voters from Ford Nation.  By doing the event he effectively conceded that his Finance Minister Joe Oliver would lose his seat. A significant percentage of Oliver's riding was one of the areas of Toronto that most strongly supported John Tory and its residents have nothing but distaste for the Fords so you know Harper has given up on it. It was symbolic of the campaign closing that Harper's campaigning with the Fords was about despair rather than hope (but it did give the cartoonists a field day).

My estimate of likely seats won based on average of the closing polls is below:

I think all seat projections need some qualifications that can't be quantified. Others such as 308 present seat ranges. I regard such error estimates as meaningless. Probability error for polling results is based on statistical theory (assuming the sample is genuinely random). Error ranges for seat projections make no real sense to me (with one exception I won't go into).

Seat projection is as much art as science and errors are likely to be greater in some circumstances, some of which are present in this election:

  • First, the Liberals are going from third to first and large changes may break in significant ways from previous voting patterns. 
  • Second this election is fundamentally about strategic (better described as tactical) voting - anybody but Harper. Strategic voting may elect a slightly larger number of NDP MPs (all would be incumbents) across the country than the trend would suggest. In my 1999 study on this topic the overall trend suggested just one New Democrat should win. Nine were actually elected. 
  • Third, if micro-targeted strategic voting is truly effective, it may mean fewer Conservatives will win than the trend suggests.
  • Fourth, Quebec broke radically from its past last time and now seems to be experiencing a series of idiosyncratic shifts that may well produce a number of surprising outcomes.
  • Fifth, the Liberal surge is strong enough that it could produce a few perverse effects, letting Conservatives win in a circumstance where a New Democrat might otherwise have been successful and the more appropriate strategic choice.
I have previously written that the campaign hinged on the mid-September niqab announcement.  As a counter-factual what if  we suppose there had been no such development. Instead on the same day the Gagnier story had broken. What might have been the impact? Now I actually I think that from what we know of the Gagnier affair, it was not likely going to have much impact, but lets presume it had been more scandalous than now appears. Might it have overwhelmed the other factors that had helped the Liberals up to that point. We will never know, but it is clear that sheer chance can have an outsized impact on election outcomes. What is not due to chance is the sheer size and scope of the distaste that most Canadians have for the Harper government. That is the most important factor determining the outcome of this election.

Saturday, October 17, 2015

The story of Campaign 2015: The Niqab and Anyone but Harper

A version of this post can be found at iPolitics

The 2015 campaign and its likely outcome can be boiled down to two factors: the Federal Court of Appeal decision on September 15 that allowed Zunera Ishaq to be formally sworn in as a Canadian citizen while wearing a niqab, and the fact that while a significant percentage of the Canadian population was content to see either the NDP or Liberals win, they wanted more than anything to see Stephen Harper leave. They were ABH - Anyone but Harper.

Widespread feelings of antipathy towards Harper have fundamentally framed the choice — but until mid-September it wasn’t clear which of the other two parties was going to be seen as the appropriate alternative. The polls then were telling us that it was a three-way national tie.

That’s where the niqab came in. A part of Quebec’s francophone population, which up to that point was willing to vote NDP, had a strong, visceral, negative reaction to the niqab. It was powerful enough to push large numbers to abandon the NDP for the Bloc and the Conservatives (Take a look at the Bloc's ad on this). Tom Mulcair defended the right of women to choose what they want to wear, and paid a price.

There was an early impact. Any new development takes a few days to register with the electorate (keep this in mind when thinking about the Gagnier affair, and don’t expect to see any significant impact before Sunday in the polls). Shortly after September 15, the NDP rise in Quebec stopped and then started to fall — fast. After a short delay the impact grew and was reinforced by the first French TV debate, but it was clearly visible before. The NDP decline began to be reflected in national polls — again, prior to the debate. The Liberals had the same position on the niqab but managed to escape the backlash largely because their Quebec support is disproportionately anglophone and allophone.

You can see it in the following two graphics — the first is the Quebec polls from September 1 to mid-October. (Note that I use a three-poll moving average to smooth the natural fluctuations you get from polling).



And with a short delay, a gap between the Liberals and the NDP opens up in the national polls.

The gap gave the clear signal to anti-Harper voters. The Liberals and Justin Trudeau have been the clear winners because Quebec turned against the NDP as a consequence over the niqab — a bitter irony for Stephen Harper, who tried to exploit anti-niqab sentiment and had been running anti-Trudeau TV commercials in constant rotation.

There has long been evidence of strong dislike of the Harper government among many Canadians. It’s why you hear so much talk of strategic voting. The expectation of many advocates is that such voting would efficiently topple just enough Tories to do Harper in. Whatever its micro impact in individual constituencies, it’s clear that we’re seeing the effect of strategic anti-Harper positioning on a broad national scale.

But what about the rest of the campaign — is this really all that mattered? There were some Liberal gains on the NDP between mid-August and mid-September in Ontario that could be attributable to their efforts to outflank the NDP on the left, but it might also have been simply a movement back to the pre-Alberta election polling norm. Such movements did occur in Western Canada a little earlier.

The debates (except in Quebec) were generally a fiasco — bad television with tiny audiences largely composed of the politically-committed. Trudeau did exceed expectations in the debates, which helped him in news coverage, while Mulcair fell short. Trudeau had an effective ad (the one on the escalator) while the NDP made the mistake of not advertising early enough — campaigning as a frontrunner when they really weren’t that far ahead.

But the real explanation for the pattern this election has taken is really quite simple: the niqab controversy wound up signalling to Canadians how they could best express their opposition to Harper.

And the 2015 election has been about Harper, his protests to the contrary notwithstanding. Its zeitgeist has been captured well in Michael Harris’s acerbic columns in iPolitics. Canadians can expect to wake up to a very different House of Commons come Tuesday morning.

Sunday, October 11, 2015

Who got it right in 2011: Pollsters and Accuracy

This has been cross posted at iPolitics

It can be difficult at the best of times to make sense of the flood of polling data issuing forth from Internet and airwaves. And there’s a fair amount of social media buzz out there about which pollster is getting it right in this election.

Two of Canada’s major polling firms, EKOS and Nanos, have been reporting different parties in the lead — the Liberals on the part of Nanos over the past week, the Conservatives in the case of EKOS (although the latter’s most recent release gives the Liberals a slight lead that puts them in a statistical tie with the Conservatives, given the margin of error).

So who is getting it right? We won't know for sure until we take the ballots out of the box on Oct. 19; until then, we can do little more than speculate.

What we can do is look back at earlier elections. In 2011, when it came to national numbers, the firm with the fewest errors was Angus Reid, a repeat of its 2008 performance. Reid finished just ahead of Nanos, which in an earlier incarnation as the firm SES had been closest to the mark in the 2006 election.

The table below compares the performance of polling companies in 2011 with respect to national vote shares:



No one pollster gets it right all the time. Compas, no longer providing polling reports, was last on this list in 2011 but the most accurate in 2004 (with Ipsos and Léger just a whisker behind). EKOS ranked relatively low in 2011 (to its credit, EKOS performed a post-election evaluation of its performance). However, EKOS was closest to the election result in 1997, while Environics had that honour in 2000. The title has been widely shared.

One interesting pattern easily discerned from this list is that most firms underestimated how well the Conservatives would do in 2011. On the other hand, Conservative support was widely overestimated in 2004 and Liberal support was underestimated. As a result, we had election night surprises: a Conservative majority in 2011 and a stronger than expected Liberal minority in 2004.
While getting close to the actual result is important to firms, in terms of election outcomes the national number is something of a beauty contest. When it comes to seats, what’s happening in the provinces and regions is what really counts, and it’s difficult and expensive to obtain accurate polling numbers that capture all of Canada’s political diversity.
The table below takes the regional numbers from the pollsters above (except Compas) and then creates an average of errors in the regions for each party. A total lets us compare them. You will see the order of finish is somewhat different.


While the rankings differ from the national picture, that’s not what is most important. Many values are relatively large. In a first-past-the-post system, a small deviation of two or three points one way or the other can have a significant impact on ridings won or lost.
The under-estimate of the Conservative vote led directly to forecasters’ missing the impending Harper majority. Going into voting day, Harper had an average 6.2 percentage point lead over the second place NDP — but wound up with a nine point lead. Of particular significance was a 9.4 percentage point poll lead in Ontario that wound up on voting day as a 19-point lead.
Since 2011 we have had polling fiascos in Alberta in 2012 and British Columbia in 2013. The polls in these two elections created strong expectations of a win by the opposition that never materialized. This year, despite the radical change the Alberta election produced, the polls were generally accurate in forecasting the outcome (Léger Marketing was closest to the mark).
Methodologies have changed and diversified. For example, we have traditional telephone polling from Nanos and Environics, computerized telephone polling from EKOS, Forum Research and others, online surveys from large Internet panels from Ipsos, Abacus, Léger Marketing and Angus Reid and others. Resistance to answering polls has also increased, adding to the variation in results and the uncertainty.
A new Internet methodology that has seen some use in the United States is the Google Consumer Survey, which is a short survey that randomly pops up on computer screens. According to polling guru Nate Silver, a Google survey was the second-most accurate poll in the 2012 presidential election; this could be a glimpse of our polling future.
I am old enough to remember provincial election campaigns with no polls. However, polls have proliferated as never before, as have the individual riding surveys that attempt to drill down into the dynamics of close-fought local races.