How voting intentions evolved: A view from Quebec
Citizens and pundits not infrequently complain that there are too many polls during election campaigns. But the large volume of polls has many advantages. First, people do not have to rely on a single source of information on the state of public opinion. When polls give different readings of the situation, we are able to put the numbers into perspective and are reminded that the margin of error in polls is real. Second, researchers and voters can trace the evolution of public opinion throughout the campaign, and get an objective measure to set against politicians’ claims as to the level of support for their parties or policies.
The polls showed substantial shifts in voting intentions in the course of the recent federal election campaign, especially in Quebec. In the end, the polls predicted the results quite accurately for some parties, but not for others. In what follows, I look at the apparent evolution of voting intentions, analyze the discrepancy between the polls and the actual vote, and comment on the polling methods used.
In Quebec, where it all started …
To trace the apparent evolution of voting intentions, we assembled the published estimates of 58 polls1 conducted in the 53 days between August 20, two and a half weeks before the election was called, and October 11, three days before the election. The results of these polls were then spread over the days when they were in the field, and we took the average of all the polls that were in the field on a given day. We used this information to trace the apparent evolution of the polls using time-series analysis.
Figure 1 displays what these analyses tell us about the evolution of support for the different parties in Quebec.2 At the beginning, the Bloc had a narrow lead with a little over 30 per cent, while the Conservatives and the Liberals were virtually tied at about 25 per cent and the NDP was trailing at about 12 per cent. By early September, voting intentions for the Liberals began to drop to the benefit of the Conservatives. In mid-September, after one week of campaign, the Conservatives were in a tie with the Bloc at around 30–32 per cent. At that time, it seemed to observers that anything was possible and that the Conservatives could “take” the province.
What happened then? From around September 16 to the French debate on October 1, the Conservatives lost close to 10 percentage points, almost entirely to the Bloc. When the debate took place, the Bloc was almost 20 points ahead of both the Liberals and the Conservatives. The debate did not change this state of affairs much, except that the Liberals managed to get slightly ahead of the Conservatives. Meanwhile, after gaining a few percentage points in voting intentions, the NDP fell back to its pre-campaign level.
Observers have attributed the evolution of voting intentions in Quebec to some of the decisions of the Harper government – cuts in some cultural programs, proposed changes to the sentencing of youth offenders – or to a reaction to the “costs of the Bloc” campaign led by high-profile Tory candidate Michael Fortier. However, the statistical analyses that I performed do not provide confirmation of these hypotheses, which would require an analysis of what media and party spokespersons in different regions said over time.
Meanwhile, elsewhere in Canada …
According to the polls, before the election was officially called, voting intentions for the Conservatives had risen quite substantially in Ontario, while those for the Liberals and the NDP were falling. When the election was called, however, the polls show a rise in Liberal support of more than seven points and a decline in Conservative support of more than eight points. As a result, throughout the campaign until the debate, the two parties were effectively tied in Canada’s largest province – no movement could be traced in voting intentions. However, the debate may have had an impact on the Conservatives’ fortunes, since the polls show that following the debate they lost 3.1 percentage points, which went almost equally to the Liberals and the NDP.
The evolution for the Conservatives and Liberals in Canada as a whole looks similar to what it was in Ontario, except that the timing is different. After reaching almost 40 per cent in voting intentions at the beginning of the campaign, the Conservatives seemed to lose ground, finishing at 33 per cent (the forecast from the time-series analysis). Meanwhile, after going down from 35 per cent in August, voting intentions for the Liberals did not evolve much after the start of the campaign, staying at around 26 per cent.
The meaning of the overall trend in voting intentions emerges only when we focus on the regions, since the level of support for the different parties varies from region to region. Moreover, election campaigns are not always the same. During the previous federal election campaign, in late 2005 and early 2006, Ontario set the pace for evolution of voting intentions: there were shifts in Ontario from the beginning of the campaign, while in Quebec voting intentions shifted only in the last two weeks. This time, the impression that the Conservatives were losing ground in Canada was largely a reflection of their losing ground in Quebec.
What happened in the end?
At the end, the analyses for Quebec forecast the Bloc at 43.4 per cent (with a confidence interval of 40.1 to 46.7 per cent), the Liberals at 21.8 (19.1 to 24.6), the Conservatives at 16.0 (12.5 to 19.6) and the NDP at 11.2 (8.6 to 13.8). The confidence intervals are quite large to take into account possible shifts in voting intentions during the last days of the campaign. While the Liberal share of the vote (23.7 per cent) and the NDP share (12.2 per cent) were well predicted, the Conservatives’ vote (21.7 per cent) was underestimated by 5.7 points; they ended with 2.1 points more than the upper limit of the confidence interval. Conversely, the Bloc was overestimated by 5.3 points, and they ended 2.0 points under the lower limit of the confidence interval. In Ontario, both the Conservatives (at 39.2 per cent) and the Liberals (33.8 per cent) were well estimated by time-series analyses of the polls, but the NDP was clearly overestimated. It received 18.2 per cent of the vote while it was forecast at 23.6 (with a confidence interval between 22.2 and 24.9). In Canada as a whole, while the NDP, the Bloc and the Liberals were well estimated, the Conservatives, with 37.6 per cent of the vote, were underestimated by 4.9 per cent; they ended up with 2.3 points more than the upper bound of the confidence interval. This casts doubt on the real evolution of voting intentions for the Conservatives. Did they really lose ground, or did Conservative supporters hide their opinion from the pollsters?
When polls go wrong, they usually go wrong in a specific direction, underestimating the right side of the political spectrum and overestimating the left side. A number of hypotheses have been offered to explain this. One is Elisabeth Noelle-Neumann’s model of the “spiral of silence,”3 which states that, when a political party is “unpopular” – as seen, for example, in the media – people who want to vote for that party hide their voting intentions. As a result, the party then falls even lower in the polls and its supporters tend to hide their intentions even more – hence the “spiral.” However, the phenomenon could also stem from the fact that right-wing voters’ characteristics correspond to those of people who are harder to reach by pollsters and less likely to reveal their voting intentions. For example, they are likely to be older. French political scientists speak of the paradox of the “Communist grandmother”: older respondents who reveal left-wing voting intentions but are unrepresentative.
Another interpretation, especially within the Bloc Québécois, is that the discrepancy is due to the Bloc’s inability to get its supporters out to vote. But earlier election data do not bear out this hypothesis. An analysis of a similar situation in the Quebec election of 1998, for which polls had overestimated the Parti Québécois’s share by four points, has shown that Parti Québécois supporters are at least as likely to show up to vote as supporters of other parties.4 This is the case generally for parties leaning to the left. In addition, in this election, Quebec had one of the highest participation rates in Canada: at 61.1 per cent, it was two points higher than in Ontario and in Canada as a whole.
A challenge for pollsters
Overall, polls were accurate in their prediction of the Liberal share of the vote, but not in their prediction of the Conservative share both overall and in Quebec, of the NDP share in Ontario or of the Bloc share in Quebec. This election posed a particular challenge for pollsters: there were late shifts in voting intentions during the last days of the campaign in response to the debate, and the existence of numerous left-centre parties likely induced strategic voting at the last minute. Indeed, under these circumstances, the polls themselves may have had an impact on voting intentions, though probably not a substantial one.
Finally, a word about methods, as this election was characterized by the appearance of new methods to conduct polls. Angus Reid used Internet polling, while Ekos used IVR (interactive voice recognition) where respondents are polled via automated phone calls. As a rule, the pollsters are generally very transparent and give precise and clear information on the methods they use. Since these methods are new to polling in Canada, one would have expected the media to be careful and state clearly what type of methodology was used. Unfortunately – and in contravention of section 326 of the Canada Elections Act – most of the time, Internet and IVR polls were presented and interpreted without any mention of the method used. However, in the end, polls conducted over the Internet and using IVR did not differ significantly from polls carried out with more traditional methods using the telephone.
Notes
1 The tracking/rolling polls are entered according to the number of days on which they are based. For example, a three-day rolling poll is entered in the database for each of the three days.
2 The models used are Arima models that control for autocorrelation of errors from one day to the next. All the graphs and results of the analyses may be found at https://www.webdepot.umontreal.ca/Usagers/durandc/MonDepotPublic/Canada2008
3 Elisabeth Noelle-Neumann, The Spiral of Silence: Public Opinion – Our Social Skin (Chicago: University of Chicago Press, 1984) and “The Spiral of Silence: A Theory of Public Opinion,” Journal of Communication, Vol. 24 (1974), pp. 43–51
4 Claire Durand, André Blais and Sébastien Vachon, “Accounting for Biases in Election Surveys: The Case of the 1998 Quebec Election,” Journal of Official Statistics, Vol. 18, No. 1 (2002), pp. 25–44.
