While early exit polls showed Sen. Barack Obama leading in the Pennsylvania Democratic presidential primary, and broadcast and cable news networks consequently delayed calling a winner in the election, when the actual ballots were counted Sen. Hillary Clinton had carried the Keystone State by some 9 percentage points.
Some pundits quickly suggested that the gap between the exit polls and the final tally reflected the Bradley Effect—white voters telling pollsters they had voted for the black candidate, when in fact they had not (an effect first identified in California’s 1982 gubernatorial election lost narrowly by Tom Bradley, an African-American—hence the name.) It wasn’t the first time in Campaign 2008 that Obama’s strong unweighted exit poll numbers did not translate into actual votes—the Illinois Senator had “underperformed” in New Hampshire and in several Super Tuesday states, according to a compilation of early exit polls by Brendan Loy. Loy further noted that: “… Obama generally does 7-8 points worse in the actual results than he did in the leaked, unweighted exit polls.”
But it’s unlikely that the color of the candidates caused the exit poll problems. Instead, it appears that the Shy Tory Factor influenced the exit polls in Pennsylvania, a global phenomenon that has surfaced in numerous past elections where race wasn’t a consideration. The Shy Tory Factor is when conservative voters provide misleading answers to pollsters or refuse to participate in exit polls (where it is called “non-response bias” by pollsters). It has been seen in elections in England, France, Italy, Australia and the U.S. More conservative candidates perform better at the ballot box than they do in pre-vote polls and exit polling. (Reuters, for example, noted that “exit polls have not always proved reliable in Italy” in reporting on the recent election of conservative candidate Silvio Berlusconi.)
What’s behind it? The prevailing theory is that these Shy Tory, or Shy Conservative, voters opt out of polling, or offer misleading answers, because they don’t view the elite media, who sponsor the opinion and exit polls, as truly neutral. They realize that their candidate (Berlusconi, Margaret Thatcher, Ronald Reagan, George W. Bush, John Howard) is not the choice of liberal reporters or mainstream commentators and consequently they are more reluctant to share their preference with intrusive pollsters.
The practical effect of the Shy Tory Factor is to skew poll results. Take Pennsylvania. While Clinton supporters may not be “Tories” in ideological terms, they are older, less-educated, and more likely to resent the media anointment of Obama. Their motives for refusing match the Shy Tory model. If these Clinton voters shied away from exit polls, it means backers of the other candidate (Obama) were oversampled. The Clinton-Obama race had further complications. Refusal rates for exit polls are historically greater among older voters to start with. Most exit poll takers are young (students, etc.), and it would not be surprising if—despite being trained to avoid interviewer-caused selection bias—these temporary workers gravitated to polling younger voters, who have continued to favor Obama.
There is, of course, a third possible explanation for the disconnect between the exit polls and the tabulated vote—that of election fraud. After the New Hampshire primary, some on the Left suggested that Sen. Clinton’s victory involved rigged voting machines, and others (such as posters on The Brad Blog and TruthDig ) have questioned the validity of the Pennsylvania primary as well.
A common misunderstanding about the accuracy of exit polls has contributed to these conspiracy theories. (“Mystery Pollster” Mark Blumenthal has researched exit polling’s historical inaccuracy.) They simply aren’t a valid way to audit elections. For starters, exit polls carry a margin of error—supposedly about 3 percent in national elections, when all else goes well, and higher in primaries. And like all polling, exit polls rely on a representative sample of voters that is projected to all voters (which is what the “weighting” process is all about). As can be seen with the Shy Tory Factor, when given voters won’t participate, it skews the sample. (See this interview with Joe Lenski of Edison Media Research for a frank assessment of the problems with exit poll samples). The exit poll refusal rate has been growing in the U.S. It was an average 35% nationwide in the 2004 presidential election and is higher for older voters and in more conservative areas of the country.
These flaws were ignored by Democratic activists and bloggers when, as evidence of fraud, they pointed to those pivotal states where exit polls had projected Sen. John Kerry as the winner but where President Bush triumphed when the actual votes were tallied. In explaining the discrepancy, Edison/Mitofsky Research (the firm that conducted the 2004 exit polls) concluded that Republican voters had refused to participate in exit polling in greater numbers than Democrats, leading to an overestimation of the Kerry vote totals. Further voter sampling problems surfaced in the 2006 Congressional election exit polls.
None of this will, however, convince the conspiracy buffs who believe that the Clinton machine—borrowing alleged Republican tactics—manipulated the primary voting process in states like New Hampshire and Pennsylvania by a conspiracy targeting electronic voting machines. Such a vote fraud enterprise in practice would require the complicity of hundreds of election officials, computer technicians, etc., spread across numerous precincts, and would also demand a breathtaking level of coordination and planning. And everyone involved would be committing numerous felonies as well. But if you believed the Republicans were capable of such crimes in 2004, it’s not as hard to believe that the Clinton campaign would engage in vote fraud as well.
A media creation
There are some ironies in these exit poll problems. Exit polls, after all, are a media creation. They allow network anchors and political commentators to pontificate about voter preferences and beliefs. They allegedly tell us how given groups (whites, blacks, Hispanics, liberals, conservatives, Catholics, Jews) voted, and why they voted the way the way they did (contributing, one could argue, to the public practice of identity politics). If media cheerleading for Obama has increased exit poll refusal rates among Clinton voters, then the lack of balance in the coverage of the Democratic race in 2008 has contributed to the margin of error in these surveys.
The Edison/Mitofsky Research folks don’t like to talk about refusal rates, because they know it raises questions about the validity of their exit polls. A strong argument can be made based on the 2004, 2006, and 2008 results that the accuracy of exit polls has been so compromised that they should be abandoned as an analytical tool in political news coverage. And other than a few media executives and polling firms, who would be sorry to see them go?
Copyright © 2008 Jefferson Flanders
All rights reserved