Can’t say I envy political pollsters during this topsy-turvy Campaign 2008. Who likes to be embarrassed on a national (if not global) stage by getting it wrong?
All of the major pre-vote polls predicted a Barack Obama victory in Tuesday’s Democratic New Hampshire presidential primary. All were wrong, as Sen. Hillary Clinton confounded conventional political wisdom and predictions of pundits and campaign insiders and won, 39% to 37%, over Sen. Obama.
One explanation offered by many in the “political industrial complex”, such as Frank Newport of Gallup and long-time political researcher Peter Hart, is that pollsters stopped surveying too early, and missed a late shift by women voters and independents to Clinton. The historical precedent for this last-minute decision scenario: the presidential election of 1948 (“Dewey Defeats Truman”), when polling shut down prematurely and missed Harry Truman’s late surge.
The truth is that we don’t know for sure what happened. While most of the polls correctly predicted the outcome of the Republican primary (Sen. John McCain over former Gov. Mitt Romney), the predicted margins of victory for Sen. McCain varied widely, and one poll (Suffolk-WHDH) picked Romney as the GOP winner.
The flawed New Hampshire pre-vote polls suggest that Campaign 2008 may challenge the ability of opinion polls to predict trends or electoral outcomes with any certainty.
Here are five factors that will spell trouble in the months ahead for anyone looking to rely on political opinion polls.
1. Traditional turnout models may not be able to predict who will show up to vote in a non-traditional year when the candidacies of a woman and an African-American are drawing new “wildcard” voters to the polls (a point first made by Frank Luntz, pollster for Fox News). Gauging the intentions of prospective voters is very difficult; in the past the methodology has relied on a series of predictors (such as homeownership and voting in the last election) that may no longer be as valid.
2. Round-the-clock news coverage has made for a more volatile political environment, as prospective voters are quickly made aware of any campaign developments, positive or negative. The 24X7 news cycle of the Internet and the cable news networks means that voters can be exposed to potential “decisive moments,” like Sen. Clinton’s emotional Portsmouth cafe cry, in the last days and hours of a race. If the 2008 primaries feature more last-minute voter decisions, it will be hard for polls to reflect his new reality.
3. Perceived media cheer-leading for the Obama and Clinton candidacies because of their historic nature (the first credible female and African-American presidential candidates) may make some voters hesitant to reveal their true intentions. This is a variation of the Shy Tory or Shy Republican Factor, where voters won’t admit to pollsters that they plan to vote for conservative candidates. (Right-of-center politicians, such as Margaret Thatcher, Ronald Reagan, John Howard and Silvio Berlusconi, have often performed better in the election than in pre-vote polls.)
4. The growing Do Not Call ethos in many American households—which has led to call screening, unlisted numbers, and a general reluctance to answer phone surveys—makes it harder to assemble a representative sample. (While federal law allows political and survey calling, it doesn’t mean such intrusion are welcomed). And reaching those prospective voters who rely solely on cell phones (often younger voters), an estimated 15% of Americans, is another emerging challenge.
5. A growing It’s None of Your Business backlash may be in play. Response rates for political polls have been dropping. Telephone response rates have dropped from about 70% in the 1970s to 30% today (although pollsters insist survey quality hasn’t declined). Consider the refusal rate for exit polls: it’s much greater in the United States than in other countries (I’ve seen non-participation numbers ranging from 10% to 40% of voters, depending on the election). In the 2004 Presidential election, Republican voters refused to participate in exit polling in greater numbers than Democrats; most likely another manifestation of the Shy Tory effect, as some GOP voters may have regarded the exit polls as an extension of the “liberal media.” In fact, the predictive shakiness of the 2004 and 2006 national exit polls has been linked to problems with voter sampling.
What these factors mean for Campaign 2008: the polls, whether pre-election or exit, should be treated with great caution by citizens, pundits, journalists, and political observers. The Democratic New Hampshire primary results should serve as a warning to anyone relying on what is more muddled art than precise science.
Copyright © 2008 Jefferson Flanders
All rights reserved