January 2009: Reconsidering the Scott Brown phenomenon, drone war questions, and other observations

A tip of the snowy ski cap to the late, great New York newspaper columnist Jimmy Cannon for borrowing his signature phrase: nobody asked me, but…


In the past, the Massachusetts electorate has opted for divided government, electing budget-conscious Republican governors (e.g., William Weld, Mitt Romney) to counterbalance tax-and-spend Democrats in the state legislature.

In the special election for the Senate seat, Brown’s reform message appealed to many voters disgusted at the perceived arrogance and entitlement of Democratic politicians on Beacon Hill and in Washington. Brown understood these populist sentiments. When CNN’s David Gergen asked Brown during the crucial final senatorial debate whether he was willing to “sit in Teddy Kennedy’s seat” and block health care reform, Brown responded:

Well, with all due respect it’s not the Kennedys’ seat, and it’s not the Democrats’ seat, it’s the people’s seat. And they have the chance to send somebody down there who is an independent voter, and an independent thinker and going to look out for the best interests of the people of Massachusetts.

Brown stressed his independence during the campaign. While he didn’t disguise his center-right positions on national security issues, government spending, and the Obama health care plan, Brown was careful to note that he supported the status quo on abortion and gay marriage (legal in Massachusetts). Brown’s mainstream views on social issues matched those of independents, the largest group of voters in the state.

It’s true that Brown’s Democratic opponent, Martha Coakley, ran a spectacularly inept campaign, coming across as a chilly elitist with no political instincts (offending Catholics and Red Sox fans with dismissive comments in the last weeks of the contest). Former Boston Globe columnist Mike Barnicle had the best line about Coakley’s sorry performance: “She has approached the public with the demeanor of a substitute teacher with little interest in her students’ lives.” Yet even a more competent Democrat candidate would have been hard-pressed to beat the charismatic Brown.

THE BROWN-COAKLEY RACE HIGHLIGHTED CONTINUED PROBLEMS WITH THE ACCURACY OF POLITICAL POLLING. The founding director of the Associated Press polling unit, Mike Mokrzycki, suggested that Brown’s surprising numbers in pre-election polls might be inflated by the willingness of his supporters to answer pollster’s questions. Mokrzycki thought Coakley’s “silent supporters” might close that gap when real ballots were cast on Election Day. As it happened, there was no late surge for Coakley.

Mokrzycki’s theory conflicted with recent polling trends; it’s been conservatives who have been reluctant to divulge their political opinions to pollsters in what has been dubbed the Shy Tory Factor. As Mokrzycki conceded, non-response rates for telephone polls has dropped below 10% and a response rate greater than 20% is “extraordinarily good,” so any segment of the electorate that is refusing to participate can skew poll results. Mokrzycki and other professional pollsters claim they are still drawing properly-weighted random samples and their findings haven’t been distorted. Recent electoral history would suggest otherwise.

THE ACLU HAS QUESTIONED THE LEGAL BASIS FOR PRESIDENT OBAMA’S USE OF PREDATOR DRONES, a challenge that I predicted was imminent in an essay for Washington Decoded (“Drone Wars“). In a Freedom of Information Act (FOIA) request filed January 14, the ACLU asked for data on the targeted killings by drone, seeking to find out “when, where and against whom drone strikes can be authorized, and how the United States ensures compliance with international laws relating to extrajudicial killings.”

Expect more of these challenges to come from the Left and members of the international law community. How the Justice Department responds will be fascinating, for the Obama Administration’s legal approach to the war on terror has been confused and riddled with contradictions.

IS IT POSSIBLE TO PREDICT, IN ADVANCE, WHO WILL BE A GREAT TEACHER? Amanda Ripley, in a fascinating Atlantic Monthly piece (“What Makes a Great Teacher“) looks at the data collected by Teach for America on teacher effectiveness. What Teach for America found are the qualities that make for great teachers include: a history of perseverance, a zest for life, and a track record of leadership and achievement.

THIS MONTH’S WORDS OF WISDOM COME FROM IRISH ORATOR, STATEMAN, AND POLITICAL PHILOSOPHER EDMUND BURKE (1729-1797): “Your representative owes you, not his industry only, but judgment; and he betrays, instead of serving you, if he sacrifices it to your opinion.”

Copyright © 2010 Jefferson Flanders
All rights reserved


Exit poll error? It’s the Shy Tory Factor, not the Bradley Effect

While early exit polls showed Sen. Barack Obama leading in the Pennsylvania Democratic presidential primary, and broadcast and cable news networks consequently delayed calling a winner in the election, when the actual ballots were counted Sen. Hillary Clinton had carried the Keystone State by some 9 percentage points.

Some pundits quickly suggested that the gap between the exit polls and the final tally reflected the Bradley Effect—white voters telling pollsters they had voted for the black candidate, when in fact they had not (an effect first identified in California’s 1982 gubernatorial election lost narrowly by Tom Bradley, an African-American—hence the name.) It wasn’t the first time in Campaign 2008 that Obama’s strong unweighted exit poll numbers did not translate into actual votes—the Illinois Senator had “underperformed” in New Hampshire and in several Super Tuesday states, according to a compilation of early exit polls by Brendan Loy. Loy further noted that: “… Obama generally does 7-8 points worse in the actual results than he did in the leaked, unweighted exit polls.”

But it’s unlikely that the color of the candidates caused the exit poll problems. Instead, it appears that the Shy Tory Factor influenced the exit polls in Pennsylvania, a global phenomenon that has surfaced in numerous past elections where race wasn’t a consideration. The Shy Tory Factor is when conservative voters provide misleading answers to pollsters or refuse to participate in exit polls (where it is called “non-response bias” by pollsters). It has been seen in elections in England, France, Italy, Australia and the U.S. More conservative candidates perform better at the ballot box than they do in pre-vote polls and exit polling. (Reuters, for example, noted that “exit polls have not always proved reliable in Italy” in reporting on the recent election of conservative candidate Silvio Berlusconi.)

What’s behind it? The prevailing theory is that these Shy Tory, or Shy Conservative, voters opt out of polling, or offer misleading answers, because they don’t view the elite media, who sponsor the opinion and exit polls, as truly neutral. They realize that their candidate (Berlusconi, Margaret Thatcher, Ronald Reagan, George W. Bush, John Howard) is not the choice of liberal reporters or mainstream commentators and consequently they are more reluctant to share their preference with intrusive pollsters.

The practical effect of the Shy Tory Factor is to skew poll results. Take Pennsylvania. While Clinton supporters may not be “Tories” in ideological terms, they are older, less-educated, and more likely to resent the media anointment of Obama. Their motives for refusing match the Shy Tory model. If these Clinton voters shied away from exit polls, it means backers of the other candidate (Obama) were oversampled. The Clinton-Obama race had further complications. Refusal rates for exit polls are historically greater among older voters to start with. Most exit poll takers are young (students, etc.), and it would not be surprising if—despite being trained to avoid interviewer-caused selection bias—these temporary workers gravitated to polling younger voters, who have continued to favor Obama.

Vote fraud?

There is, of course, a third possible explanation for the disconnect between the exit polls and the tabulated vote—that of election fraud. After the New Hampshire primary, some on the Left suggested that Sen. Clinton’s victory involved rigged voting machines, and others (such as posters on The Brad Blog and TruthDig ) have questioned the validity of the Pennsylvania primary as well.

A common misunderstanding about the accuracy of exit polls has contributed to these conspiracy theories. (“Mystery Pollster” Mark Blumenthal has researched exit polling’s historical inaccuracy.) They simply aren’t a valid way to audit elections. For starters, exit polls carry a margin of error—supposedly about 3 percent in national elections, when all else goes well, and higher in primaries. And like all polling, exit polls rely on a representative sample of voters that is projected to all voters (which is what the “weighting” process is all about). As can be seen with the Shy Tory Factor, when given voters won’t participate, it skews the sample. (See this interview with Joe Lenski of Edison Media Research for a frank assessment of the problems with exit poll samples). The exit poll refusal rate has been growing in the U.S. It was an average 35% nationwide in the 2004 presidential election and is higher for older voters and in more conservative areas of the country.

These flaws were ignored by Democratic activists and bloggers when, as evidence of fraud, they pointed to those pivotal states where exit polls had projected Sen. John Kerry as the winner but where President Bush triumphed when the actual votes were tallied. In explaining the discrepancy, Edison/Mitofsky Research (the firm that conducted the 2004 exit polls) concluded that Republican voters had refused to participate in exit polling in greater numbers than Democrats, leading to an overestimation of the Kerry vote totals. Further voter sampling problems surfaced in the 2006 Congressional election exit polls.

None of this will, however, convince the conspiracy buffs who believe that the Clinton machine—borrowing alleged Republican tactics—manipulated the primary voting process in states like New Hampshire and Pennsylvania by a conspiracy targeting electronic voting machines. Such a vote fraud enterprise in practice would require the complicity of hundreds of election officials, computer technicians, etc., spread across numerous precincts, and would also demand a breathtaking level of coordination and planning. And everyone involved would be committing numerous felonies as well. But if you believed the Republicans were capable of such crimes in 2004, it’s not as hard to believe that the Clinton campaign would engage in vote fraud as well.

A media creation

There are some ironies in these exit poll problems. Exit polls, after all, are a media creation. They allow network anchors and political commentators to pontificate about voter preferences and beliefs. They allegedly tell us how given groups (whites, blacks, Hispanics, liberals, conservatives, Catholics, Jews) voted, and why they voted the way the way they did (contributing, one could argue, to the public practice of identity politics). If media cheerleading for Obama has increased exit poll refusal rates among Clinton voters, then the lack of balance in the coverage of the Democratic race in 2008 has contributed to the margin of error in these surveys.

The Edison/Mitofsky Research folks don’t like to talk about refusal rates, because they know it raises questions about the validity of their exit polls. A strong argument can be made based on the 2004, 2006, and 2008 results that the accuracy of exit polls has been so compromised that they should be abandoned as an analytical tool in political news coverage. And other than a few media executives and polling firms, who would be sorry to see them go?

Copyright © 2008 Jefferson Flanders

All rights reserved

Add to Technorati Favorites!

Campaign 2008’s troubled opinion polls

Can’t say I envy political pollsters during this topsy-turvy Campaign 2008. Who likes to be embarrassed on a national (if not global) stage by getting it wrong?

All of the major pre-vote polls predicted a Barack Obama victory in Tuesday’s Democratic New Hampshire presidential primary. All were wrong, as Sen. Hillary Clinton confounded conventional political wisdom and predictions of pundits and campaign insiders and won, 39% to 37%, over Sen. Obama.

One explanation offered by many in the “political industrial complex”, such as Frank Newport of Gallup and long-time political researcher Peter Hart, is that pollsters stopped surveying too early, and missed a late shift by women voters and independents to Clinton. The historical precedent for this last-minute decision scenario: the presidential election of 1948 (“Dewey Defeats Truman”), when polling shut down prematurely and missed Harry Truman’s late surge.

The truth is that we don’t know for sure what happened. While most of the polls correctly predicted the outcome of the Republican primary (Sen. John McCain over former Gov. Mitt Romney), the predicted margins of victory for Sen. McCain varied widely, and one poll (Suffolk-WHDH) picked Romney as the GOP winner.

The flawed New Hampshire pre-vote polls suggest that Campaign 2008 may challenge the ability of opinion polls to predict trends or electoral outcomes with any certainty.

Here are five factors that will spell trouble in the months ahead for anyone looking to rely on political opinion polls.

1. Traditional turnout models may not be able to predict who will show up to vote in a non-traditional year when the candidacies of a woman and an African-American are drawing new “wildcard” voters to the polls (a point first made by Frank Luntz, pollster for Fox News). Gauging the intentions of prospective voters is very difficult; in the past the methodology has relied on a series of predictors (such as homeownership and voting in the last election) that may no longer be as valid.

2. Round-the-clock news coverage has made for a more volatile political environment, as prospective voters are quickly made aware of any campaign developments, positive or negative. The 24X7 news cycle of the Internet and the cable news networks means that voters can be exposed to potential “decisive moments,” like Sen. Clinton’s emotional Portsmouth cafe cry, in the last days and hours of a race. If the 2008 primaries feature more last-minute voter decisions, it will be hard for polls to reflect his new reality.

3. Perceived media cheer-leading for the Obama and Clinton candidacies because of their historic nature (the first credible female and African-American presidential candidates) may make some voters hesitant to reveal their true intentions. This is a variation of the Shy Tory or Shy Republican Factor, where voters won’t admit to pollsters that they plan to vote for conservative candidates. (Right-of-center politicians, such as Margaret Thatcher, Ronald Reagan, John Howard and Silvio Berlusconi, have often performed better in the election than in pre-vote polls.)

4. The growing Do Not Call ethos in many American households—which has led to call screening, unlisted numbers, and a general reluctance to answer phone surveys—makes it harder to assemble a representative sample. (While federal law allows political and survey calling, it doesn’t mean such intrusion are welcomed). And reaching those prospective voters who rely solely on cell phones (often younger voters), an estimated 15% of Americans, is another emerging challenge.

5. A growing It’s None of Your Business backlash may be in play. Response rates for political polls have been dropping. Telephone response rates have dropped from about 70% in the 1970s to 30% today (although pollsters insist survey quality hasn’t declined). Consider the refusal rate for exit polls: it’s much greater in the United States than in other countries (I’ve seen non-participation numbers ranging from 10% to 40% of voters, depending on the election). In the 2004 Presidential election, Republican voters refused to participate in exit polling in greater numbers than Democrats; most likely another manifestation of the Shy Tory effect, as some GOP voters may have regarded the exit polls as an extension of the “liberal media.” In fact, the predictive shakiness of the 2004 and 2006 national exit polls has been linked to problems with voter sampling.

What these factors mean for Campaign 2008: the polls, whether pre-election or exit, should be treated with great caution by citizens, pundits, journalists, and political observers. The Democratic New Hampshire primary results should serve as a warning to anyone relying on what is more muddled art than precise science.


Copyright © 2008 Jefferson Flanders
All rights reserved