September 2012: The inexact art of political polling

A tip of the hat to the legendary New York newspaper columnist Jimmy Cannon for borrowing his signature phrase: nobody asked me, but…

What should we make of the controversy over the presidential election polls? Some conservative pundits are claiming that mainstream media pollsters are overestimating the potential vote for President Obama. Is there a calculated overweighting of Democrats in these polls? Can we rely on any of these polls?

The truth is that polling is a very inexact art (not a science). All public opinion polls should be taken with a mountain (not a grain) of salt.

What very few in the polling business want to admit: a number of subjective decisions (size of sample, weighting methodology, response rate, question phrasing, question order, etc.) that determine what a given poll’s results will look like.

The furor over the presidential polls showing Obama opening up a lead over Governor Romney in several swing states, and well as nationally, does reflect, in part, decisions made by pollsters. But I doubt that there’s any special liberal “home-cooking” going on.

Current political polls are already deeply flawed.

Here are five reasons why the polls should be regarded with great skepticism:

  • Pollsters are reaching fewer and fewer Americans, making it harder to assemble a truly representative random sample. The Pew Research Center admits it gets less than 10 percent of those they contact answering their questions, down from 36 percent in 1997. The other major pollsters have the same problem, and heavy cell phone usage makes it even harder to find the correct demographic mix.
  • Respondents may not be revealing their true views. The “Shy Tory Effect”—where conservatives are less likely to tell pollsters what they really believe&,dash;acts to inflate projected vote totals for more liberal candidates and causes.
  • Questionable survey design&mdeash;where the wording of questions or the ordering of the questions influence response—can skew results.
  • In close political races, the margin of error—which is often one or two percentage points— often makes predicting a winner impossible. A coin toss will have the same level of reliability.
  • In the presidential race, the difficulty in figuring out who will actually vote in November introduces significant subjectivity. It accounts for some polls showing Obama winning a state and others showing Romney winning the same state. If the pollster believes turnout will be like the presidential election of 2008, when minorities and younger voters surged to the polls, an Obama victory is likely. If a pollster believes turnout will be like 2004, with more GOP partisans showing up, then a Romney victory is likely.

In the end, the only poll that matters is the one on Tuesday, November 6th. That’s the one that counts. Debating the validity of public opinion surveys before then may energize partisans, but for the rest of us, it appears to be quite a waste of time. And don’t expect the exit polls to be any better—the flaws in the way they are conducted make them unreliable, to say the least.

In any event, we’ll know the actual winners soon enough.


Copyright © 2012 Jefferson Flanders
All rights reserved

Purchase the new Cold War thriller by Jefferson Flanders, Herald Square, at Amazon.com.

Read an interview with the author.

Advertisements