Sunday, October 21, 2012

The Imaginary World of Polls

In the build up to the American Presidential election, we're hearing a lot about the impact that the debates are having in the polls, how in day to day polls, the lead that Obama has is increasing slightly or decreasing slightly, with pollsters giving the reasoning behind this based on the performance (or lack of) in the Presidential debates, or due to Romney's behind the scenes comments.

The Irish Times reported yesterday that "46% of likely voters said they would vote for Obama in the election, while 45% said they would back Romney. Yesterday, Obama was ahead by three percentage points at 46% compared to 43% for Romney."

Interesting stats right? Might we infer that Romney's strong performance in the first debate and solid performance in the second debate has seen him make up ground? Should Obama be worried?

Unfortunately for the pollsters, their little polls have a problem. They aren't accurate to infer anything of the sort.
"The precision of Reuters/Ipsos online polls is measured using a credibility interval. In this case, the poll has a credibility interval of plus or minus 3.0 percentage points for likely voters."
So any gain that we have seen from Romney could be completely (or mostly) due to the fact that the polls have a margin of error of plus or minus 3.0 percentage points. Of course that's not as interesting as declaring a tightening in the race, or assigning causation between what's happening in the race and what's happening in your polls.

While the Irish Times note the margin of error at the bottom of the article, the rest of the piece reports the figures as if they are accurate, as if the margin of error doesn't exist. Our media need to get more savvy and honest in reporting these statistics and make it clear to readers that nothing at all can be inferred from these numbers.

1 comment:

  1. To be honest it seems like the margin of error issue is the tip of the iceberg when it comes to polling. Do you watch The West Wing? They do loads of good scenes/whole episodes on the pros and cons of heavily relying on polling data. The one that sticks out right now is this issue of 'likely' voters. Sure it's good that they are ask people they poll whether they are likely to vote but when an upset happens it's going to be because of 'unlikely' voters, if that makes sense?

    Also, as the American presidency isn't won on popular votes, it's won in the electoral college, these sweeping national percentages can never tell the full story. As always the race will come down to a few key states: Ohio, Pennslyvania and Florida are usually the big three, what happens nationally does not dictate the race.

    Now go watch The West Wing which has far more articulate things to say on these matters than me.