Until now I have been using data from the polling archives at RealClearPolitics for this blog. Today I began looking at the larger archive of polls at the Huffington Post’s Pollster site. One nice feature of this site is that they offer a copy of their data in a format (“.csv”) that can be easily imported into spreadsheets or the gretl econometrics package.
I produced the table above from a regression of the size of President Obama’s lead over Governor Romney using the 146 national likely-voter polls in the Pollster database with fieldwork starting after June 30th and ending October 28th. Along with “dummy variables” to capture any differences among the 37 polling organizations represented in this sample I included a few other important predictors:
- the number of days remaining between the end of fieldwork and Election Day, November 6th;
- a dummy variable for polls whose fieldwork began after the first Presidential debate on October 3rd;
- an “interaction” term that is the product of these last two variables to allow the estimated trend line to differ before and after the debate; and,
- dummy variables for the method of polling using Pollster’s categories of in-person telephone interviews, automated telephone interviews, and Internet interviews.
As you might imagine, only a few of the polling organizations diverge so markedly from the consensus that we can statistically measure any effect for them. I narrowed down the search to the ten organizations that appear in the table above which have discernible partisan house effects.
Five organizations show “statistically significant” house effects at conventional levels (p<0.05). Three report figures with a measurable pro-Republican bias, Gallup, ARG, and Rasmussen, while two, JZAnalytics and the openly partisan DemocracyCorps, report figures favorable to the President. Gallup’s three recent likely-voter polls diverged so substantially from the polling consensus that it tops our list with an estimated four-point tilt for Romney. ARG and Rasmussen are often suspected of GOP leanings, and this analysis estimates that their polls lean about two percent more Republican than the model’s consensus. Polls conducted by JZAnalytics, either alone or with co-sponsors Newsmax and the Washington Times, report results 2.6% more Democratic than the consensus. Polls by DemocracyCorps run a bit over two percent more Democratic.
Rasmussen’s pro-Republican leaning is especially important when you consider how it dominates the polling landscape. There are 39 Rasmussen polls in the sample, or 27% of these 146 polls from the Pollster database. I’ve looked at what the polling consensus would be like in a world without Rasmussen in this post.
Another five organizations reported results that deviated sufficiently from the consensus that they met the criterion of statistical significance at the 10% level. Three of these come from organizations that conducted only one poll in this period, and all three had bias figures approaching four percent. NPR’s two polls averaged 2.8% more favorable to the President, while the YouGov/Economist polls lead a bit under two points in the President’s direction.
I found no systematic differences by type of interviewing method used. Interviewing by live telephone, automated telephone, or the Internet does not produce results that systematically favor one candidate over the other. The fact that automated interviewing shows no consistent effects suggests that including cell phones in the sampling frame may not matter at all. The firms like Rasmussen and PPP that use automated interviewing are banned from calling cell phones by Federal rule. Yet there is no evidence of a bias in automated interviewing where cell phones are excluded.
The full results appear here.
