Alabama’s citizens will head to the polls to vote in the most competitive Senate election that state has seen for decades. Democrat Doug Jones is trying to wrest the state into his party’s column while Republicans rally behind Roy Moore. Most anyone reading this blog knows about the issues in this race, so I’m going to focus solely on the polls as archived at RealClearPolitics. Since RCP does not publish data on polling methods, I examined the individual reports for each poll and, when the method used was unclear, contacted the polling agencies directly.
Polling results in this race have shown little convergence as we approach election day. The Republican dominance in Alabama’s elections has meant that few national polling agencies have paid much attention to the state over the recent elections. As a result, few national polling organizations have much experience surveying Alabama’s voters. That has changed a bit as the election achieved national prominence, but still the vast majority of Alabama polls come from organizations will limited track records. Over at FiveThirtyEight, Harry Enten observes that “Alabama polls have been volatile, this is an off-cycle special election with difficult-to-predict turnout, and there haven’t been many top-quality pollsters surveying the Alabama race.”
“Top-quality” pollsters rely on live interviewers making calls to both landline phones and cell phones. FiveThirtyEight adds the additional criterion that the polling agency be involved with national organizations like the American Association for Public Opinion Research. I will limit my analysis to just whether calls were made to a sample of cell phone owners. As it turns out, this factor alone has a profound effect on a poll’s estimated margin between the candidates.
Here is a list of the available polls based on their method of interview.
Only one poll among those that included interviews with respondents via cell phone shows a lead for Moore; in contrast, only one of the landline-only polls puts Jones ahead. The “swing” is quite substantial, about an eight-point differential based on the method used.
This difference arises from the much higher usage of cell phones by younger respondents who prefer Jones in most polls. For instance, in today’s poll from Fox News likely voters under 45 year of age preferred Jones 59% to 28%, while voters above that age preferred Jones by only a one point margin, 45% to 44%.
I also modeled the difference in support between Jones and Moore using my standard predictors, time left before the election, and dummy variables for polling methods. I also added a dummy which is coded one beginning on November 9th when the story about Moore’s alleged molestation of young girls was released in the Washington Post. The variable measuring proximity to the election proved statistically insignificant, leaving just three dummies, whether the pollster made calls to cell phones, whether live interviewers were used, and whether the story had broken in the Post.
In polls taken before the publication of the molestation story, Jones trailed Moore by an average of eleven points. Since then Jones has seen an average gain of six points, not enough on its own to return the race to even. However polls that interviewed respondents via cell phones show a slightly larger difference of nearly seven points in Jones’s favor. Taken together, these results suggest that Jones has averaged a 1-2 percent lead in polls taken since the Washington Post story that included calls to cell-phone users. (Update: Jones’s margin of victory over Moore was 1.5 percent statewide, right in line with this prediction.)
I also included a term for whether live interviewers were used. Since all polls that include cell phone owners must use live interviewers by law, this remaining group represents organizations that polled only landline owners with human interviewers. I find a small, though statistically insignificant (p<0.17) positive effect on Jones’s support from people surveyed by live interviewers. It is hard to interpret what this effect might signify. It could represent an unwillingness among Moore’s supporters to admit their intentions to a human interviewer but have no such hesitation when the interview is conducted by a robot. If so, we might attribute some of the difference between cell phone and landline results to use of human interviewers in polls that include cell users.
