The New Delusion: Sandy Elected Obama

Over the weekend Republican-leaning commentators at Nate Silver’s 538 Blog have been advancing the theory that President Obama’s victory can be attributed to “Superstorm” Sandy which hit the Northeast on October 29th, just a week before Election Day.  What especially irked Republicans was the reaction of New Jersey Governor Chris Christie who embraced the President in the wake of the storm despite having delivered the keynote speech at the Republican National Convention and being an active spokesman for Governor Romney.

The view that Christie “threw Romney under the bus” and thus assured the President’s re-election was the polar opposite of Republican commentary right after the storm hit.  Spokesmen like Rudy Giuliani criticized the President for campaigning in Western states like Nevada when New Jersey and New York had been ravaged. Liberal commentators like John Cassidy at The New Yorker naturally took the opposite view, suggesting that the President’s “handling of Sandy has raised his standing, and his poll ratings.”

I find neither of these arguments convincing.  First, many of the claims about a late Sandy-based surge in the polls rely on differences in the President’s level of support that fall well within the margin of error.  Cassidy, for instance, points to a three-point swing in Pew’s results as evidence, a change that could just as easily be attributed to sampling variability.  Including a term in my model for polls conducted after Sandy showed a  pro-Obama effect of one percent, but it, too, did not achieve statistical significance.

Rather than looking at the pre-election polls, I suggest we examine the exit polls taken on Election Day.  Looking at the national exit polls, we might conclude that Sandy had a large effect.  Fully 15% of voters reported that Sandy was “the most important factor” in determining how they voted, and 73% of those people reported voting for Obama.  To anyone accustomed to analyzing surveys, this 15% figure seems implausibly high. For 15% of the national electorate to attribute their vote to Sandy, when the vast majority of them lived outside the affected area, is hard to comprehend. Moreover, despite the obvious correlation between attitudes about the storm and support for the President, the direction of causality is questionable.  An equally plausible explanation for this correlation is that both answers reflect the partisanship of the person being interviewed.

We might thus wonder what effect the storm might have had on voters in swing states like Ohio, or even Florida with its history of hurricanes.  Unfortunately, if we drill down to the individual state exit polls, we find that this question was asked only among voters in New York and New Jersey, states whose outcomes were never in doubt.  Even voters in neighboring states like Pennsylvania and Connecticut were not asked about Sandy. (No exit polling was conducted in Louisiana this year so we cannot assess the claim that voters in that state might have been more attuned to the issue because of their experience with Katrina.  Mississippi voters were polled but not asked the Sandy question.)

In order for the hurricane to have affected the outcome of the election, we would need to see its effects in the swing states where the election was actually decided.  Since we do not have direct measures of peoples’ opinions about the effects of the storm in these states, and because such answers are inherently untrustworthy, we have to look at a more indirect measure of Sandy’s effects.  We can compare the opinions of voters in the swing states who reported deciding between the candidates in the last days before the election and see whether we can detect a late movement toward the President in the aftermath of the hurricane.  Here are the results from the exit polls in the nine swing states I examined before.


The chart shows quite clearly that, if anything, late deciders broke more in favor of Governor Romney than President Obama.  Of voters who made up their minds in October or earlier, the President’s average lead in these nine swing states was about five points.  Among voters who decided during “the last few days” before the election or on Election Day itself, his margin shrank to just over a percentage point.

More importantly, we can determine how large a contribution each group of voters made to the overall margin between the candidates.  Take Wisconsin as an example.  Late deciders made up 10% of the electorate in that state, and those voters favored Romney by eight points.  Taken together we can attribute an effect on the overall margin of 10% times 8%, or 0.8% in favor of Mr. Romney.  The other 90% of Wisconsin’s voters favored the President by 10%, for an overall effect on the margin of 9.0% in favor of the President.

The average effect of the decisions of late deciders in these nine states is zero compared to a four percent margin in the President’s direction among voters who decided in October or before.  Thus there is absolutely no measurable effect of Superstorm Sandy on the results in these swing states.  If anything voters who decided after the storm hit showed a slight preference for Mr. Romney in comparison to voters who decided before.

Electoral College Margin Right on the Mark

President’s Electoral Vote Share Follows Historical Trend

Political scientists use the term “swing ratio” to measure how a party’s share of the seats in a legislative body like the U.S. House of Representatives varies with changes in the share of the popular vote cast.  The swing ratio is often calculated from the best-fit line using a simple linear regression of shares of seats against shares of votes.  I have applied the same concept to the Electoral College as represented in the chart above, plotting the winner’s two-party* share of the Electoral College in each election since 1960 against the corresponding share of the popular vote. The best-fit line between the share of the Electoral College vote and the share of the popular vote for President is

% Two-Party Share of Electoral College = 3.76 x (% Two-Party Share of Popular Vote) – 132

The swing ratio is the slope coefficient, 3.76.  For each one-percent increase in the popular vote share, the winner’s share of the Electoral College grows by nearly four percent.

President Obama’s performance on Tuesday falls right in line with recent historical expectations.  He won 51.2% of the two-party vote on Tuesday earning him 61.7% of the votes in the Electoral College (including Florida).  That figure is just a bit higher than the 60.5% predicted by the equation above.

While the President won both this year’s popular and electoral votes, he did so by reduced margins. Despite those losses, the President’s share of the Electoral College was greater than that won by either Jimmy Carter or George W. Bush, whose shares of the popular vote were quite similar to the President’s.  In 2004 President Bush won an identical 51.2% of the popular vote but could garner only 53.3% of the Electoral College, some nine points behind President Obama’s showing on Tuesday.  These comparisons reinforce Nate Silver’s warnings yesterday about problems for the Republicans in the Electoral College over the years ahead.  While both Ronald Reagan and his successor George H.W. Bush won larger than expected majorities in the Electoral College, “Bush 43” underperformed in both his elections.

President Obama is the only President since 1960 to see his margin of victory fall when he ran for a second term.  All three Republicans who served two terms over this period — George W. Bush, Ronald Reagan, and Richard Nixon — received a higher percentage of both the popular vote and the electoral vote when they ran a second time.  Richard Nixon in particular skyrocketed from his narrow 1968 victory over Hubert Humphrey and George Wallace to a landslide against George McGovern four years later.  A nearly comparable improvement was recorded by Lyndon Johnson in 1964 if compared to the results for his predecessor John Kennedy four years earlier.  Ronald Reagan’s dramatic success in the Electoral College in 1980 left him little room for improvement four years later, but he still managed to increase his margin in the Electoral College from 90.9% to 97.6% in 1984.

I’ve also marked the point on the graph corresponding to the minimum share of the popular vote required to secure a victory in the Electoral College.  That figure is 48.4%.  Based on the contemporary relationship depicted above, the possibility of a President winning a minority of the popular vote but a majority in the Electoral College remains a distinct possibility.

*I follow the standard practice of measuring shares of the two-party vote to remove the effects of third parties.  This approach generally does not appear to distort the results.  Three of the four elections with substantial third-party challenges, George Wallace in 1968 and Ross Perot in both 1992 and 1996, fit the overall relationship quite closely.  In 1980 though, Ronald Reagan’s extraordinary success in winning over 90% of the Electoral College may have been helped by John Anderson’s candidacy.  Reagan won eighteen states by a margin smaller than Anderson’s share of the vote, and in ten of those states the margin of victory was less than half Anderson’s vote. Excluding the 1980 election flattens the line by a small amount as one would expect after a large positive “outlier” is excluded.  The line in this case crosses through the points representing President Nixon’s totals in 1972 and President Obama’s in 2008.  The basic relationship remains fundamentally unchanged. (Return)

Trends, Pollsters and Methods

Here is the final version of the trends and house effects model that I have been estimating over the course of the past month.  It shows a one-time drop in support for the President after the first debate in Denver of about 3.2%.  That compares to the model’s estimate that Mr. Obama would have had a 3.7% lead in the polls tomorrow had the President continued along the same path he was following prior to the election. With the debate effect included, the model predicts the President’s national level of support among likely voters tomorrow at a mere 0.5%.

Notice, though, that the most recent polls appear slightly more favorable to the President.  The model suggests that the President may have doubled that margin after Hurricane Sandy hit the Northeast.  Polls whose fieldwork began on or after October 31st appear to give the President another half point on average, but the estimate fails to reach conventional levels of statistical significance.

We now have sufficient data to get reasonably stable estimates of partisan house effects.  For this report, I have applied more stringent criteria in the identification of these effects which are described in the technical post.  Applying those criteria gives us four pollsters whose estimates clearly favored one or the other parties compared to the consensus.  Gallup, Rasmussen and ARG all lean Republican while DemocracyCorps tilts Democratic.

I invested more effort than in earlier installments trying to determine whether any of the three common methods of interviewing — in-person telephone, automated telephone, and over the Internet — advantaged one party or the other.  Remember that pollsters who rely on automated systems cannot call cell phones, so the difference between the two telephone methods should cast some light on the concerns about excluding cell-phone users from polls. At first glance, if only the polling methods are included and no house effects estimated, then automated methods appear to lean Republican by somewhat over two percent.  However one of these pollsters who rely on automated methods is Rasmussen.  If I estimate a house effect for that firm, I find no statistically significant effect for automated methods in either direction.  With the exception of one poll by Gravis Marketing, all the other automated polls were conducted by Public Policy Polling for one of two clients.  So the finding of no significant deviation for automated methods is really a tribute to PPP who appear to have developed methods that produce unbiased results even though the firm relies on calling landlines. By implication there is no evidence that calling only landlines necessarily under-represents Democratic-leaning younger and minority voters who rely on cell phones more than do older whites. It appears that people who rely on cell phones have similar political opinions to those held by their demographic peers who rely on landlines.  Appropriate demographic weighting appears sufficient to compensate for the exclusion of cell-phone users.

Internet polling is a different matter. Three organizations conduct Internet polls — Ipsos/Reuters, JZAnalytics, and YouGov/Economist. I took a similar route to my analysis of automated methods, including or excluding one or the other of these pollsters in hopes of eliminating the Internet effect.  Unlike in the case of Rasmussen, including unique house effects for these pollsters showed little difference among them.  All three firms using Internet polling show an average pro-Democratic tilt of about one percentage point.


Polls Converge as Election Nears

The graph above plots the standard deviation of each week’s national likely-voter polls using the midpoint of the fieldwork period as the basis for classification.  Values before about mid-August are based on small numbers of polls and are thus more noisy.  Starting with the week of September 9th, there are at least 10 polls included in each week’s estimate.

As expected the variation across pollsters has diminished as we get close to Election Day.  Over the past week the standard deviation of the estimates for Obama and Romney support,  and for the margin between them, has converged to about 1.1-1.2, about a third the value seen during the month of September.

Estimates of support for President Obama have shown more variability than those for his opponent.  The average standard deviation over this period was 1.95 for the President compared to 1.56 for Governor Romney.  Naturally the lead shows much more variability than the estimates for either candidate with an average value of 2.73.

Critics of Rasmussen’s polling suggest his results converge with the results of the other pollsters as Election Day nears.  If we plot his polls against the average for all other pollsters, we find less evidence for convergence than the critics allege.

While it is obvious that Rasmussen’s polls ran more Republican than the consensus, as I have shown before, there is still a substantial gap of about two points between his results and the other pollsters.  His most recent figures put the race as tied nationally while the consensus has President Obama ahead by somewhat over two percent.

How Undecideds Split – Evidence from 1980-2004

Undecideds split evenly using data from five elections

Post-Election Reported Vote for Pre-Election Undecideds and Third-Party Voters

I extended the analysis of undecided voters described in the preceding post using the American National Election Studies for all Presidential elections starting in 1980 where an incumbent was running for re-election.  There is no evidence that undecideds broke disproportionately for the challenger in any of these elections.  Reagan did best among the challengers, picking up 43% of the undecided vote compared to Carter’s 34% in 1980.  However this advantage probably had as much to do with Reagan himself as with his challenger status that year.  Four years later when Reagan was the incumbent he outdrew Mondale among undecideds 47-41.  In the three other elections undecideds were just as likely to give their votes to the incumbent as to the challenger.

How Undecideds Split – Evidence from 2004

No evidence they disproportionately prefer the challenger

Some people posting over on Nate Silver’s 538 blog keep making the claim that undecided voters usually split 2:1 in favor of the challenger when they get to the polls.  One can imagine a good reason for this pattern, namely that the incumbent is much better known than the challenger.  By that argument, people who are still undecided in the last days of the campaign probably have a stronger anti-incumbent attitude than a pro-challenger one.  When forced to make a decision, the more powerful negative attitude prevails, and the voter opts for the challenger. (There is also some evidence from psychology that negative attitudes may influence behavior more than positive ones.  The emphasis on negative campaigning seems to take this view.)

One could, of course, make the counter-argument, that undecideds may be upset with the incumbent but uncomfortable with the challenger.  People with that balance of opinion might eventually choose the “devil they know” over the one they know not.

Unfortunately the proponents of the view that the challenger is the ultimate beneficiary of undecideds rarely if ever muster any evidence for this view.  To test this hypothesis requires identifying people who were undecided before an election then later asking them how they actually voted. For this you require what is called a “panel” study where the same people are interviewed both before and after an election.

Now, as it happens, the academic political science community has been conducting just this sort of study for decades.  Most of the American National Election Studies include a pre-election and post-election wave of interviewing.  I used the 2004 dataset since that is the most recent election where we had an incumbent President running for re-election. The trajectory of the 2012 election also seems  quite similar to eight years ago.  Here are the results:

The table presents the same data in two different ways.  The top table includes all 1,211 respondents who completed an interview in the pre-election wave conducted mostly in September and October. I placed all respondents who did not choose Bush, Kerry, or Nader in the pre-election wave into a residual “undecided” category.  (The results are unchanged if only pure undecideds, people who said “don’t know” when asked their preference, are considered.)

The top table shows that it was harder to secure a second interview with people who preferred Nader or who were undecided in the first wave.  Nearly half those respondents did not complete a post-election interview, compared to about 30% for major party voters.  Since all but one of the respondents in the post-election wave reported having voted for President, we might expect people who failed to complete a second interview were also less likely to have voted.  So one plausible suspicion about undecided voters is supported by these data, that undecideds are less likely to vote than decideds.

However, there is no support for the notion that undecideds split disproportionately for the challenger.  By my definition of undecideds, Bush and Kerry each received 38% of their votes.  If we limit the definition of undecided to only people who explicitly said “don’t know” to the pre-election preference question, fifteen such undecided people said they had voted for Kerry versus fourteen who said they had voted for Bush.

I note with amusement that of the fifteen people who said they would vote for Nader in the pre-election wave not one of them report casting a ballot for him at the polls.  Kerry got four of their votes; Bush got two.  Half of them could not be reinterviewed.

(Technical note:  For the budget conscious like me, it is a lovely thing that the ANES studies are freely distributed as SPSS “portable” files which can be read directly by the open-source SPSS clone, PSPP.)


The Model for Ohio

No trends found for Ohio, just a two-point debate effect.

I usually do not publish raw regression results on the main pages of this blog, relegating them instead to the Technical Topics category.  However rather than spend time building an uninformative graph of the Ohio campaign, I’ll just report the results here and explain their meaning.

I have applied the simple trends and house effects model that I developed for national polling to the results for Ohio.  Once again I am using the Pollster archive and including all polls of “likely” voters conducted after June 30th.  I also included a variable for the first Presidential debate and dummies for the pollsters to measure house effects.  I get these results:

Model 15: OLS, using observations 1-66
Dependent variable: Dem_Lead

                 coefficient   std. error   t-ratio   p-value 
  const           5.31164      1.00663       5.277    2.21e-06 ***
  DaysBefore     −0.0235519    0.0147563    −1.596    0.1161  
  Debate1        −2.23855      0.845398     −2.648    0.0105   **
  Rasmussen      −2.65019      0.821597     −3.226    0.0021   ***
  Gravis Mktg    −2.28954      0.925727     −2.473    0.0164   **
  ARG            −2.39894      1.26821      −1.892    0.0637   *
  NBC/WSJ/Marist  3.37662      1.27493       2.648    0.0105   **
  Wenzel/CtznsU  −5.55798      2.20545      −2.520    0.0146   **
  Qunn/NYT/CBS    3.74099      1.31111       2.853    0.0061   ***
  WaPo            3.72465      2.17768       1.710    0.0927   *

Mean dependent var   2.939394   S.D. dependent var   2.833203
Sum squared resid    250.9900   S.E. of regression   2.117065
R-squared            0.518953   Adjusted R-squared   0.441642
F(9, 56)             6.712523   P-value(F)           1.86e-06

First, we see at best only a very weak trend in the President’s favor over the course of the summer, one that fails to prove significant even at the 0.10 level.  The best interpretation is that there have been no discernible trends in Ohio since June, just a one-time drop of two points in the President’s lead from a bit over five percent before the first Presidential debate to three since then.

Seven pollsters showed statistically significant deviations from the consensus for Ohio though only Rasmussen and ARG also appeared on the list for national polls. The measured effects for those two organizations in Ohio are both slightly larger than the effects measured in national polling.

Polls by the three major media organizations — NBC/WSJ/Marist, Quinnipiac/NY Times/CBS, and the Washington Post — all had results over three points more Democratic than the consensus.  While these organizations are often criticized as being “in the tank” for Obama (leaving aside the Wall Street Journal, of course), I don’t find any such partisan bias in their national polling.  The poll with the largest outlier, Wenzel/Citizens United, had a obvious partisan sponsor and reported results consistent with its Republican ideology.


Partisan House Effects in National Polls

Until now I have been using data from the polling archives at RealClearPolitics for this blog.  Today I began looking at the  larger archive of polls at the Huffington Post’s Pollster site.  One nice feature of this site is that they offer a copy of their data in a format (“.csv”) that can be easily imported into spreadsheets or the gretl econometrics package.

I produced the table above from a regression of the size of President Obama’s lead over Governor Romney using the 146 national likely-voter polls in the Pollster database with fieldwork starting after June 30th and ending October 28th.  Along with “dummy variables” to capture any differences among the 37 polling organizations represented in this sample I included a few other important predictors:

  • the number of days remaining between the end of fieldwork and Election Day, November 6th;
  • a dummy variable for polls whose fieldwork began after the first Presidential debate on October 3rd;
  • an “interaction” term that is the product of these last two variables to allow the estimated trend line to differ before and after the debate; and,
  • dummy variables for the method of polling using Pollster’s categories of in-person telephone interviews, automated telephone interviews, and Internet interviews.

As you might imagine, only a few of the polling organizations diverge so markedly from the consensus that we can statistically measure any effect for them.  I narrowed down the search to the ten organizations that appear in the table above which have discernible partisan house effects.

Five organizations show “statistically significant” house effects at conventional levels (p<0.05).  Three report figures with a measurable pro-Republican bias, Gallup, ARG, and Rasmussen, while two, JZAnalytics and the openly partisan DemocracyCorps, report figures favorable to the President.  Gallup’s three recent likely-voter polls diverged so substantially from the polling consensus that it tops our list with an estimated four-point tilt for Romney.  ARG and Rasmussen are often suspected of GOP leanings, and this analysis estimates that their polls lean about two percent more Republican than the model’s consensus.  Polls conducted by JZAnalytics, either alone or with co-sponsors Newsmax and the Washington Times, report results 2.6% more Democratic than the consensus.  Polls by DemocracyCorps run a bit over two percent more Democratic.

Rasmussen’s pro-Republican leaning is especially important when you consider how it dominates the polling landscape.  There are 39 Rasmussen polls in the sample, or 27% of these 146 polls from the Pollster database. I’ve looked at what the polling consensus would be like in a world without Rasmussen in this post.

Another five organizations reported results that deviated sufficiently from the consensus that they met the criterion of statistical significance at the 10% level.  Three of these come from organizations that conducted only one poll in this period, and all three had bias figures approaching four percent.  NPR’s two polls averaged 2.8% more favorable to the President, while the YouGov/Economist polls lead a bit under two points in the President’s direction.

I found no systematic differences by type of interviewing method used.  Interviewing by live telephone, automated telephone, or the Internet does not produce results that systematically favor one candidate over the other.  The fact that automated interviewing shows no consistent effects suggests that including cell phones in the sampling frame may not matter at all.  The firms like Rasmussen and PPP that use automated interviewing are banned from calling cell phones by Federal rule.  Yet there is no evidence of a bias in automated interviewing where cell phones are excluded.

The full results appear here.

The Race Without Rasmussen

Rasmussen Reports accounts for 38 of the 141 recent likely-voter polls in my dataset from Pollster or fully 27% of all the observations.  Earlier I have shown that Rasmussen has a pro-Romney “house effect” of over two percent.  Given the combination of Rasmussen’s dominance and its pro-Republican bias, we might wonder what the polling consensus would be if there were no Rasmussen Reports.  Here is the graph from my most recent post after excluding Rasmussen’s polling from the sample.

Without Rasmussen, we do not see evidence that the campaign has stagnated.   We do find that the other pollsters recorded a somewhat larger drop for President Obama after the first debate putting him a point behind Governor Romney.  However, a model that excludes Rasmussen shows no evidence of stagnation but keeps the President on his slow upward trend with a slim margin of 0.6% in the polls on Election Day.  Here is the graph from the earlier post with Rasmussen included for comparison.

Full regression results are here.

Is it 2004 all over again?

Using the 141 national likely-voter polls from the Pollster database since July 1st we can examine how the dynamics of the race have changed since the first Presidential debate.  So far the pattern looks very reminiscent of the period after the first Presidential debate in 2004 between George W. Bush and John Kerry.  In that election, support for Bush dropped about five points after the first debate, leaving the President with a two-point lead that he maintained up until Election Day..  The 2012 Presidential campaign seems to be following the same trajectory except that President Obama no longer has a lead to maintain.

The polls indicate that support for the President fell by about four points immediately after the first debate leaving him essentially tied with Mitt Romney.  If the President had reverted to trend at that point and followed the dotted line, he would have gained back a lead in the polls of about 1.3% by Election Day.  So far, however, the polling suggests no such return to form.  On the basis of the polling since the first debate, the candidates look to remain roughly tied from now until the election.

(These data include the adjustments for “house effects” described in my previous post.  The results for the full regression model are here.  The later debates show no measurable effects.)