Modeling Senate Elections Redux

I have reworked my model for Senate elections using data for elections in 2016 and 2018. That model relied on three factors to predict the vote for the Democratic candidate:

  • the “net favorability” (favorable – unfavorable) of the incumbent Senator;
  • a measure of the state’s favorability toward Donald Trump; in 2016, I used his proportion of the two-party vote; in 2018, I used his job approval rating; the two measures proved to have identical effects;
  • the ratio of spending by the campaign for the Democratic candidate versus spending by the campaign for the Republican candidate.

Using Net Approval for Donald Trump

In the original formulation, the favorability of the incumbent Senator was measured on a “net” basis, favorable – unfavorable, while the measure for Trump support was not.  Since most everyone polled has an opinion about the President’s job performance, the approval rating alone is typically sufficient. Favorable and unfavorable job approval ratings for Donald Trump generally sum to about 96 percent.

Asking about other politicians results in much higher “don’t know” responses. On average the sum of favorable and unfavorable responses for the average Senator in this sample of races is just 79 percent with 21 percent undecided. Net approval only measures the difference between approvers and disapprovers and leaves out the undecideds.

In this reformulation of the model I put the two measures on an equal footing by imputing a net job approval figure for Trump. I have done so assuming the sum of positive and negative figures for him equals 96 percent. Then simple algebra results in the formula

(Approve – Disapprove) = 2 X Approve – 96

Using net approval for both measures improves the model’s clarity since both scores are measured in the same units, and the constant term reflects the situation where a state has a value of zero (50 approve, 50 disapprove) on both support for Trump and favorability toward the incumbent Senator (and the campaigns are spending identical amounts of money).

Using Base-Two Logarithms for Spending Figures

One other change I’ve made to the model is measuring campaign spending using logs to the base two rather than ten. Using base two makes the associated coefficient easier to interpret. An increase of one unit in this measure represents the difference between a race where both campaigns spend the same amount of money and a race where one candidate spends twice as much money as her opponent (since log2(2/1) = 1).

In this formulation we are left with two predictors. One is the difference between the Democratic candidate’s net approval and the same figure for Donald Trump. A Senate candidate who has a six-point advantage over Trump in net approval wins on average one more point at the polls (0.17 X 6 = 1.02).

The campaign spending coefficient indicates that candidates whose campaigns spend twice as much as their opponents can expect to add 1.4 percent to their margins on election day.

In a race where both the net favorability of the incumbent Senator and that of Donald Trump equals zero, and the candidates spend identical amounts of money making the logarithm of the spending ratio also zero, then the Democratic candidate loses the average state with 49.4 percent of the two-party vote. That value reflects the slight Republican tilt of the average state in terms of its vote for Senator.

Which Factor is More Important?

One way to compare the coefficients in this model is to convert them to “standardized” units. Standardized coefficients measure the effect of each predictor after dividing the dependent and each independent variable by its standard deviation. (Usually the means are subtracted as well forcing the intercept in the standardized model to zero.)  These standardized coefficients measure the effect in standard deviation units of a one standard deviation increase in each predictor and, in that sense, provide a standard for comparing their importance.

In this model the standardized coefficients are not all that different from one another. The standardized coefficient for the net approval variable is 0.54; for campaign spending it is 0.42.  It’s not surprising that the more partisan approval variable is slightly more important, but the difference between the two is relatively small.

Senate Elections in a Time of Economic Contraction

The novel corona virus pretty much guarantees that the American economy will decline this year. While the President and most pundits have focused on how a falling economy might affect his re-election, an economy in recession also improves the Democrats’ chances of taking control of the Senate in 2021. A ten percent decline in real GDP translates into the Democrats winning about 53 percent of the national vote for Senate candidates.

Pretty much every forecaster predicts that the economy will contract substantially over the next three months as large portions of the American economy remain idle in the face of the COVID-19 pandemic.  Most of these forecasts are clearly guesswork since we still have only a glimmer about the toll the virus will take on the U.S. economy.  Fortune magazine describes forecasts for the second quarter as ranging from “horrible” to “catastrophic,” with the estimated change in real Gross Domestic Product (GDP) in the range of -8% to -15%.  Morgan Stanley‘s estimate is especially grim, predicting a decline of -38%. Like many other analysts Morgan Stanley expects the economy to rebound some in the third quarter, but the rebound will not be sufficient to overcome the enormous declines of the first half of 2020.  They expect the year to end with real GDP down by 5.5%.

These declines eclipse anything we have seen since World War II.  The economy contracted by about 3.3% during the recession year of 2009 and fell between 2.2% and 2.9% in the earlier recessions of 1958, 1975, and 1982.

Back in 2016 I constructed a “simple model of Senate elections” that looked at how political and economic factors influence the nationwide Senatorial vote since the War.  Three factors proved to have statistically significant relationships with the share of the vote won by the President’s co-partisans in those years. One of these factors favors the Republicans, the fact that Donald Trump will head the ticket in November.* The President’s party has won, on average, 51 percent of the two-party vote for the Senate in years when the President heads the ticket, compared to just 47 percent in elections when the President is not running.  (This includes both off-year elections, and open-seat Presidential elections like 2016.)

Two factors favor the Democrats in 2020.  One is a weak “regression-toward-the-mean” effect based on the votes won in the Senate elections six years earlier. Senators who win election with an above-average share of the vote in one election are likely to see their vote decline slightly when they run for re-election six years hence.  Republicans did unusually well in the 2014 mid-term elections so we might expect their vote shares fall back slightly in 2020.

The economy also plays a role. My model uses the year-on-year change in real per-capita disposable income as of September as a measure of the state of the economy.  I will use this “simple model” to estimate the effects of the likely recession on the upcoming Senate vote in 2020.

Forecasters rarely estimate the change in real per-capita disposable income and focus instead on changes in real GDP or employment. Unsurprisingly, though, changes in real GDP do filter through to personal income as shown in this chart.

I have marked the seven recessionary years, ones where real GDP fell year-over-year.  One thing to notice is that even when real GDP remains flat, personal income is still predicted to grow by one percent.  Moreover, only 37 percent of changes in real GDP are transmitted to personal income.

I have used this “simple” model to examine how different predictions for the state of economy in November might translate into Senate electoral outcomes.**  The baseline appears on the line below with zero growth in GDP. The Republicans are predicted to win about 48% of the nationwide vote for Senate candidates in such an election. This estimate combines the positive effect of having the President on the ticket with the negative effect of the Republicans’ substantial victory in the 2014 midterm elections where their candidates won 53.5 percent of the two-party vote. In the context of my model those factors predict that the Republicans will win 48.3 percent of the nationwide two-party vote for Senate.

Because changes in GDP are attenuated when translated into changes in per-capita disposable income, even a drop of ten percent in GDP results in a vote for the Republicans that is just one percentage point less than if GDP remained flat. Even if the worst predictions of the forecasters hold true, and GDP falls by twenty percent, the predicted Republican vote falls to only 46 percent.


*Presidential popularity does not appear to play a role in on-year elections, though it does matter for elections held in off-years.

**These results are based on a reestimation of the published model including data for the 2016 and 2018 elections. While the coefficients change slightly, none of the substantive conclusions are altered.

Money in Senate Elections

Senate campaigns that outspent their opponents by two-to-one in 2016 and 2018 typically gained a bit over one percent at the polls. Spending by outside groups, and the “quality” of challengers, had no measurable effects.

My earlier model of recent contests for the U.S. Senate relied entirely on two measures of popularity, the favorability score for the incumbent Senators in their states, and support for President Trump in those same states.  While those two measures alone explain 81 percent of the variance in the vote for Senatorial candidates, the model obviously lacks a few important items, most notably data on campaign spending and on challenger “quality.”  In this post I add measures of both these factors.

For campaign spending I have used the figures reported to the Federal Election Commission and compiled by OpenSecrets.  I chose to use spending rather than funds raised because in most cases campaigns spent nearly all of what they raise, and sometimes more. For instance, here is the record for campaign spending in the 2018 Missouri Senate race where incumbent Claire McCaskell lost to Republican Josh Hawley, then Attorney General.

The other major source of campaign financing is, of course, spending by outside groups.  Here, OpenSecrets separately reports funding in support of and opposed to each candidate.  My measure of outside spending adds together monies spent supporting a candidate and those spent criticizing her opponent. I use the base-10 logarithm of spending which has a better fit to the data and incorporates the basic economic intuition of decreasing returns to scale.

Spending by the Campaigns

I first added the campaign spending figures for Republicans and Democrats separately with results as shown in column (2). Democratic spending appears to have had a larger effect than Republican spending, but a statistical hypothesis test showed the two coefficients were not significantly different in magnitude. So in (3) I use the difference between the two spending measures, which is equivalent to the base-10 logarithm of the ratio of Democratic to Republican spending.*

An increase of one unit in these logarithms is equivalent to multiplication by ten. So the coefficient of 4.39 tells us that a ten-fold increase in the Democrats’ spending advantage would improve their share of the two-party vote by somewhat over four percent.  While a ten-fold advantage might seem implausibly high, some races have seen such lopsided spending totals. In Alabama’s 2016 Senate election Republican incumbent Richard Shelby spent over twelve million dollars on his race for re-election; his Democratic opponent spent less than $30,000. In that same year, Hawaii Democrat Brian Schatz spent nearly eight million dollars while his opponent spent $54,000.  These sorts of drastic imbalances typically appear in non-competitive races where the incumbents are seen as shoo-ins to retain their seats.

To see more intuitively how spending affects results I have plotted the predicted change in the Democratic vote for various ratios of Democratic to Republican spending.  The state codes represent the seven most competitive races as identified by my model. (I will examine the implications for 2020 in a separate post.)

In states where the Democrats outspent the Republicans by a ratio of two-to-one, the Democrats were rewarded on average with an increase of about 1.3 percent in their vote shares.

Spending by Outside Groups

In sharp contrast to the results for spending by the campaigns themselves, I find no systematic influence for spending by outside groups. Neither including separate terms for pro-Democratic and pro-Republican outside spending as in model (4) above, nor including the difference between those figures in model (5), displays significant effects.

While I’m not ready to make strong claims for this rather surprising finding without an expansive review of the literature on spending in Senate campaigns,1 I don’t find the result all that surprising. Since outside groups may not, by law, “coordinate” with the campaigns they support, these groups must focus their attention on television advertising, direct mail, and other messaging strategies.  Perhaps these strategies simply are not as effective as they once were, as demonstrated by the Presidential primary candidacies of Michael Bloomberg and Tom Steyer. They both spent hundreds of millions of dollars on television advertising but garnered few votes on election day.

Effects of Challenger “Quality”

Another common factor used to explain legislative elections is the “quality” of the challengers that choose to take on an incumbent. While some people launch vanity Senatorial campaign to make themselves better known to the public at-large, most Senatorial bids are undertaken by people who already hold elective office at either the state or the Federal level.  I have coded the backgrounds for the challengers facing each incumbent in my dataset of 2016 and 2018 elections.  They fell into four categories — current or former Members of Congress, current or former members of the state legislature, governors and others who have held state-wide office, and a miscellaneous category that includes local-level politicians like mayors and non-politicians like activists.  I find no statistical effects for any of these categories either separately or in combination.

We are thus left with a model of Senate elections that includes three factors — the incumbent’s net favorability, the state’s level of support for Donald Trump, and the ratio of spending by incumbent and opposition campaigns.



*Remember from high-school math that log(A/B) = log(A) – log(B).

1I have since discovered this article examining television advertising in Senatorial elections using data for the 2010 and 2012 elections. The authors use a novel technique that compares adjacent counties that reside in different media markets. Overall, they find significant effects on vote share for negative (but not positive) advertising by the candidates and no effects for advertising by PACs. This paper by political scientists John Sides, Lynn Vavreck, and Christopher Warshaw find significant effects for television advertising in Senate races, but again they find like I do that the effects are small. A change from -3 standard deviations to +3 standard deviations in advertising produced just a 1% change in Senate races. They do not analyze the effects of spending by the campaigns versus that by outside groups.

Senate Update, March, 2020

The Democrats have a decent chance to take control of the Senate.

I have updated my Senate predictions using the fourth-quarter, 2019, favorability data for Senators and February, 2020, job approval ratings for Donald Trump. Both come from Morning Consult.  I have also cleaned up a few errors in the earlier data used to estimate the model’s coefficients. Here are the updated results:

Maine’s Susan Collins now joins Alabama’s Doug Jones as the most-vulnerable Senators up for re-election.  Both Senators face adverse political environments in the states they represent.  Mainers don’t care for Collins very much, and they’re slightly negative when it comes to Donald Trump. Unlike Collins, Jones is liked by a plurality of Alabamians, but Trump is liked so much more that it overwhelms Jones’s personal popularity.

Steve Bullock’s musings about running against incumbent Montana Senator Steve Daines find little support in the data here.  Both Daines and Donald Trump have positive ratings in Big Sky Country, with the Senator predicted to win re-election with 57 percent of the vote. Jaime Harrison also faces a pretty uphill quest in his bid to oust Lindsey Graham in South Carolina.

If these estimates were to hold, the Democrats stand a good chance of flipping the Senate in November. If Jones, Collins, Gardner, and Ernst all lose, the Democrats would net three seats. That would create a 50-50 tie and require the Vice President to be decisive.  Also defeating one of McConnell, McSally, or Tillis would give the Democrats a 51-seat majority.


The 2020 Senate Elections

In a prior series of posts, I constructed a “simple model of Senate elections” using national data across elections.  This helped identify some key factors that influence the overall vote for Senators but provided no insight on the results in specific states.  In this post I develop another “simple” model that is designed to predict the voting outcome based on two factors, a state’s partisanship as measured by support for President Trump, and the net favorability of the incumbent Senator.  I estimated the model using data from the 2016 and 2018 elections. The results appear here and are best summarized in this chart:

The lines portray how the vote for an incumbent Senate Democrat improves as her net favorability grows. The top line represents the result for a Senator from a strongly pro-Democratic state, one where only 40% of the state’s voters approve of the President.  Even a Democratic incumbent with a net favorability of zero is predicted to win nearly 55% of the vote in this state and hold the seat.  In contrast, a Democratic incumbent in a pro-Trump state like Doug Jones in Alabama fails to win 50% of the vote even if he is unusually popular despite the party mismatch.  Overall the Republicans hold a slight advantage. The model predicts that in a state where support for Trump is 50-50, the purple line, only a Democratic incumbent with at least a +8 favorability has a chance of holding the seat.

We can apply the results of this model to the 2020 Senate elections.  We only have available the current measures for Trump support and candidate favorability, so we obviously cannot predict how things will stand a year from now.  For the estimates below, I have used the most recent Trump approval rating and Senate incumbent favorability ratings as reported by Morning Consult.  The President’s score is from the month ending September 1st; the Senators’ ratings are averages over the third quarter, July-September, 2019.

The highlighted rows at the top of the table correspond to incumbent Senators whose predicted vote is below fifty percent.The top and bottom spots on the list are held by Democrats. The most vulnerable incumbent is Doug Jones’s whose slight positive favorability rating of +5 is nowhere near large enough to overcome Alabama’s warm feelings for Donald Trump.

Jones is followed by the four most commonly discussed vulnerable Republicans — Susan Collins of Maine, Cory Gardner of Colorado, Joni Ernst of Iowa, and Thom Tillis of North Carolina. Martha McSally would hold her Arizona seat by the slimmest of margins. Majority Leader Mitch McConnell is lucky to represent solidly pro-Trump Kentucky or else his dismal favorability score might lead to his defeat.

It’s anyone’s guess what Donald Trump’s approval rating might be come the election next November, though his score has remained remarkably persistent in the face of events.  Using the averages at FiveThirtyEight, we see his low point came in the summer of 2017 when he fell to 37%. Over that winter and into the spring of 2018 his approval rating improved to about 42% where it has largely remained. There was a dip in his popularity during the government shutdown, and another now as the impeachment inquiry expands.  Given the observed variation in his popularity since the Inauguration, Trump’s approval rating might move up or down by three or four points over the course of the next year.  A four-point movement would represent a ten-percent change from his current rating of 41%.  The chart below shows how each Senator’s predicted vote would change given a ten percent increase or decrease in Trump’s approval rating in each state.

The four Senators at the top of the list in the darker grey area are predicted to lose their seats even if Trump’s approval rating were to improve by ten percent.  The next three Senators survive their re-election bids if Trump’s approval runs about where it is today or improves by November, 2020.  However a ten-percent decline in Trump’s approval threatens the seats of Thom Tillis, Martha McSally, and even Mitch McConnell.

Right now the Republicans control the Senate by a 53-47 margin, plus the tie-breaking vote of the Vice President. Assuming a Democratic victory in the Presidential election next fall, the Democrats need to flip at least three seats, while losing Alabama back to the Republicans. Maine, Colorado, and Iowa look promising for the Democrats and North Carolina and Arizona are both tightly contested.

Four Republican seats have vacancies. In Georgia a special election will be conducted in 2020 alongside the regular election to fill the seat that Johnny Isakson will leave at the end of this year.  Three other Republican-held seats will also be vacant in 2020.  My model predicts the Republicans will hold all these seats with Georgia the most competitive.  (To construct these estimates I impute a favorability score for a “normal” Democrat by regressing net favorability on Trump support to account for the partisanship component of favorability.)

In strong Republican states like Wyoming and Tennessee, we see support for Trump running in the mid-fifties.  In states like these, a Democratic challenger would do well to card a favorability score better than -15.  The states where the Democrat might have some chance are Georgia and Kansas, where support for Trump splits evenly, but still the Democrats are predicted to lose those elections by three or four points.


Technical Appendix: Party and Incumbent Favorability in Senate Elections

These are the regressions which underpin the results presented in this posting.  The dependent variable is the Democratic share of the two-party vote for Senate in each state.  The favorability figure comes from Morning Consult; the Trump approval measure for 2018 comes from Gallup.

  • big chunk of favorability’s effect is partisanship; controlling for Trump support brings the favorability coefficient down
  • no measurable difference in effect of Trump support measured either using his 2016 vote or his 2018 approval
  • two elections had large residuals, Alaska in 2016 where there was a strong third-party contender, and Utah in 2016 where Mike Lee trounced a transgender female Democrat in the home of the Mormons.

The Election in Pictures

Some data updated through September 2.

Sunday, July 29th, marked the point when there are just 100 days left until the November midterm.  In this post I will try to pull together my various writings and predictions for both the 2018 House and Senate elections.  I begin with the most important factor that influences both types of races, the President’s job-approval figures.

Presidential Approval

The President’s job-approval rating is an important predictor of midterm election results in both my model for Senate elections and the one for House elections.  The Democratic advantage on the “generic-ballot” question about voting in House elections has waxed and waned as Donald Trump’s approval rating first fell after he was inaugurated then rose again over the past few months.

Donald Trump enters the 2018 election with about the same level of public support Barack Obama had in 2010. Though Obama was much more popular when he was inaugurated, that goodwill faded over the following eighteen months. In the 2010 midterm that followed, the Democrats were “shellacked,” losing sixty-three seats in the House of Representatives. Support for Trump also ebbed away during 2017, but he has rebounded slightly from his nadir last December.

Though Trump’s overall approval rating heading into the first midterm is largely identical to Obama’s, Trump is more intensely disliked.  Pew reported that the proportion of people saying they “strongly” disapproved of Obama’s performance in office grew from 18 percent in April, 2009, to 32 percent by September, 2010.  For Trump, CNN found that he took office with over forty percent of Americans already strongly disapproving, a figure that has remained relatively unchanged.  In its June, 2018, poll CNN reports a “strongly disapprove” figure of 45 percent.

The House of Representatives

As the first chart shows, the president’s job-approval rating bears directly on the “generic-ballot” question.  Based on polls through September 2nd, the Democrats’ lead on the generic ballot has grown slowly since Inauguration Day and inversely with Presidential job approval.  The grey area in the chart below represents the likely range of outcomes.  Since April, Donald Trump’s net approval rating has ranged from about -8 to -14.  Those values define the left and right sides of the shaded region.  When I include estimates of any methodological or unique “house effects,” I find that polls conducted over the Internet show a pro-Republican tilt of about 2.6 percent.  Gallup’s polling shows an ever greater Republican edge of nearly five percentage points.  I use the values for live polling, which are most favorable to Democrats, and those from the much-less favorable Gallup, to define the height of the grey area.

Notice that, according to these results, even if Trump were to achieve a net approval rating of zero, Democrats are still predicted in live polling to lead on the generic ballot by about six points on Election Day. That reflects the slow growth in support for Democratic House candidates over Trump’s presidency from about four points on Inauguration Day to a predicted six points this November. In past elections the margin of victory in generic-ballot polls has proven to be a pretty accurate predictor of the actual division of the vote.

An earlier version of this model showed a small, marginally significant positive boost for Democrats in polls of likely voters.  That difference has disappeared as the number of polls has increased.  Since Democrats are generally disfavored in likely-voter polls, especially ones conducted in midterm years, a finding of no difference between registered and likely voters is actually positive news for Democrats.

One other major problem for House Republicans has been the historically large number of their Members who are leaving, or have left, the House.  Forty Republicans will not be returning to the House next January creating an excess number of more vulnerable open seats on that side of the aisle.  Only 18 Democrats are leaving the chamber.  With 22 more retirements than the opposition, the largest midterm gap since the New Deal, the Republicans face a loss of 39 seats based on the historical relationship between the two measures.

The Senate

There is no national generic-ballot question for Senate elections because only two-thirds of the states have a Senate race in any given year.  Looking back historically over Senate elections, the fate of the President’s party depends directly on his job-approval rating and, unlike for House elections, the state of the economy as measured by the growth in real disposable personal income per capita.  Any plausible combination of approval for Donald Trump and income growth predicts that the Republicans will fail to win a majority of the popular vote for Senate in November.  One reason is that the popular vote for Senate candidates of the President’s party runs four points lower when the President is not on the ballot.

Much has been made of the 4.1 percent increase in Gross Domestic Product reported for the second quarter of the year.  That figure represents the growth in nominal GDP; after adjusting for inflation the figure is 2.8 percent. Unfortunately for the Republicans little of that growth appears to be “trickling down” to ordinary Americans.  Here are the recent trajectories for both real GDP and real per-capita disposable personal income.

Personal income has hovered around a two-percent growth rate for the last three quarters, while real GDP grew more quickly.  If voters respond to changes in the amount of money in their pockets, then the economy will not be sufficient to power the Republicans to victory in the fall.  From the chart above, a two-percent growth rate in per-capita income and even a 45 percent approval rating for Donald Trump still leaves Republican candidates short of a majority in the national popular vote for Senate.

From Votes to Seats: The House

Americans became much more cognizant of the word “gerrymander” after the redistricting that followed the publication of the 2010 Census.  Drawing lines for partisan advantage has become easier as voters have segregated themselves geographically by party.  Together the two forces have combined to create a Republican “bulwark” in the House. Democrats need to win the popular vote by more than fifty percent to take half the seats in the body.  How much more is subject to debate, but most estimates put the needed margin of victory in the range of six-to-eight percentage points, or an election where the Democrats win somewhere between 53 and 54 percent of the popular vote.

For instance, The Economist currently projects the Democrats to win 54.3 percent of the popular vote, or a margin of 8.6 percent, but take just 51.3 percent of the seats in the House.  Both Dave Wasserman at the Cook Political Report and Nate Cohn at the New York Times cite a seven-point margin as the minimum required for a slim Democratic win. My model agrees.  It uses historical voting data back to 1940 to estimate the relationship between seats and votes. I include adjustments for redistricting after each Census and for the decline in political competition since the 1994 “Contract with America” midterm. Using those data I estimate the Democrats need at least 53 percent of the national popular two-party vote to win a majority in the House of Representatives.

From Vote to Seats: The Senate

Unlike House districts, Senate seats cannot be gerrymandered because they constitute entire states.  That makes the Senate more competitive than the House.  There is no equivalent “bulwark” in the Senate; winning half the popular vote generally translates into about half the seats.  Since the model predicts that the Democrats will win a majority of the 2018 popular vote for Senate, we should expect the Democrats to win a majority of the 35 Senate seats at risk.  Winning 18 of those seats would not be enough to flip control of the Senate because of the Vice President’s tie-breaking vote.  Nineteen seats would give the Democrats control.

From Here to November

Can the Republicans rebound between now and election day?  Unfortunately, recent history suggests they will face even larger obstacles in November than they do todayPresidential approval generally declines as the election nears, and the opposition party’s advantage on the generic ballot grows.

Job approval for first-term presidents fell on average about five points between May/June and October of midterm election years.  Trump might see a smaller decay because his current popularity is historically low, around 42 percent.  Presidents whose approval was above fifty percent in May/June saw a 3.3 percent drop in approval; those who started below fifty percent in May/June saw their ratings fall an average of just 1.9 percent.

In off-year elections, generic ballot polls for both of Obama’s midterms fell as the election neared.  For George W. Bush in 2006, Republicans recovered slightly from their early summer deficit, then watched support for their candidates crater in October.  Most observers credit that sharp decline to the Mark Foley scandal that fall.


The Strange Case of 1976

1976 was a horrible year for Senate Republicans; adjusting for that fact makes a slight difference to my 2018 predictions.

Re-examining the results for my original model of Senate elections, it was hard to ignore how poorly the model fit the data for 1976.  Here is a graph of the model’s predicted vote for Senate and the actual vote that shows what an “outlier” 1976 is.  While Truman rallied Senate Democrats in 1948, even that event just hovers on the edge of the statistical “margin of error.”  The Republicans’ failure in the 1976 election after Richard Nixon was forced from office stands truly alone compared to the rest of the postwar elections in my dataset.

If 1976 had been a normal presidential election year, the Republicans’ Senatorial prospects would have looked fairly rosy.  Gerald Ford was running for re-election, real personal income was growing at two percent, and the Democrats were defending seats won in 1970 by a (two-party) margin of 56-44 at the height of the anti-war and anti-Nixon fervor.  That generally pro-Republican climate predicts the GOP should have won nearly 52 percent of the popular vote for Senate.

But, of course, the 1976 election was anything but normal.  It was the first presidential election after the Watergate scandals had forced Richard Nixon from office in disgrace.  Rather than winning the popular vote by the predicted four-point margin, the Republicans could muster only the same share of the vote they won back in 1970, 44 percent.  Though a number of seats changed hands, at the end of the day the Democrats held the same 61-seat Senate majority they did before the 1976 election.

I can adjust statistically for the anomalous 1976 election by adding a “dummy” variable to my model that is one in 1976 and zero otherwise.  Adjusting for 1976 radically improves all aspects of my model.  Its predictive power as measured by adjusted R-squared rises from 0.43 to 0.56, and all the coefficients are more precisely estimated.

Adding this dummy variable implicitly treats Gerald Ford as different from other Presidents running for re-election.  Ford was apparently so compromised by Watergate that his presence at the top of the ticket did not generate the kind of support his fellow Republican candidates for Senate might have expected.  With the 1976 adjustment, the overall effect of Presidential candidacies rises from 2.5 to 3.1 percent, suggesting Ford’s performance was suppressing the estimate for other Presidencies.

Adjusting for 1976 also increases the compensating effect (“regression-toward-the-mean”) of the prior vote for a Senatorial class.  With the suppressing effect of 1976 removed, I now estimate that the Democrats’ lopsided Senate victory in 2012 should be worth about 2.3 percent to the Republicans this November, compared to the 1.9 percent figure I presented earlier with no adjustment for 1976.

For comparison to the chart above, here are the predicted and actual values for the model adjusted for 1976:

Including a dummy variable for 1976 sets its residual to zero and places its predicted value precisely on the line.  The largest positive outliers are now 1978 and two Presidential years, Truman in 1948 and Barack Obama in 2016.

The effects of this modification on the predictions in my earlier article are quite modest.  Without adjusting for 1976, I predicted the Republicans will win 48.1 percent of the popular vote if Trump’s approval rating is at forty and real disposable personal income rises two percent.  With the adjustment that figure rises to 48.4 percent.

The overall conclusion remains that no likely combination of factors predicts that the Republicans will win a majority of the popular vote for Senate this fall.



Can the Republicans Hold the Senate in 2018?

Historical voting patterns and current economic and political conditions predict the Democrats can win 18 of the 33 Senate seats in contention this fall and perhaps take back control of that body.

The departure of Paul Ryan as Speaker provides yet another piece of evidence in favor of the Democrats taking back the House of Representatives this fall.  Faced with this prospect Senate Majority Leader Mitch McConnell and other Republican legislators and strategists argue that the party must now focus its efforts on maintaining its one-vote lead in the Senate.  In this essay I examine the prospects for a Democratic Senate victory as well.

At first glance, the Democrats’ prospects seem quite poor. The current “class” of Senators running for re-election last faced their voters in 2012 when Barack Obama was re-elected President. That helped the Democrats win or retain a number of seats in unlikely places like North Dakota, Missouri, and Montana. Twenty-three Democrats were elected to the Senate that year, compared to eight Republicans and two independents.  With so many more Democratic seats up in 2018, we might expect the GOP to maintain or even expand its slim Senate majority this fall.

However the Democrats themselves held a similar advantage in the 2016 election but failed to win back the Senate. The Republicans needed to defend the 24 seats they won during their “shellacking” of the Democrats in 2010, while just ten Democratic seats were at risk. Nevertheless the Democrats managed to swing only two seats into their column in 2016, suggesting that the relative number of seats at risk may not be a very powerful predictor of the eventual outcome.

One factor in the Democrats’ favor is the absence of Donald Trump at the top of the ballot come November.  Updating the chart from my previous article to include the 2016 election does not change this basic fact:

The Democrats did better than expected in the 2016 Senate elections, winning 53.7% of the popular vote.  That raised the average for the six “open seat” elections, when the President was not standing for re-election, to 48.8% from the figure of 47.9% I reported in 2016.  In general, though, having the President at the top of the ticket adds, on average, four percentage points to the vote for his co-partisans in the Senate. The Republicans are thus starting from behind despite the disparity in seats at risk.

Along with whether the President is standing for re-election, I identified three other factors that have systematically influenced the vote for Senate candidates since World War II:

  • the size of the previous popular vote for the current “class” of Senators facing re-election;
  • the President’s job approval ratings in polls near mid-term elections; and,
  • the year-on-year change in real disposable personal income per-capita.

That framework enables us to examine some possible scenarios for this fall.

There is some evidence that the size of the vote for a Senatorial class does influence how well that class fares six years later. The party advantaged in one election sees a drop in support when facing re-election.  In 2018 that factor helps the Republicans by about 1.8 percent, still not enough to overcome the deficit from not having the President at the top of the ticket.

As I find for House elections, a President’s “job-approval” rating influences the outcomes of Senatorial races.  A ten-point increase in the percent of Americans approving of the President’s performance results in a 1.2 percent increase in support for Senatorial candidates of the President’s party.

On the economic front, the Republicans have repeatedly touted their recent tax cut as providing an impetus to support for their party. As Bloomberg reported after a GOP retreat in February, “With President Donald Trump’s stubbornly low approval ratings and historical trends suggesting they’ll lose seats in the November mid-term elections, party leaders told lawmakers their salvation lies in hammering on the message that the tax cuts passed at the end of last year are putting more money in voters’ pockets.”

My earlier findings confirm that growth in real disposable personal income does have an effect on vote for the President’s co-partisans in the Senate.  In the simulations below, raising personal income growth by one percentage point yields an increase in the predicted vote for Republican candidates this fall by about 1.4 percent.

We can put these findings together and examine the model’s predictions for the Senatorial vote in 2018 given different combinations of presidential job-approval and growth in personal income.

or, visually,

In no likely scenario does my model predict a popular-vote majority for Republican Senate candidates this fall. Which of these scenarios might we see play out in November?

The job-approval figures I use in this analysis come from Gallup, since only that organization has published these ratings as far back as the 1940s.  Like most other pollsters Gallup has reported a slight rebound in Trump’s approval figures over the past couple of months, but they are still running around forty percent.

Gallup has begun reporting weekly ratings this year which explains the much lower variability of its 2018 results.  Also Gallup’s polling shows no systematic partisan bias in either direction but hews closely to the polling “consensus.”  However, Gallup may underestimate approval for President Trump by about 1.6 percentage points because it surveys all adults rather than just registered voters.

Personal income growth, despite the tax cuts, does not provide much reassurance for the Republicans either. Real disposable personal income per capita has shown only modest growth over the past few months, running at about 1.5 percent on an annualized basis.

While there has been a slight downtick since the January peak of 1.6 percent, it seems plausible that real personal income per-capita will show a gain in the neighborhood of 1.5 percent by November.

From the table of predicted vote outcomes, a job-approval rating of 40 and even two percent personal income growth corresponds to a 52-48 split in the popular vote favoring Senate Democrats this fall.  Because there is little “bias” in the translation from votes to seats in the Senate, winning 52 percent of the popular vote translates into a two or three seat margin this fall.

Using the equation shown in that chart, if the Republicans win 48 percent of the popular vote for Senate this fall, they should come away with only fifteen seats (45% of the 33 seats at risk).  A three-seat victory for the Democrats would flip control of the Senate even given the Vice President’s deciding vote.

How Doug Jones Won

Comparing last night’s results for the Special Election in Alabama to prior elections in that state shows the path which brought Doug Jones his unlikely victory.  Like all special elections, the Alabama election unsurprisingly failed to mobilize as many voters as last year’s Presidential contest, but the turnout last night did exceed the 2014 mid-term figure by four percentage points.  Alabamians apparently thought this race was worth the effort.

Jones gained nearly as many votes in this special election as Hillary Clinton polled in November, 2016, despite an electorate 37 percent smaller than for the Presidential election.

Doug Jones expanded the Democratic electorate in the largest Alabama counties when we compare his results to those polled by 2014 Democratic gubernatorial candidate Parker Griffith.  (There was no Democratic candidate for Senator on the ballot that year.)

Jones won 671,151 votes last night, an improvement of 57 percent statewide over Griffith’s 427,787 total three years ago.  The graph shows clearly that Jones’s advantage grew as a function of county size, measured here by total turnout.  His best performances were in Madison County, where aerospace center Huntsville is located; Shelby County, which includes suburban Birmingham; and Baldwin County, just east of Mobile.  Moore carried both of the latter counties over Jones but by severely diminished margins compared to Republican performance in prior elections. Jones doubled the Democratic vote in Lee County, home to Auburn University, and made a significant gain in Tuscaloosa where the University of Alabama is located.

Moore’s support, in contrast, was strongest in the smallest counties.  Here I am comparing Moore’s performance last night to the total vote cast in the Republican primary runoff election against Luther Strange late last September.  Moore needed to mobilize sufficient numbers of Strange voters to add to his own totals going into last night’s election.  He failed to do so.

Statewide Moore received about 35 percent more votes than he and Strange together polled in September.  However Moore saw his smallest gains in the largest counties, the opposite of the pattern we saw for Jones.

Running a simple regression of Jones’s lead over Moore against demographic variables shows the dominant power of mobilized African-Americans with smaller effects for the proportions of Hispanics and people with a college degree.

In a county with no blacks, no Hispanics, and no one with a college degree, Moore would beat Jones by 79 points, e.g., 89-10.  The size of the black population played the most important role in determining support for Jones.  His lead expanded by 1.6 percent for every one percent increase in the percentage of blacks.  Looking at the “standardized” coefficients, which provide a measure comparable “importance,” the proportion of a county’s residents holding a college degree also mattered, but its influence was a bit under half that of the size of its black population. Variations in the size of a county’s Hispanic population had a much smaller effect and just achieves significance.These three variables alone account for about ninety-four percent of the variance in Jones’s lead over Moore.

Jones’s victory with a margin of 1.5 percent over Moore suggests that the polls that included cell phone respondents were right on track.  As I wrote yesterday, “Taken together, [my analysis of polling data] suggests that Jones has averaged a 1-2 percent lead in polls taken since the Washington Post story that included calls to cell-phone users.”  Polls limited to landline households, which predicted a Moore victory, were off the mark.