How Stable is the Generic Ballot?

by Nate Silver @ 6:53 AM
Bookmark and Share Share This Content

I’m fairly deep in the weeds of building our House forecasting model, which we hope to debut for you at some point late next week. We’re basically taking a “kitchen sink” approach — that is, looking at five or six different sources of information (polls, ratings by professional forecasters like Cook and CQ, fundraising data, etc.), and seeing what has had the most predictive power over the past six election cycles. This is not an easy thing to do — the data-collection efforts alone are formidable.

One of the challenges I’ve faced is in coming to grips with the generic ballot, which is the primary indicator of the nationwide standing of the two major parties. The basic question is to what extent the generic ballot ought to take precednece over local-level indicators: for instance, if the generic ballot looks really bad for one party (as it does for the Democrats this year), but the local polls are more favorable, which indicator tends to prevail? I don’t have an answer to that yet — you’ll have to tune in next week, I suppose. Still, there are some questions about the generic ballot that I’m now in a better position to address.

For instance: how stable is the generic ballot? I don’t mean individual polls of the generic ballot, which in the case of Gallup and some other organizations, can be quite “bouncy” from week to week. Rather, suppose that you’re able to remove most of this noise: how quickly can the underlying, macro-level dynamics change when it comes to elections to the Congress?

The way that I’ve evaluated this is to collect all generic ballot polls since 1998 and looked at what they would have told us at certain intervals before each election. Specifically, I built Pollster.com-style LOESS regression curves around the generic ballot polls, and evaluated the result they would have projected on the morning of the election, and then at 10, 20, 30, 45, 60, 90, 120, 150, 200, 300 and (where there is sufficient early polling) 400 days beforehand. There is no “cheating” allowed: for instance, if a poll came out 199 days before the election, it isn’t used in the 200-day forecast, since it wouldn’t have been available to us at that time.

The one “fancy” thing I have done is to build in an adjustment to translate registered voter polls into likely voter polls, as we now do for our Senate forecasts. This is worthwhile: although there are some cycles (2006, 2008) where there is little systemic difference between registered voter and likely voter polls, there are other cases (1998, 2000, 2010) where likely voter polls tend to be 4 or 5 points more favorable to Republicans, and accounting for this as early as possible tends to improve the stability of one’s forecasts. (There have been no cases recently in which Democrats performed demonstrably better in likely voter polls than in registered voter polls: in years where Democratic mobilization is strong, the two say about the same thing; in bad years for the party, registered voter polls lowball the Republican position by several points.)

Here, for instance, is what the trendline would have looked like at various points during the 2008 election cycle:

As you can see, 2008 was a rather stable cycle. Democrats maintained a consistent lead of about 10 points throughout the entirety of the cycle, and that carried forward to election day, when they won the national popular vote by 11 points.

Several other cycles were also quite stable. For example, 2002:

In spite of the potential idiosyncrasies resulting from redistricting (not to mention 9/11) that cycle, the generic ballot remained quite well-behaved all year.

Likewise, in 2000:

A generic ballot projection a year in advance of election day would have told you pretty much the same thing as one on Election Eve. This is in spite of the fact that the Presidential race that year was one of the more volatile in recent memory.

Nor did the generic ballot move very much at all in 1998, a low-turnout year in which Democrats arguably underperformed given Bill Clinton’s favorable standing at that time:

On the other hand, the generic ballot moved a fair bit in 2004. Over the summer, the Democrats built up a generic ballot lead of as large as 8 points, but it evaporated by November; adding insult to injury, the Democrats underperformed their generic ballot standing (losing the national popular vote by 3 points) on Election Day:

There was also a fair amount of movement in 2006 — and this time, it was uniformly in a direction favorable to Democrats — although there were some weird polls late in that cycle that perhaps overshot the mark a bit:

***

In spite of these partial exceptions, the generic ballot is generally fairly stable — almost certainly more stable than something like Presidential polling. This makes a certain amount of sense: whereas something like a gaffe or a victory in a debate can considerably altar the outcome of a Presidential race, there are 435 separate elections to the House, and gaffes in individual districts tend to cancel one another out. Instead, things usually boil down to the national mood that was established during the first half of a Congress’s two-year term.

This is basically bad news for Democrats in the context of this cycle: a last-minute reversal of fortunes is unlikely. Where we sit right now, about 75 days before the election, the generic ballot will be off, on average, by only about 2 points from what it will read on Election Morning. The Democrats’ standing is poor enough now that a 2- or 3-point shift in their direction would not really be enough to prevent the party from enduring significant losses in the House.

With that said, there is another issue at hand: how much does the generic ballot really tell us about what will happen on Election Day? It might be the case that the generic ballot is fairly stable, but that doesn’t necessarily mean it’s all that useful an indicator. In addition to the fact that the consensus of polls (however careful we are about calibrating it) might be off in one or the other direction, there’s also the fact that the thing which the generic ballot is ostensibly trying to predict — the national House popular vote — is relatively irrelevant to the disposition of the chamber, or the number of seats that each party earns. Instead, what we want to know is how the generic ballot translates into each of the 435 congressional districts; this is the sort of problem that we’re hard at work upon.

Still, to expect that the national environment will just spontaneously get better for Democrats is probably not realistic. They’ll have a poor hand to play, and the task is basically in figuring out exactly how bad the current milieu will translate in terms of a loss of seats.