Things seem to be going Roy Moore’s way. President Trump endorsed him. The Republican National Committee is back to supporting him. And Moore, who has been accused of sexual contact with women when they were underaged, has led by an average of 3 percentage points in polls1 taken within 21 days of the Dec. 12 special Senate election in Alabama. The betting markets give Moore about an 80 percent chance of victory — roughly the same chance they gave Hillary Clinton just before the 2016 presidential election.
Before Election Day last year, we advised caution, however — polls aren’t perfect at even the best of times, Trump had an advantage in the Electoral College, and there were a lot of undecided voters. So what’s our advice heading into the Alabama election? Well, it’s the same — be cautious — but for slightly different reasons.
A look at all U.S. Senate election polls since 19982 shows that their average error — how far off the polls were from the actual election result — is more than a percentage point higher than the average error in presidential polling. Also, Alabama polls have been volatile, this is an off-cycle special election with difficult-to-predict turnout, and there haven’t been many top-quality pollsters surveying the Alabama race. So even though Moore is a favorite, Democrat Doug Jones is just a normal polling error away from winning. (Or, by the same token, Moore could win comfortably.)
The polls in Alabama have swung back and forth between Moore and Jones over the past month. The Washington Post first reported on allegations against Moore on Nov. 9, and after that, surveys indicated that the race was moving in Jones’s direction. He held an average advantage of 5 percentage points in polls that were taken six or seven days following the story. Since then, polls have Moore ahead by 3 points, on average, although Jones led by 3 points in a Washington Post poll. (That’s the only recent survey that meets FiveThirtyEight’s gold standard.3)
Latest Alabama Senate polls show Moore ahead
Surveys taken after accusations of sexual misconduct against Roy Moore were first reported by The Washington Post on Nov. 9
| POLLSTER | DAYS SINCE ACCUSATIONS | DOUG JONES | ROY MOORE | MOORE MARGIN | |
|---|---|---|---|---|---|
| Opinion Savvy | 0 | 46% | 46% | 0 | |
| Gravis Marketing | 1 | 46 | 48 | +2 | |
| Change Research | 2 | 40 | 44 | +4 | |
| Emerson College | 2 | 40 | 49 | +9 | |
| JMC Analytics | 2 | 48 | 44 | -4 | |
| Strategy Research | 4 | 43 | 49 | +6 | |
| Fox News | 6 | 50 | 42 | -8 | |
| Gravis Marketing | 6 | 47 | 42 | -5 | |
| Change Research | 7 | 46 | 43 | -3 | |
| Strategy Research | 12 | 45 | 47 | +2 | |
| Emerson College | 18 | 47 | 53 | +6 | |
| Change Research | 18 | 44 | 49 | +5 | |
| JMC Analytics | 19 | 44 | 49 | +5 | |
| Washington Post | 21 | 50 | 47 | -3 | |
| YouGov | 22 | 43 | 49 | +6 | |
| Emerson College | 23 | 46 | 49 | +3 | |
| Gravis Marketing | 24 | 48 | 44 | -4 | |
| Strategy Research | 25 | 43 | 50 | +7 | |
When an electorate seems to be volatile and there aren’t many high-quality pollsters in the field (by traditional standards), it’s wise to tread lightly. It’s possible that voters will turn on Moore again in the final week of the campaign.
But even if Alabama’s special election were just a normal Senate campaign with normal candidates, a lead in the low single digits would be far from secure. Simply put, Senate polling has not been especially predictive over the past 10 cycles. Among the 2,075 Senate polls in the FiveThirtyEight database that were taken within 21 days of an election, the average error has been 5.1 percentage points. And that has been fairly consistent across cycles. The 2016 Senate polls featured an average error of 5.2 percentage points.
The average error tends to understate the chance for a large miss, however. The true margin of error (i.e., the 95 percent confidence interval) is significantly wider. For the polling in Senate campaigns since 1998, it’s closer to +/- 13 percentage points for any individual poll. There hasn’t been a single public poll taken in Alabama since the Republican primary runoff that has given either Jones or Moore that large of a lead.
And the chance of a big error may be unusually high in Alabama. Because of the unusual timing of the election, pollsters may have a difficult time determining who is going to turn out to vote. In the 59 Senate polls for special and runoff elections since 1998 that we have (admittedly, not a huge sample), the average error has been 5.8 percentage points. That’s nearly a point larger than the average error for all Senate polls.
There’s more. Alabama’s election could present even more challenges to pollsters. Some Republicans may be ashamed to admit to pollsters that they are voting for someone facing sexual misconduct allegations. Others may be reluctant to vote at all because of the scandal. As I have previously pointed out, slight differences in who pollsters expect to turn out can make major differences in how big Jones’s or Moore’s polling lead is. Both the Post and YouGov have shown that likely voters are more favorable to Moore than the larger universe of registered voters. So, Moore may have a turnout advantage, though it’s difficult to know.
The polling in Alabama does have one thing going for it. Because the campaign has become high-profile, eight public polls have been taken that fall within the three weeks leading up to the election. Based on past Senate campaigns, the more polls that are included in an average the more predictive it tends to be.
In 72 Senate elections since 1998 in which the FiveThirtyEight database had five to nine polls to average, the polling average has been off by an average of 4.2 percentage points. That’s about a point lower than the error of any single poll. It’s conceivable that by Dec. 12, we could have a total of more than 10 polls taken in the final three weeks of the campaign. Since 1998, the average error in the 81 Senate elections with 10 polls or more has been 3.5 percentage points.
Still, even if averaging polls gives us a better understanding of the race than looking at polls individually, that doesn’t mean the average will correctly identify the eventual winner. It’s a close race — close enough that the normal inexactness of polling gives either candidate a good chance of winning. Moore’s 3-point lead is smaller than the average error when there are only five to nine polls to average. Additionally, Moore’s average edge is smaller than the true margin of error for the average. Based on past campaigns, the true margin of error for an average of five to nine surveys is about +/- 10.4 percentage points.
Of course, Democrats shouldn’t go into Alabama assuming that a poll miss will necessarily benefit Jones. In fact, the favorite is just as likely to benefit as the underdog. Just last month, for example, Democrat Ralph Northam won the Virginia gubernatorial election by 9 points after an average of polls had him winning by only 3. An error of 6 points or more that goes in Moore’s direction could turn this race into a relative blowout.
Finally, it’s also possible that the Alabama polls are close to perfect. In the 2016 Louisiana Senate runoff, the last Senate election not to take place on Election Day, the final 21-day average was off by only 2 points. Just because the chance of a polling miss is relatively high in Alabama doesn’t mean it will occur.
The bottom line is that with less than a week to go in the campaign, Moore seems to have the edge — but he’s far from a sure thing.