Gallup has been synonymous with polling for decades. But the pollster that made its name forecasting a Franklin Roosevelt win over Alf Landon in 1936 has decided, at least for now, to stop polling the presidential horse race in favor of focusing on issues. It won’t poll the 2016 primaries and may not poll the general election.
Gallup Editor-in-Chief Frank Newport told me that he wants to contribute to making the “government process work better” and that he thinks issue polling can do that better than horse-race polling.
So how should we view Gallup’s decision in terms of election polling in general?
Gallup had a very bad 2010 and 2012
Gallup’s decision to get out of the horse-race game comes after two consecutive elections in which its results were way off. Gallup’s final generic congressional ballot in 2010 had Republicans winning by 15 percentage points; they won by 7 points.
In the runup to the 2012 presidential election, Gallup seemed to ignore warning signs that its polls didn’t include enough minority voters, and the pollster’s final survey showed Mitt Romney ahead by a point. President Obama won re-election by 4 percentage points. Gallup’s longtime relationship with USA Today ended after 2012. (Gallup didn’t publish likely voter horse-race polling during the 2014 midterm campaign.)
So Gallup was bad at horse-race polling, and it’s good that it’s stopping, right? Not really.
We’re worse off without Gallup
Gallup uses rigorous polling methodologies. It employs live interviewers; it calls a lot of cell phones; it calls back people who are harder to reach. More than that, it took the criticism it received after the 2012 election seriously, even bringing in outside help to figure out what went wrong. Gallup rates as solidly average in FiveThirtyEight’s pollster ratings in large part because of those techniques. It’s had two bad elections recently, but it’s never a good idea to judge a pollster on just a couple of election cycles; Gallup has also had good years.
Polling consumers are far better off in a world of Gallup’s than in a world of Zogby Internet polls and fly-by-night surveys from pollsters we’ve never heard of. There is plenty of shadiness in the polling community, and Gallup seemed to be opening its doors.
Gallup says it will still conduct issue polling, but here’s the problem: Elections are one of the few ways to judge a pollster’s accuracy. And that accuracy is important: We use polls for all kinds of things beyond elections. How do Americans feel about the economy? Do elected leaders have the trust of the public? Is there support for striking a deal with Iran? By forgoing horse-race polls, Gallup has taken away a tool to judge its results publicly.
Still, Newport told me that there are other ways to check the accuracy of Gallup’s polls, including by comparing its results to government surveys. “We were able to track after the Affordable Care Act went into effect the drop in the uninsured, which turned out to be very close not only to other polls but the government polling when it finally came out,” he said. Newport also suggested that Gallup might conduct horse-race election polls without publishing the results.
Polling will survive
There are still plenty of good polls, and Gallup’s decision, by itself, doesn’t change the overall polling landscape that much (of course, if Gallup is simply an early adopter and other pollsters follow suit, that’s another question). ABC News, CBS News, CNN, Fox News and NBC News have all published live-interview primary polls in the past couple of months (they all have a better track record than Gallup, according to our ratings). We have more state-level primary polling in Iowa and New Hampshire in this presidential election cycle than through the equivalent point in the 2012 primaries. Until these pollsters also exit, we’ll still have quality data to pore over.