Scare Headlines Exaggerated The U.S. Crime Wave

We’ve updated this article with the homicide data from Anaheim, California, which we received on Sept. 14.

If you’ve read reports of a U.S. crime wave this year and wondered how many cities it was really affecting, you’re not alone. We’ve spent the last week trying to answer that question and have compiled 2015 homicide data for the 60 biggest cities. The results confirm that there has been an increase in homicides this year in big U.S. cities of about 16 percent.

But that doesn’t come close to reversing the long-term decline in homicides. And it’s a less dire picture than the one painted by reports in several large media outlets, which generally highlighted those cities that have suffered the biggest increase in homicides.

The reports have been based on just a small, possibly cherry-picked sampling of cities. The country’s broken crime-data system makes it impossible to know what’s happening everywhere, and the “if it bleeds, it leads” journalistic imperative means the places we hear about often are the biggest outliers.

The wave of crime-wave reporting began this spring with NPR, CNN, the BBC and USA Today, based on just a handful of cities. In August, after the Major Cities Chiefs Association (MCCA) announced results of a survey of a few dozen of its members about crime in their cities, many other outlets — including Reuters, Voice of America and Time — added to the chorus. And last week, The New York Times put the crime wave on its front page, saying that “cities across the nation are seeing a startling rise in murders.” The Times article was accompanied by a chart showing crime trends in 10 cities, which in turn provoked refutations from The Washington Post and the Marshall Project, based on fewer than 20 cities each.

Ideally we’d have current data for all of the roughly 18,000 U.S. police departments that report their crime numbers to the Federal Bureau of Investigation, just as we have up-to-date information on employment and other economic trends. But the latest numbers from the FBI’s Uniform Crime Reporting (UCR) program can’t tell us anything about crime this year: They run through June 2014.

So we narrowed our focus to the 60 most populous cities in the country — 60, because the 10 highlighted by the Times ranged from No. 1 (New York City) to No. 60 (St. Louis). For each city, we sought the year-to-date and year-ago homicide counts, the crime figures that researchers say are the least subjective and hardest for departments to manipulate. We set out to get as close to a complete data set as we could from media reports, the chiefs survey and the websites of state UCR programs and city police departments — with some help from leads suggested in response to a call for help on Twitter.

When we couldn’t find the data published anywhere, we contacted the departments — and as a byproduct got some insight into the wide variation in crime-reporting capabilities even among the nation’s biggest cities. Some departments got back to us within a couple of hours (e.g., Phoenix and Raleigh, N.C.). Several had the data on their website, broken down by month and going back several years (Tulsa, Oklahoma, and Riverside, California). Others treated the inquiries as public-records requests and said they’d need up to 10 days to get back to us (Anaheim, California).

As a result, the data is a bit of a mishmash. Some is from media reports, which tend to include homicide numbers in articles about a cluster of homicides, so they could be skewed slightly upward. Some cities had data only through May 31; others sent us numbers through Monday. Some media reports used vague end dates such as “mid-August” or somewhere between Aug. 11 and 31; for those, we estimated a date. Some of the media reports we relied on showed totals for murders, and cities generally follow FBI reporting conventions in removing homicides that are ruled justified from their counts. We traded some reliability and comparability of data for timeliness.

We ended up with homicide numbers for all of the 60 biggest cities. And in 26 of them, homicides were up by 20 percent or more from a year ago. Overall, homicides were up by 16 percent.¹ But the picture varies a lot by city: Homicides are up 76 percent in Milwaukee, but down 43 percent in Boston. They’re also down in 19 other cities.

Most of these changes aren’t statistically significant on a city level. Even amid the national upward trend, and in some of the country’s most populous cities, homicides remain, thankfully, a rare event. That means some increases that look large on a percentage basis affect the raw totals only slightly, to an extent that could arise by chance alone. A 20 percent increase in Seattle sounds a lot more significant than an increase to 18 homicides from 15 through Aug. 29. Homicides in Arlington, Texas, through Aug. 31 are down by 50 percent — to four from eight.

To determine whether changes like these were statistically significant, we estimated how likely it would be to see the changes we’ve seen in the cities by chance alone. In doing so we followed the lead of Bruce Frederick, a senior research fellow at the Vera Institute of Justice, who wrote a commentary article on the subject for the Marshall Project last week.²

The general idea is this: Assume the 2014 homicide rate is the true rate and then calculate how much of a statistical outlier this year’s rate is. If the 2015 rate would have arisen by chance alone less than 5 percent of the time, we deem it statistically meaningful.³ In 16 of the 60 cities, there was a significant increase; we’d expect about three cities to show an increase that big by chance alone. In two cities — Boston and Arlington — there was a significant decrease.

That means that most cities have not shown a significant increase in homicides over the last year. But the overall increase of the cities is highly statistically significant. It’s the same principle that governs the interpretation of a national presidential poll: The sample of respondents in any one state could be so small that there isn’t enough data to say where the race stands there, even if together the states give one candidate a big nationwide lead.

Nonetheless, there are some big reasons not to assume that crime is on a long-term increase. While a 16 percent increase in U.S. major-city homicides is statistically significant, it comes after decades of declines — the murder rate fell by more than half nationally from its peak in 1980 to 2012.

And big annual fluctuations aren’t unusual. For instance, in 2009, 17 of today’s 60 most populous cities had statistically significant decreases in the number of homicides, while just four had increases. In 1998, those numbers were 20 and four, respectively. Just a decade ago, in 2005, there were 15 cities with significant increases, roughly as many as this year. In 1986, 28 cities had significant increases; just two had decreases.

A big jump or fall in murders in a city in one year could be a statistical fluke (variation in the percentage of gunshot victims who die from year to year) or the result of factors that subside the next year — for instance, this year has been unusually hot, and hot weather has been associated with higher crime in some studies. We asked cities to send us their data for years before 2014, and 10 provided numbers for 2013. In seven of them, whatever the trend has been this year reversed the trend from a year earlier. For example, there were five homicides in Riverside this year through July 31. That’s down from nine through July 31, 2014 — which was up from five through the same date in 2013. In Tulsa, the homicide count through July 31 was 38 in 2013, 21 last year and 38 this year. In these cities, it’s 2014, not 2015, that looks like the outlier. And that may be true more broadly: The latest UCR report for cities with at least 100,000 people shows that total homicides in the cities in the first six months of 2014 were down by 10 percent from the same period in 2013.

Our data-collection effort also allows us to compare the cities whose homicide data was included in prior compilations with those whose data wasn’t. And what we found is that The New York Times highlighted cities with unusually large increases. The Times published a table showing homicide data for 10 cities. Cumulatively, homicides were up 28 percent in those cities. In the other 50 cities for which we gathered data, homicides were up only 9 percent. The text of the Times table specified that it only showed cities where murder rates were up from a year ago. The article also mentioned that “homicides remain at a relatively steady rate this year” in a few cities but didn’t provide data for those cities.

Before we set out on our own data-gathering exercise, we asked the Times for more information about the homicide data in cities where the numbers hadn’t increased so far this year and that weren’t included in its table. Times spokeswoman Danielle Rhoades Ha replied in an email, “Our story discussed the rising murder rates in more than 30 cities and went into detail on some of them. The story also made clear that there are cities where the murder rate is not increasing. We are not going to discuss the story in further detail.”

Cities with bigger homicide increases also were overrepresented in the data from the chiefs survey. The 26 cities among the 60 largest in population whose police departments responded to the MCCA survey showed a cumulative increase of 22 percent.⁴ Cities that did not appear in the survey results or the Times table had only a 5 percent increase in homicides.⁵

Darrel Stephens, executive director of the MCCA, acknowledged the potential flaws in his group’s survey in an interview last month. “It was one of those quick-and-dirty surveys,” Stephens said. “It had no scientific validity at all. It was to get a quick picture of what was taking place across the country.” He added, “A real survey guy probably wouldn’t give it much credence.”

Fewer than half of the U.S. police departments that are members of the group responded to the survey, which was conducted using the informal kind of SurveyMonkey poll we’ve used to survey pollsters.⁶

For all its flaws, the chiefs survey marked a significant advance in our knowledge of 2015 crime rates in big cities. The 32 cities that participated are 32 more than have 2015 data available in FBI UCR reports. They’re updated online only through June 2014.

“Year-old data is like looking at a star through a telescope,” Milwaukee police Chief Edward Flynn said in a telephone interview. “It’s nice to know what light was like on that star 2 million years ago, but it doesn’t tell us anything about what is happening now.”

Flynn is frustrated with the data disconnect faced by big-city police departments. Many of them employ data analysts and track their stats in real time — Flynn has 16 analysts, up from zero when he took over the department in 2008, and he rattled off his city’s crime stats from memory. Yet he and his peers must rely on informal exchanges and potentially skewed media reports to find out what crime trends those in other big departments are seeing.

“It’s up to us to Google newspapers and find out what’s happening in cities that are similar to us,” Flynn said. “There is no one-stop shopping for that.”

Some chiefs don’t do any data shopping at all. “In Metro Atlanta there are about 60 law-enforcement agencies,” Robbie Friedmann, professor emeritus of criminal justice at Georgia State University, said in an email. “I have never seen any sharing beyond jurisdictional boundaries.”

Flynn and other police chiefs met in Washington, D.C., last month to discuss the crime spike some were seeing in their cities. They identified potential factors that could account for the concurrent increase in distant cities, such as the growth of the heroin trade and of criminals’ use of automatic weapons; inadequate services for offenders to return to civilian life when they are released from prison; and a decrease in federal funding for local policing.⁷

And they discussed the difficulty chiefs face in getting data from one another’s cities.

“You look at economic data, at labor data, at all kinds of data the government collects that is immediately available,” Stephens said. “We don’t have anything like that in policing.”

There would inevitably be some problems with immediate release of national crime data. One concern is that data police departments submit is subject to revision. Some departments succumb to political pressure and suppress crime numbers. A more systemic problem is that departments don’t all define some crimes, such as assault, the same way.

Compiling the data currently takes a long time. State governments check the data submitted locally and then the FBI does the same, collating it from roughly 18,000 departments. That process can take more than a year, partly because participation is voluntary, and local agencies that do decide to report the data for a given year can wait to do so until April of the next year, FBI spokesman Stephen Fischer Jr. said in an email.

But Stephens of the chiefs association counters that economic data often gets revised after release, too. The provisional data still has value — there is plenty to learn from initial jobs numbers, for instance.

And the status quo isn’t tenable, according to some criminologists, who say cities need to know what is happening elsewhere to understand larger trends that may affect them, such as an increase in gang activity or the effects of a growing demand for drugs.

“The necessity to do a one-off, very unrepresentative, unscientific survey to find out the crime rates in major American cities is just appalling,” Richard Rosenfeld, a criminologist at the University of Missouri-St. Louis, said in a telephone interview.

The FBI is working on the problem, Fischer said. The bureau “is currently undertaking a wholesale redesign and redevelopment” of the UCR program, he said, with one goal being more frequent publication of crime data. Some cities also are reporting data on individual crimes to the FBI’s National Incident-Based Reporting System, but not enough for it to be a comprehensive resource.

In the meantime, why can’t police departments simply share their monthly data through, say, a Google spreadsheet or an email list? Stephens doubted that could work.

“Even to collect it for the 100 largest cities in America … is going to require some powerful computing, some quality-control checks and reminders to people that you have to get the data in,” he said. The low usage rates for an FBI database that is meant for departments to share data on violent crimes in their jurisdictions shows the challenges of ensuring compliance.

Even if the FBI could tell us how many crimes of each type were committed in each part of the U.S. in real time, the data would have limitations. Police departments only count crime that has been reported to them. Criminologists often focus on homicide numbers to get around that problem. Murders are the most comparable over time and space: It’s easier to define a murder and harder to avoid counting it to suppress crime rates. Yet changes in homicide rates reflect many other factors aside from policing. Poverty and gun availability can also affect homicide rates, yet are mostly outside police departments’ control.

“Over the last 40 years, this country has aggressively disinvested in the social safety network and has apparently come to the implicit conclusion that it’s a good idea to delegate America’s social problems to the criminal-justice system,” said Flynn, the Milwaukee chief.

It’s also unclear exactly what police departments would do if they had one another’s data. Although cities’ crime rates do tend to correspond with one another, individual trends can vary greatly. The chiefs association, for instance, asked survey respondents what percentage of homicide suspects or arrestees had prior gun-related arrests. Just 13 departments responded, and their answers ranged from 1 percent to 52 percent.

“As a general rule, the only thing important to cities is what happened in their community, not in other parts of the country,” said Jim Bueermann, former police chief in Redlands, California, and now head of the Police Foundation, a research organization.

Rosenfeld is more of a believer in the usefulness of other cities’ data. “The use is the same as the use for unemployment or heart-disease data nationwide,” he said. “You need to know whether your crime problem is purely local and idiosyncratic, for instance produced by an uptick in local gang conflicts, or whether it transcends local boundaries.”

“American cities have more in common than they have differences with each other,” Flynn said. “Mature cities should be able to look at broader trends and patterns of activity, instead of focusing relentlessly on the local angle of a story.”

CORRECTION (Sept. 14, 6:50 p.m.): An earlier version of this article incorrectly stated the magnitude of the decline in federal funding for the Byrne Grant Program from fiscal year 2010 to fiscal year 2015. Funding declined by 27 percent, not 73 percent.