“Ghostbusters,” a revival of the 1984 original, hits theaters nationwide on Friday. As a reboot of a beloved, male-led science fiction film from the 1980s with a female-led cast, the reboot has proved somewhat controversial in the circles you would expect. But regardless of the quality of the film, it serves as a perfect demonstration of why internet movie ratings are inherently a problem.
Most fundamentally, single-number aggregations — like those used by sites such as Rotten Tomatoes, Metacritic and IMDb — are a pitiful way of explaining the diverse views of critics. More specifically, a vocal portion of men on the internet — shall we say — go out of their way to make their voices heard when it comes to judging entertainment aimed at women, and that appears to be happening with the new “Ghostbusters.”
But let’s back up. Last year, as part of an investigation into the inflated ratings on Fandango’s website, I looked at the world of online movie ratings in general. The moral of this story: Each site that aggregates ratings and reviews has its own skew one way or another, and it’s up to the user to determine which heuristic most accurately matches what they’d consider an ideal rating. (Also, don’t trust always-positive movie reviews from sites trying to use that review to sell you movie tickets. That, too.)
Here were the ratings curves for some of the most popular aggregators when I did this analysis in October:
Earlier this year, I also looked at IMDb’s user rating skew for television shows. Essentially, male users were more likely to rate television shows with a female-heavy audience lower than female users would rate male-centric television lower. Men were tanking the ratings of shows aimed at women.
But this “Ghostbusters” thing? It lays bare so, so much of what we’re investigating when it comes to the provenance and reliability of internet ratings.1 Namely, they’re inconsistent, easily manipulated and probably not worth half the stock we put in them.2 Here are a few stats I collected early Thursday for the new “Ghostbusters” movie:
- IMDb average user rating: 4.1 out of 10, of 12,921 reviewers
- IMDb average user rating among men: 3.6 out of 10, of 7,547 reviewers
- IMDb average user rating among women: 7.7 out of 10, of 1,564 reviewers
The movie isn’t even out in theaters as I’m writing this, but over 12,000 people have made their judgment. Male reviewers outnumber female reviewers nearly 5 to 1 and rate “Ghostbusters” 4 points lower, on average.
But it’s not just IMDb, there are serious disagreements on “Ghostbusters” across the whole universe of ratings aggregation sites:
- Metacritic score of 61 out of 100, based on 41 critics.
- Of the reviews aggregated by Metacritic, 24 were positive, 16 were mixed, 1 was negative.
- Tomatometer score of 74 percent “fresh” based on 138 reviews.
- Of those reviews, the average rating was 6.5 out of 10.
- Looking only at top critics, it had a Tomatometer score of 54 percent “fresh” based on 35 reviews.
- Of top reviewers, average rating was 6.3 out of 10.
What I find fascinating here — probably because it’s related to a larger story about Rotten Tomatoes I’m working on — is the difference between the average review on Rotten Tomatoes and the Tomatometer score. Rotten Tomatoes’ Tomatometer uses a simple, binary heuristic to aggregate reviews: Did the reviewer like the movie or not. It doesn’t matter if they somewhat liked a movie or absolutely loved it, and it doesn’t matter if they left a theater somewhat dissatisfied or nauseated with loathing. The review is sorted into either the “fresh” or “rotten” bin regardless of intensity, whereas Metacritic — in addition to some other funky modeling — does try to capture those degrees of intensity.
So even though 74 percent of critics enjoyed the film, according to Rotten Tomatoes, “Ghostbusters” has an average score of 6.5 out of 10, so reviewers clearly didn’t love the film. A similar issue, only in the opposite direction, is apparent in the site’s top critics score. “Top critics” is a somewhat exclusive category on Rotten Tomatoes, encompassing prolific and experienced critics at top publications with high circulations. Contrary to the hoi polloi of the Tomatometer, about half of reviews of “Ghostbusters” from “top critics” were negative, but it earned a middling-but-positive 6.3 out of 10, on average.3
So, do you have a preconceived notion of how good or bad the “Ghostbusters” movie is going to be? Of course you do, you clicked on an article with “Ghostbusters” in the title. Well, there’s plenty of statistics to choose from here. But looking at them all, here’s the picture we get:
Based on the IMDb reviews, a lot of men on the internet (who may or may not have actually seen the film) really hate the new “Ghostbusters.” A lot of women on the internet (who may or may not have actually seen the film) seem pretty into it. Based on the Metacritic score and the average Rotten Tomatoes scores, “Ghostbusters,” like most summer movies, is merely a mediocre-to-good film, critically speaking. And based on the Rotten Tomatoes scores for top critics, professional critics are split on whether it’s worth seeing. But based on the larger pool of critics, three out of four of them think it’s worth a ticket.
I, like the vast majority of people on earth, have not yet seen the new “Ghostbusters” film. And to be honest, I don’t have a lot of skin in the game — based on my birth date alone, “Pokemon Go” is a bit more in line with manipulating my nostalgia for money than busting ghosts is.
The point is that this is a hugely instructive case for why internet ratings need to be approached with way more nuance than they currently are.4 People put far too much faith in numbers that are preliminary, decontextualized and, in the end, oversimplified.