Is there such a thing as a perfect work of art? You could argue that Michelangelo's “Pieta,” Orson Welles' “Citizen Kane,” Frank Gehry's Guggenheim Bilbao and Picasso's “Guernica” come close, but perfection is an abstract concept and not really something that is ever attainable.
Conversely, are there works of art that are entirely without worth? The big screen oeuvre of Rob Schneider and the entire genre of hair metal come to mind, but again it's impossible to argue that something is utterly and completely devoid of merit.
That being said, there are more and more websites that are trying to turn critical appraisal into mathematical precision, often giving creative works either perfect scores of 100 or scores of absolute zero. I have taught a class on reviewing the arts for several years now, and while I encourage my students to use sites like Rotten Tomatoes and Meta Critic to find reviews to read and study, I caution them about those sites' overall scoring systems.
I recently took some time to examine how the sites calculate their scores and discovered some disturbing trends. On the surface, they both mean well and attempt to provide consumers with a valuable service—by aggregating and averaging the critical appraisals of various works of art, they try to give a clear, objective sense of whether something is worth your time and money.
The process appears pretty straightforward when critics use some type of scoring mechanism themselves. For example, Roger Ebert gives the films he reviews a score of one to four stars. That means that if he gives a film three stars, Meta Critic averages him in as a 75. But wait, a 75 is a C at an academic institution, and anyone who reads Ebert knows that three stars from him is a heck of a lot better than a C. So, the math is already fuzzy when it comes to these critical averages.
But what happens when a critic doesn't use a rating system? Many A-list critics do not use these systems (Ebert is an exception), so how do the sites average in these reviews? Rotten Tomatoes says it doesn't use a review in their averaging if the critic does not use a scoring system. According to their site: "Each critic's original rating scale (star, letter grade, numeric) is converted to a number between 1 and 10, and then the numbers are averaged. Reviews without original ratings are not counted, and a minimum of five reviews with original ratings is required."
That would mean the opinions of many of the most influential and experienced critics are not computed into the Rotten Tomatoes average. But wouldn't that seriously skew the rating? Things get even stranger when you search on Rotten Tomatoes for the work of the New York Times' Manohla Dargis, an esteemed critic who does not use a scoring system. You actually find scores next to her reviews. For example, the site lists a score of 3.5/5 stars for the recent film “Chronicle,” and at the top of her index page on Rotten Tomatoes it says, Manohla Dargis "Agrees with the Tomatometer 76% of the time." Something definitely does not compute.
Dargis’ reviews on the Times’ site actually do include a 1-5 rating system. However, this rating mechanism is for readers who want to weigh in on a film, and it is not used by Dargis, herself. If Rotten Tomatoes is actually averaging in that information as Dargis’ opinion, then the math is way off. (I reached out to Rotten Tomatoes for some clarification, and I have yet to receive a response.)
How does Meta Critic handle using reviews by a reviewer like Dargis who does not include a scoring system? Their approach is quite subjective. The site explains, "our staff must assign a numeric score, from 0-100, to each review that is not already scored by the critic. Naturally, there is some discretion involved here, and there will be times when you disagree with the score we assigned." So basically, if Dargis won't score her own reviews, Meta Critic will do it for her.
Where the practice of Meta Critic assigning scores falls apart most is on reviews where the website says Dargis felt a film was perfect (100) or worthless (0). The site lauds 33 of her film reviews with a perfect score of 100, and it slaps seven films with a complete goose egg. However, when you dig into some of the critiques, nothing is really perfect. In her review of Avatar (100 on Meta Critic), Dargis derides the film for "some of the comically broad dialogue," and when writing about Moneyball (also 100 on Meta Critic), she opines, "There are some overhead shots of the A’s emerald field too, including one of a large American flag being unfurled, that feel like the efforts of a director needlessly looking for big symbolic moments, perhaps particularly post-Sept. 11."
In the end, where does all this crazy critical computing leave us? It underscores the fact that reviews of the arts are always subjective and cannot be turned into hard numbers. I encourage my students to use both Rotten Tomatoes and Meta Critic because both sites aggregate and organize the work of a lot of smart and creative critics. But I advise them to avoid the numbers, focus on the words of the critics and decide for themselves.
Add new comment