what's in a star?

If you read Sam Sifton's review of Fishtag in last week's New York Times, you saw that he "awarded" Michael Psilakis's new restaurant zero stars, giving it a "fair" rating from the paper's additional labels of "satisfactory," "fair," and "poor" which accompany the paper's zero-star ratings. So I guess it's in the middle of the pack among the worst restaurants in New York. Maybe Sifton liked a few of the light fixtures or something.

Yet Sifton describes the restaurant as "occasionally marvelous," saying the chefs "grill striped sea bass with the best in the city" and serve a "marvelous lamb burger...that you may find yourself thinking about days later." They also prepare "some of the city’s greatest toast: grilled to smokiness, its crust giving way to a luscious interior" served with a selection of "estimable cheeses," and a choice from their "fine list of bracing whites and dry reds." They have an "excellent bruschetta," the mussels are "weird and exciting and new," and the octopus is also "estimable," with "intense and focused" flavors. You can wash it all down with some "excellent Ethiopian French-press coffee."

For a restaurant so full of marvelous, excellent and estimable things, he didn't seem to like it enough to give it any stars. Sifton definitely has some negative things to say (particularly about the decor, noise and layout of the menu), and having good toast, cheese and wine isn't going to earn anyone stars for difficulty, but this review certainly doesn't feel as negative as his other "fair" ratings. Or contrast it with Frank Bruni's one-star reviews, such as the one he gave Kobe Club in 2007 (which even received the slightly higher "satisfactory" rating) that prompted Jeffrey Chodorow to take out a full-page advertisement in the Times in response.

Although maybe I'm reading Sifton's review wrong, because Saveur publisher Merri Lee Kingsly was so ruffled she took a page from Chodorow's book and sent an angry letter to Eater defending Psilakis, telling Sifton to "stop being so rude to the chefs in our world that work so damn hard every single day and put every bit of passion and love into the food they cook..." (A lot of people have been wondering if her outburst was simply a way to bring attention to Saveur's recently announced restaurant reviews, which sound like they are going to be almost uniformly positive).

Anyway, all of this got me wondering about the relationship between a critic and the rating system of the publication he or she reviews for, which in turn led me to the Marc Shepherd's excellent analysis of star ratings and, in particular, the number of stars doled out by various New York Times critics. Mimi Sheraton, for instance, gave out zero-star reviews 20-25% of the time, which sounds unbelievable in comparison to more recent critics, particularly Ruth Reichl and Frank Bruni, who rarely gave out zero-star reviews. In one of his "Diner's Journals" Bruni even responded to a reader who wonders if he simply chose to write about zero-star restaurants less often, to which he responded that zero-star reviews should be "somewhat rare." It makes one wonder if Bruni was actually more generous than Sheraton with stars or simply ignored the types of zero-star restaurants that Sheraton chose to review. Sifton is a little stingier; using data from Shepherd's scorecard he has awarded zero-star reviews about 14% of the time.

Of particular interest is Shepherd's analysis of how the recent rarity of one-star reviews has changed their meaning:

Because the zero-star rating is so seldom used, some of the other ratings have lost their nominal meanings. For decades, one star has supposedly meant "good," but Frank Bruni’s one-star reviews seldom sound good. It’s the rare restaurant nowadays that would be pleased to receive just one star. The Times’s zero-star reviews carry an additional label: Satisfactory, Fair, or Poor. (No other paper does this.) Bruni has given “Poor” only twice, he has never given “Fair,” and his “Satisfactory” reviews never sound very “satisfied.”

Which also leads to a discussion of a restaurant's expected rating - a newly-opened, expensive restaurant with a big-name chef in the kitchen may expect to get three stars, in which case two stars would be quite a disappointment, while a a smaller, more unassuming place headed by a young chef may be thrilled to get two stars. In this case, a two-star review may sound quite negative for the former and positive for the latter. This comes into play quite often with the Michelin Guide, where even one star is a great honor, unless a star is "taken away" from a restaurant, in which case the consequences have been tragic.

I'm not going to get into the unending discussion of whether we should listen to critics or not, but it's worth noting just how much background you have to have to really understand a restaurant rating. It's not enough to know that a restaurant received one star or four stars or zero stars and that little box explaining what those stars mean isn't going to tell you very much. You really have to get to know your critic.

So about all those Yelp ratings...