The 7-9 scale - why resistance is futile

[Erik Andersson]

The dead horse woke up again in the DMoM&M-thread. As far as I see ratings the are roughly like this (multiply with 10 for the 1-100 scale):


*/1.0-4.9: Crap
**/5.0-5.9: Flawed
***/6.0-7.4: Ok
****/7.5-8.9: Good
*****/9.0-9.9: Excellent

So if a game gets *** or 6.5 I expect it to be ok, slightly flawed perhaps, but also entertaining for some. It's obvious that half of the 1-10 scale is used only for crap, and people generally don't care about the exact degree of crappiness.

This might not be optimal for a ratings purist, but for most people it wouldn't really matter if all the action was between 9-10, with 9.1-9.49 mening "crap", 9.5-9.59 mening "flawed" and so on. The interesting part is that the rating is easily understood, because as far as I can tell the main purpose of a single numerical rating is to easily and quickly tell very roughly what the reviewer thought about the game. If you want to know more in detail how the reviewer made up that number you could try reading the review or start flaming on various message boards until you get the general idea.

Changing the scale in such a way that a 6.0 on the usual scale becomes a 4.0 is nothing but misleading. To me it is as useful as saying that "flawed" is the new "ok", "ok" is the new "good", and "good" is the new "excellent". It just doesn't help anyone at all.

Saying that you want 5.0 to represent an average game is silly, because people can't relate to an average game. If you have a rating of all citizens there will be some point of reference, but with games it's impossible to say what the average would be of. The average of all games sold, or maybe released? I haven't even played 10% of all fps-games, so I have no idea what the average would be in that case, but my guess is that it would be a pretty crappy game. This would of course mean that we're back in the 7-9 range for most games that people actually know about.

[shang]

The 7-9 scale is actually quite intuitive for me, because here in Finland, the school grades are 4-10 (for some reason), with 7 being average, 8 good, 9 excellent and so on. I don't know about the rest of the world, though.

[MatthewF]

In America, it's 60-100. Anything below a 60 is failing.

[Jazar]

Here's the crux of the matter:

Who wants to pay $50-$60 for an "OK" game?

[Gordon Cameron]

If one is a fan of the genre, one might, though I'd probably downgrade it to "pick up as a budgie."

[Vincent_GC]

The 7-9 scale is actually quite intuitive for me, because here in Finland, the school grades are 4-10 (for some reason), with 7 being average, 8 good, 9 excellent and so on. I don't know about the rest of the world, though.

Same thing in the states more or less.

I just don't like using the number system at all and thus I don't use it on my site. It really doesn't convey anything to the reader yet it's the most visable and saught after piece.

Also (going out on a limb here), I assume that those that are more serious about thier game purchases will read the review anyway to reach thier own conlusion.

[Mike O'Malley]

http://moot.typepad.com/photos/uncategorized/budgie.jpg

[Quaro]

Yes, exactly, it's the way it matches up with the grading system. A 5 will always be a failing grade in most people's minds. If you want to use the full range I think it's a good idea to use the star system -- 2.5 stars means 'ok' to my mind.

[Matt Perkins]

Wait a second here, people are now arguing for the 7-10 system? Really?

Stop it! You're being crazy. Why have 10 numbers if you can't use them all. Why do we have to have everything be 7.x or 8.x? wtf?

And for the love of all, until some sites recently started using the whole scale, people were up in arms about not using the whole scale.

Get a grip people! GRIP!

[stusser]

If one is a fan of the genre, one might
Exactly. Take darkstar one for example. A pretty damn mediocre game, really, but I bought it and enjoyed it because I'm starved for space fighters.

I still like daily radar's miss/hit/direct hit system. Basically it's thumbs down, thumbs up, or thumbs way up, with the latter reserved for the half-life2's and oblivions. It had the distinct advantage of not being arbitrary.

[Igor Muravyev]

Why not just be like CGW and don't post #s !!! Woot go CGW.

[steve]

Why not just be like CGW and don't post #s !!! Woot go CGW.
Instead, they post scores from four different publications, all with different rating systems and no context for the numbers!

Woot?

[Matt Perkins]

Instead, they post scores from four different publications, all with different rating systems and no context for the numbers!

Woot?
I'll admit, I don't read CGW...but they do? Really? Doesn't that completely defeat the stated purpose of not giving a number in the first place?

[Mark Asher]

I still think a letter grade system is the most intuitive, though I'd be happy with a thumbs-up/down approach too. Everyone knows an A is excellent and a C is average, or at least I think they do.

[McBain]

I'm with Mark here.

An A-F letter grade is both intuitive and allows for the use of the entire scale.

I think a "D" would be a lot more reasonable for Dark Messiah (the game that sort of kicked up this debate this time) than a 4. Just because the last time I remember seeing a game get a 4 from a major publication, I think it must have been one of IGN's scathing reviews of a HeadGames' EXTREME game line.

[Troy S Goodfellow]

I'll admit, I don't read CGW...but they do? Really? Doesn't that completely defeat the stated purpose of not giving a number in the first place?

And with G4W apparently reprinting 1up reviews that already have scores attached to them, the scoreless mission statement is going a little awry.

I have confidence that they know what they are doing more than I do.

Troy

[metta]

Instead, they post scores from four different publications, all with different rating systems and no context for the numbers!

Woot?

Much as it galls me, I agree with steve.

CGW (or GFW:TOM) has turned it's review section into a place that reviews other peoples reviews. They go a step further than merely printing other scores, they publish quotes within the body of the GFW:TOM (or CGW) reviews.

I don't want half my review space taken up with "...this is what Blah Blah Kasan at Gamespot trenchantly observed", or "...Woopdee Doo at Game Mag worried that...".

[ashileedo]

With compilation sites like Meta Critic and Gamerankings, not having consistent scoring can skew overall averages from sites like this. Tom's 4 out of 10 score (which would've been a 6 out of 10 with any other site) is going to impact their overall score at Meta critic and gamerankings.

[Drastic]

Averages being skewed is obviously a pretty severe problem. I think the best solution is to simply have every reviewer and review site rate every game an 8; this will also ease the server load on averaging sites by reducing the calculations necessary. I don't see a downside.

[Matt Perkins]

Averages being skewed is obviously a pretty severe problem. I think the best solution is to simply have every reviewer and review site rate every game an 8; this will also ease the server load on averaging sites by reducing the calculations necessary. I don't see a downside.
I lolled. :)

[Fugitive]

Stop messing up my searches for truly awful games with the merely mediocre! How else am I supposed to get my daily dose of schadenfreude?

[steve]

With compilation sites like Meta Critic and Gamerankings, not having consistent scoring can skew overall averages from sites like this. Tom's 4 out of 10 score (which would've been a 6 out of 10 with any other site) is going to impact their overall score at Meta critic and gamerankings.
One site shouldn't have that much of an impact, since it's offset by zillions of others handing out your standard 7-9 ratings.

Metacritic weighs some reviews more heavily than others, but even then, unless all of its "more important" sites have equally low ratings, it won't have that much of an impact in the long run. If this review is first, though...

[Jeff Green]

Instead, they post scores from four different publications, all with different rating systems and no context for the numbers!

Woot?


No need to "woot" us at all, as that died with the last CGW. This Dark Messiah review of Tom's, in fact, is the review he wrote for us for Games for Windows issue #1. This is our review, posted by 1Up. So this "4" score is our score.
Yep, we got shit for taking the scores away, and now we're in shit again for bringing them back and not abiding by the 7-9 scale! Yay!

[stusser]

Yes, I heard that Bill Gates personally ordered GFW to put scores back in, so he could add several points to all of the MGS products.

[Tom McNamara]

Letter grades don't translate internationally. Even Canada has a different letter grade scale than the States.

[Shawn_Elliott]

Instead, they post scores from four different publications, all with different rating systems and no context for the numbers!

Woot?

Every opportunity that arises, eh?

[Hans Lauring]

I still think a letter grade system is the most intuitive, though I'd be happy with a thumbs-up/down approach too. Everyone knows an A is excellent and a C is average, or at least I think they do.

On a scale from A to blue I rank this idea as a 3.

[Rob Beschizza]

I dislike ratings, but if I were to choose a system, I'd go for a simple "Yes or No" system like Ebert's, or Green/Amber/Red lights. The latter, I think, because I don't think it's often done.

[steve]

No need to "woot" us at all, as that died with the last CGW. This Dark Messiah review of Tom's, in fact, is the review he wrote for us for Games for Windows issue #1. This is our review, posted by 1Up. So this "4" score is our score.
Sellout! Bowing to the pressure of the man!

[Damien Neil]

I forget who it was that used it, but my favorite scale ever has been "buy/rent/skip".

[Jeff Green]

Sellout! Bowing to the pressure of the man!

Yeah yeah. You'll give me shit no matter what I do!

[DeepT]

With compilation sites like Meta Critic and Gamerankings, not having consistent scoring can skew overall averages from sites like this. Tom's 4 out of 10 score (which would've been a 6 out of 10 with any other site) is going to impact their overall score at Meta critic and gamerankings.


I think they normalize the scores do you do not have that problem.

[krayzkrok]

Frankly, unless the entire world suddenly agrees to use exactly the same system, all of this is pretty much moot.

[RichVR]

Personally, the only numbers that I really cared about in PCG or CGW (when they had stars) were either 90 and above (they really liked it) or below 50 (especially 20s and teens) because I love to read about games that were being trashed. I also enjoyed the CGW when the stars were all the same for every game. Is that worth something now?

Anything between that was all about what the reviewer wrote. Don't get me wrong, I was always interested in what the reviewer said for any game that wasn't a total dog. But essentially, I looked at numbers first for games that I wouldn't play anyway. Games that I might be interested in were all about the reviews, not the numbers.

The reviews in any publication weren't the end all and be all for me. Sometimes I picked up a game that got a bad review, and enjoyed it. But most of the time the reviewer and I agreed about a game.

Yeah, I didn't like the CGW "we don't do numbers, but here's what a bunch of other guys gave this game" but I still bought it. Hell, I didn't kill my subscription while Vederman was giving out Ann Landers advice either. And that pretty much SUCKED.

Guess I'm just a glutton for punishment.

Rock on, paper publications! You have one idiot that will keep buying your stuff.

[Brendan]

Skip the scores all together. (And don't do a half arsed job like CGW/GFW).

If you can't be bothered to read the review then why buy the magazine?

[Vincent_GC]

Personally, I find that the magazines still have some good stuff to read, just not in the review/preview's section (with very rare exceptions, notably PCG exclusives).

There is other stuff in these mags that are still worthwhile for me to keep a subscription, namely the opinion articles or other stuff (Tom Vs Bruce in CGW for example).

But as for relying them for reviews and most previews.....the internet just has them beat.

[Jason McCullough]

Can I suggest this entire blog (http://www.lifewithalacrity.com/)?




2005-12: Systems for Collective Choice (http://www.lifewithalacrity.com/2005/12/systems_for_col.html)
2005-12: Collective Choice: Rating Systems (http://www.lifewithalacrity.com/2005/12/collective_choi.html)
2006-01: Collective Choice: Competitive Ranking Systems (http://www.lifewithalacrity.com/2006/01/ranking_systems.html)
2006-08: Using 5-Star Rating Systems (http://www.lifewithalacrity.com/2006/08/using_5star_rat.html)


Using 5-Star Rating Systems

In Collective Choice: Rating Systems (http://www.lifewithalacrity.com/2005/12/collective_choi.html) I discuss ratings scales of various sorts, from eBay's 3-point scale to RPGnet's double 5-point scale, and BoardGame Geek's 10-point scale.
http://www.lifewithalacrity.com/images/stars_1.gif (http://lifewithalacrity.blogs.com/.shared/image.html?/photos/uncategorized/stars_1.gif)
Of the various ratings scales, 5-point scales are probably the most common on the Internet. You can find them not just in my own RPGnet, but also on Amazon, Netflix, and iTunes, as well as many other sites and services. Unfortunately 5-point rating scales also face many challenges in their use, and different studies suggest different flaws with this particular methodology.
First, one study (http://sloan.ucr.edu/category/working-papers/product-reviews/) using Amazon data has shown that many undetailed ratings (where the rater isn't required to add any additional information other than the rating they select) show a bimodal distribution (http://en.wikipedia.org/wiki/Bimodal_distribution). In other words the distribution of ratings tends to cluster around two different numbers (e.g., 1 and 5) rather than offering a normal distribution where the ratings cluster around a single height (e.g., 3). Thus the median of these ratings is not an accurate reflection of product quality, but instead is a statement of conflicting opinions.

My new project at work is something related, and it's surprisingly complicated.

[Chris Nahr]

I think the blog entry on the 5-star scale ignores the problem of selective reviewing. Regarding his examples, fans of a show will always rate episodes highly because even the worst episode has the setting and characters that they like. Consumer reviews tend to be 1 or 5 because consumers only bother to post reviews when they're either irate or enthusiastic.

Game reviews tend to skew upwards because video games have a significant technical component that affects rating, and the technical quality is likely to be high among those games that a magazine bothers to review in the first place. Some reviewers (like Tom Chick) actually do give a game a low rating if they just plain don't like it, but most will set 3 stars as a baseline for a technically competent game, even if they think the gameplay is mediocre.

Whether one should do that or not is actually a difficult question. There's a similar problem with reviews of action films. Many genre fans (myself included) will think that basically any action film is at least okay if the stunts and special effects are good, but newspaper reviewers will often completely ignore these aspects and rate all movies exclusively based on story, characterisation etc. Result: most action films get trashed indiscriminately. Is that a service to the readers?

[chet]

Jeff Green, I give your post a score of 750.
Steve "the suit", I give your snarky post a score of 50.

Do you guys want to guess top end of your respective scales, each has a different set of numbers...

Jeff, can you understand the confusion of just throwing those numbers out there with no context? I welcome the cutting of content from other sources, I welcome the return of scores, but the Tom's 4 hangs out there like the numbers above, nonsensical in the current scoring system.

Chet

[Tom Chick]

This isn't entirely new. I've actually been doing this for CGW/GFW ever since they've asked us to go back to numerical ratings. Sword of the Stars and Dark Messiah were both a 4. Joint Task Force and Stronghold Legends were a 3. The Age III expansion was a 9.

I presume their other writers have as well. I know Shawn Elliott's 6 for Battlefield 2142 is an example of that.

But I understand the confusion, so hopefully they can clarify -- if they haven't already -- that their 1-10 scale is different from the typical 7-9. Because it probably would have saved me a fair amount of grief if I'd given Dark Messiah and Sword of the Stars the 6.blahblahblah I would have given them on a 7-9 scale.

Aww, who am I kidding? Once you go below a seven on any scale, you're basically kicking puppies to anyone who digs that game. :)

-Tom

[Erik Andersson]

No need to "woot" us at all, as that died with the last CGW. This Dark Messiah review of Tom's, in fact, is the review he wrote for us for Games for Windows issue #1. This is our review, posted by 1Up. So this "4" score is our score.
Yep, we got shit for taking the scores away, and now we're in shit again for bringing them back and not abiding by the 7-9 scale! Yay!

Are you honestly surprised? I guess I didn't make my point very clear in the initial post but I'll try again: I can understand if you don't want to use grades at all, but I can not understand using a non-standard scale. As a matter of fact I can't see why anyone would want a non-standard scale. Sure, there is no offical standard, but since the 1-10 scale has been used for quite some time in game reviews readers will have certain expectations. The only reason I can see for using a single numerical rating at all is for the reader to quickly and easily see roughly what the reviewer thought. If the reader thinks: "ok, a 4 is a 6 everywhere else meaning that a 7 is...?" then I think you've made a mistake.

When I read Tom's review I thought the grade would be in the "flawed but with some redeeming quality, enjoyable for some" area, which usually means a 6/60. I was very surprised to see a 40, because that is normally the "crap, plain and simple" category. The main reason I was confused is of course because I just followed a direct link without first reading the review score manual, but I don't see why anyone should have to.

[steve]

Jeff Green, I give your post a score of 750.
Steve "the suit", I give your snarky post a score of 50.
You obviously didn't read my post long enough. Unless that's out of 50, in which case, you speak the absolute truth.

It's curious that the game press seems to be the only entertainment press using these weird and differing review systems. Almost all movie reviewers use 5-stars or letter grades (which pretty much perfectly map to each other). I can't remember ever seeing a movie or music review using a percentage system. (I'm sure someone will find one, however.) Why did anyone feel the need to invent a different system in the first place?

I still contend that there's a tendency for people to think you can quantify game reviews because the games themselves are so much about math. (And all the people playing and reviewing the games probably have strong computer/math backgrounds.) So, with a bigger number you have more variations, which means more elements were considered to reach that final number. Or something.

Like there's really a difference between a game that gets 92% or 9.2 and one that gets 88% or 8.8. The actual gap between those ratings is practically a rounding error, but most people would probably say there's a meaningful, quantifiable difference between the two scores.

[steve]

Skip the scores all together. (And don't do a half arsed job like CGW/GFW). If you can't be bothered to read the review then why buy the magazine?
While I have no intention of dropping ratings since people actually like them, I think one side effect of dropping ratings is that almost every review would read like a list of why a game sucks, even the great ones.

One thing giving a rating forces the writer to do is better justify their overall opinion, as in the text has to match the rating. The only time I send reviews back to writers is when I don't feel they match. At that point, they have two options: change the text, or change the rating. If they think it's a 4.5 star game and the text is too negative, either downplay the negatives, bump up the positives, or drop it to 4-stars or 3.5-stars.

If you pay close attention, most reviews fixate on the negatives of a game, I suspect as a way to avoid criticism. It's also a lot harder to write incredibly positive reviews. We're so scared of being labeled fanboys that we only show true passion for something when we hate it. (Also, you can be a lot funnier when trashing something.)

So, even in a 5-star-equivalent, non-rated review, I'm guessing the text would be 30% positive, 70% negative. Just the positive parts would be "wow, this is great," and the negative would be downplayed.

But the reader would still walk away with, "Holy crap, that's not a great game."

[Jancelot]

I like the 7-9 scale because it's intuitive and ingrained in our brains since youth (and apparently more places than just the US). Switching to a scale where 5 is the "average" score is really confusing. A-F, 5 stars or 7-9 are standard, understand and straightforward.

[Hans Lauring]

I'm with Steve on this.
When I got to my position I inherited a 1-10 grade that actually try to use the entire range. Our sister mag about digital photography has just introduced their 'new and improved test' which is a 1-10 system weighing heavily on the 7-9 part of the scale. They use decimals, so it's really a percentage scale and right now I'm editing a camera test from their magazine to my format and even with the expanded, technical and long explanation I really don't se how any expert can distinguish between 10 cameras ranging in picture quality from 60% to 82%. That is I see the difference between the 6 and the 8, but not the many cameras ranging between 7,9 and 8,2.

And games are even less quantifiable than cameras imho.
But geeks like their numbers and the worst offenders are those sites that put everything into subcategories to give everything an air of science.

And then there's the whole aggreate score calculation...

[Jeff Green]

Jeff Green, I give your post a score of 750.
Steve "the suit", I give your snarky post a score of 50.

Do you guys want to guess top end of your respective scales, each has a different set of numbers...

Jeff, can you understand the confusion of just throwing those numbers out there with no context? I welcome the cutting of content from other sources, I welcome the return of scores, but the Tom's 4 hangs out there like the numbers above, nonsensical in the current scoring system.

Chet

Oh god yes, absolutely I see the confusion. We spent our entire podcast this week talking about it. It would appear to be a completely Quixotic venture, in fact--especially without the context, yes. The context *is* there in the print mag, and I know it will be there soon (if not already) online. But with or without an "explanation", the idea that we can actually fight the accepted wisdom that anything below a 7 is "D" or "F" is going to require endless explanation. I don't know if that makes us "wrong". But good god is this tiring. My ideal solution? Letter grades.

[steve]

I don't know if that makes us "wrong". But good god is this tiring. My ideal solution? Letter grades.
What exactly was wrong with the 5-star system? Why would letter grades somehow be better, especially since the stars match up perfectly? (The only other rating system I'd consider would be letter grades, not because of matching up but because they have an obvious real-world corollary that requires no explanation.)

I suppose wanting to normalize ratings with other Ziff publications makes sense, but it seems like those other guys should have to justify their own ratings systems instead of changing the one that most reviews in all other forms of entertainment uses.

[steve]

I like the 7-9 scale because it's intuitive and ingrained in our brains since youth (and apparently more places than just the US). Switching to a scale where 5 is the "average" score is really confusing. A-F, 5 stars or 7-9 are standard, understand and straightforward.
I wish someone would actually break out their percentage reviews to admit that it's all about 7-9. So, you'd have:

90-100%-Awesome
80-89%-Great
70-79%-Okay
60-69%-Not so good
0-59%-Varying degrees of awful

[Matt Perkins]

I like the grades better because you can do B+/-, for instance, and you know it's still a B rating either way. A similar star rating would have to be 4.5 or... I don't know, 4.1?

Other than that, both are good for me.

[Jeff Green]

I wish someone would actually break out their percentage reviews to admit that it's all about 7-9. So, you'd have:

90-100%-Awesome
80-89%-Great
70-79%-Okay
60-69%-Not so good
0-59%-Varying degrees of awful

Amen, brother.

And to answer your other question--you answered it. It was about getting in line with the other Ziff publications. So, the scoring system we're using is the same one shared by EGM, Official Playstation Magazine, and 1UP.com.

[Pentadact]

hanging the scale in such a way that a 6.0 on the usual scale becomes a 4.0 is nothing but misleading. To me it is as useful as saying that "flawed" is the new "ok", "ok" is the new "good", and "good" is the new "excellent". It just doesn't help anyone at all.

Must say first that I whole-heartedly agree with the first post - the point of a score is to communicate something to the reader, and redefining the vocabulary you use to do that (even if it's explained elsewhere) is a failure to do that. And...


If you can't be bothered to read the review then why buy the magazine?

...the whole point of a magazine is that you don't have to read it all to get some value out of it. Even more so with a website. 90% of the audience are there to skim, and dip into something that catches their interest. With no scores, nothing does - they have to base that decision on information gleaned elsewhere or the superficialities of screenshots.


Averages being skewed is obviously a pretty severe problem. I think the best solution is to simply have every reviewer and review site rate every game an 8; this will also ease the server load on averaging sites by reducing the calculations necessary. I don't see a downside.

I haven't been here long enough to be familiar with Drastic's 1,400 other posts, but I really really like the idea that he might just jump into a thread whenever wild hyperbole is called for to make a point.


I really don't se how any expert can distinguish between 10 cameras ranging in picture quality from 60% to 82%. That is I see the difference between the 6 and the 8, but not the many cameras ranging between 7,9 and 8,2.

And games are even less quantifiable than cameras imho.

I'm with you on not knowing what the difference between a 7.9 picture quality and an 8.2 is, but using that scale on things as huge, complex and artistic as games is completely the opposite. There's a gaping chasm between those two, particularly since so many games fall in that upper quartile of the scale.

Just quickly, I'd like to bitch for a sentence or two at all scoring systems involving fractions. If you need finer distinctions, why the hell did you pick a scoring system with only three/five/ten options? There's a site that marks out of twenty, but allows quarter-points. WHAT.

ANYWAY: Most games reviewed in our mag land in the upper half of the score bracket, but we do use the full scale. We have 40% games, 30% games, 20% games, 10% games, all the way down to 1%, which we've only awarded once. In fact we've even used zero, in that we gave a game N/A on the grounds that it quite literally wasn't playable. A 50% in our mag says don't buy it, because who wants to spend £30 and hours of their life on something mediocre when there are so many 70-90 games available? But it doesn't mean it's worthless, and it doesn't mean there isn't an interesting difference between that and the 1% game. The degrees and flavours of awfulness unrecommended games come in are some of the richest sources of amusement in games writing, and we need that half of the scale to rip them apart all over it.

For me personally, there's also a modest principle at stake. Brave games that are ruined by a few flaws like The Ship (61%) shouldn't be lumped in with utterly worthless cash-ins like the Big Brother game (1%), or offensively egregious date-rape sims like Leisure Suit Larry: Magna Cum Laude (3%). And if you stretch the 7-9 scale to fill all the numbers, that's what happens. The many talented people who did great work on it have earnt the right not to be kicked, and I'm glad our scoring system let me reflect that while I didn't recommend buying it, these guys are doing interesting things that could become great things down the line.

[Jazar]

Just heard the GFW podcast and I must say that you guys are fighting a losing battle. No one cares about average or even above average. People want a good game ("B" grade) or a great game ("A" grade). Anything less is somewhat of a failure. I do agree with you Jeff that letter grades are the best way to go IMO.

[Alan Au]

If you use a letter grade system, do you include an "S" rating above "A"?

- Alan

[Damien Neil]

But I understand the confusion, so hopefully they can clarify -- if they haven't already -- that their 1-10 scale is different from the typical 7-9. Because it probably would have saved me a fair amount of grief if I'd given Dark Messiah and Sword of the Stars the 6.blahblahblah I would have given them on a 7-9 scale.

Aww, who am I kidding? Once you go below a seven on any scale, you're basically kicking puppies to anyone who digs that game. :)

Well, yeah. On a 7-9 scale, a 6 is off the bottom--complete crap, don't even think of going there. Of course someone who thinks the game isn't utter garbage will disagree with a less-than-seven rating.

[Tom McNamara]

*cough*


Letter grades don't translate internationally. Even Canada has a different letter grade scale than the States.

[Jazar]

Ok here are the scores: Worthless, Poor, Ok, Good, Great.

[AndrewPf]

1-2: Worthless
2-4: Poor
5-6: OK
7-8: Good
9-10: Great

[Matt Perkins]

*cough*
Yeah, good point. Well, screw those guys! :P

I don't know, I don't much like the stars system, but I guess that's as good as it gets. Better than the 7-9 system by a long shot.

[jfletch]

I forget who it was that used it, but my favorite scale ever has been "buy/rent/skip".

Daily Radar - Hit, Direct Hit, Miss, and poop (whatever it was called).

A variation on the whole number 1-5 scale. Since virtually every site or mag I have read that uses that scale uses the whole scale, I prefer that one. No mag or site will ever use the full 1-10 scale, sure they may make some fancy pronouncements about doing it but eventually their learned behavior from school will kick in and it is back to 7-9. It's 7-9 for a reason, after all.

[Rorschach]

I always translated the Daily Radar ratings as:

Direct Hit - Everyone should buy this game
Hit - Buy the game if you have an interest in the genre or subject matter
Miss - Rent or buy if you're very interested in the subject matter but beware
Poop - Avoid at all costs

[Matt Perkins]

Yeah, I like those ratings too.

[Vincent_GC]

Those ratings aren't too bad. Better than numbers at least.

But still, is it completely necessary to have a score attatched to a review? I always thought a shot, consice summary (mabye with a few bullets for pro/con) would be sufficiant, quick, and a hell of a lot more accurate.

[DennyA]

No need to "woot" us at all, as that died with the last CGW. This Dark Messiah review of Tom's, in fact, is the review he wrote for us for Games for Windows issue #1. This is our review, posted by 1Up. So this "4" score is our score.
Yep, we got shit for taking the scores away, and now we're in shit again for bringing them back and not abiding by the 7-9 scale! Yay!
Wait, you're going to the crappy new 1-Up scale in GfW?

Jeff, you guys had the most logical, understandable scale of all before you dropped the scores. Now you're going to the one that only makes sense to people in an alternate universe?

This is like when they killed Coke, went to New Coke because they thought everyone would like it, and then when they didn't they brought out the crappy high-fructose corn syrup Coke Classic and pretended it was as good as the original.

[Brad Wardell]

No need to "woot" us at all, as that died with the last CGW. This Dark Messiah review of Tom's, in fact, is the review he wrote for us for Games for Windows issue #1. This is our review, posted by 1Up. So this "4" score is our score.
Yep, we got shit for taking the scores away, and now we're in shit again for bringing them back and not abiding by the 7-9 scale! Yay!

I hear Microsoft is keeping the jar with your soul in it on the next shelf over from where they keep my soul. Sometimes..late at night, I can almost see them... ;)

[MatthewF]

The best function of a score that I can think of is that it gives you a general idea of how good the game is without having to read through the review. Instead of an average score, though, I'd rather just see it broken down into categories, with descriptions of the high and low points of each. I know some magazines do this, but they still insist on providing an average score. Sometimes, though, the parts are greater than their sum.

[DennyA]

At this point, Ziff might as well just rate things by color, given how much sense the ratings system makes when compared with the reality of what every single other publication that rates anything does.

NWN2? Oh, that game got a puce.
Half-Life 2: Episode 3 definitely looks like it's getting a medium gray!
I can't believe Company of Heroes got a chartreuse!

[Jeff Green]

Wait, you're going to the crappy new 1-Up scale in GfW?

Jeff, you guys had the most logical, understandable scale of all before you dropped the scores. Now you're going to the one that only makes sense to people in an alternate universe?

This is like when they killed Coke, went to New Coke because they thought everyone would like it, and then when they didn't they brought out the crappy high-fructose corn syrup Coke Classic and pretended it was as good as the original.

This is the Ziff Davis scoring system. So don't crucify me plz. It's shared by every single Ziff publication, which is why we are doing it. It was a Ziff mandate--not a Microsoft one. And, hey, EGM has been using this exact same scale for years and no one has ever given a shit. So I don't know why this is suddenly the end of the world....

[flyinj]

I think it's great if they change it to a pure 1-10 system. That is, if everyone reviewing games does. Having one site change to it, while every other site stays with 7-9... well, it just doesn't work.

[DennyA]

Well, it's the most boneheaded thing Ziff has done since they thought offering me Softbank stock would keep me from leaving the company. :)

Sorry, Jeff, should have realized it was a mandate from above. Hopefully the uproar will be loud enough that you can convince the powers that be to go back to the ratings system that worked just peachy for over a decade.

Oh my God. I've become one of those guys posting on a message board about how the Ziff-Davis overlords are screwing up CGW. What's happened to me?

[Chris Nahr]

That would never have happened back when Johnny Wilson was in charge!

[Rob_Merritt]

This is the Ziff Davis scoring system. So don't crucify me plz. It's shared by every single Ziff publication, which is why we are doing it. It was a Ziff mandate--not a Microsoft one. And, hey, EGM has been using this exact same scale for years and no one has ever given a shit. So I don't know why this is suddenly the end of the world....

Because us pc gamers (Windows Gamers?) are drama queens. :D

[Doug Erickson]

I think having one reviewer produce the same review across multiple publications is far worse (JParish and UGnG; MPeckham and NWN2) than the disingenuous 1-10 scoring system.

Really, the reason no-one complains about EGM's scoring system (although folks have complained quite regularly at GAF) is because most EGM scores are giant jokes among their audience. They're the Famitsu of US console gaming reviews. We've all accepted that the marquee big-budget Japanese favorites -- especially those with potential nostalgia to milk (see: Final Fantasy, Mario, Zelda, Metal Gear et al) -- will get a bit of a pass, and the B-list stuff the fans like will get run through the wringer. Is EGM even considered relevant any more as a critical authority?

[Chris Nahr]

I find EGM's reviews quite informative and usually agree with them when I know the game in question. Then again, I also think GAF is a bunch of obnoxious kids so I guess I don't count as the unnamed passive subject that would consider EGM "relevant as a critical authority"...