3am: The perfect score

[TomChick]

3am: The perfect score

We all know there's no such thing as a perfect game, at least not in the literal sense of the word. Neither should there be any such thing as a perfect score.

[BDGE]

I just got a copy this afternoon and it is a very impressive effort. Surprisingly this "platformer" has more inventive level maps than either of the two traditional GBA efforts starring the pink puffball, there is clearly a uniform design ethic grounding this work. I wasn't much a fan of that Magic Mirror game and it's slapdash approach to threading obtuse hub rooms into some gimongous ball of yarn overworld to explore. And then there was that other 'experimental' Kirby game Air Ride...

Hal Labs has really wisened up.

[Gendal]

As I said in another thread here, it's the first real DS title. Hopefully a good harbinger of things to come.

[Chris Nahr]

From the description it sounds a lot like Yoshi Touch & Go, except with Kirby instead of Yoshi and with more levels... is that about right? That would be great because I love Yoshi Touch & Go.

[stusser]

Not to derail, but I'm curious... how do you take screenshots of a DS game? Some kind of devkit?

[CannonFodder-jm]

DAMN, Tom, you need to pace yourself!

You don't just one day decide to go to the gym and start benching 600 lbs.

I love updates, but you gotta plan this kind of output with your doctor or something.

[Natus]

I've happily handed them out to God of War, Guild Wars, and now Kirby.

Tom,

is there any way you can link to these other reviews? I'd love to read them if they are online.

[TomChick]

Natus, thanks for the sentiment, but the God of War and Guild Wars reviews are in Computer Games Magazine. Subscriptions are cheap, though and you get content you won't find anywhere on the web! :)

Stusser, a lot of console reviews just use screenshots provided by the publisher.

-Tom

[stusser]

Really? Another illusion shattered.

[Jason McMaster]

yeah, all of my reviews minus pc games are all publisher screens

[Kalle]

No TV/Video card? Tsk, lazy reviewers. :)

[Wholly Schmidt]

Originally Posted by Kalle

No TV/Video card? Tsk, lazy reviewers. :)

That wouldn't help him with the DS, which was the original question.

[Old Man Gravy]

While I'm surprised that a game whose protagonist is described by the reviewer as lazy and uninteresting still merits a perfect score, I'm nevertheless happy to see reviews that don't penalize games with terrific, innovative gameplay ideas just because of crappy, 2d graphics.

Also, you can get spam-subsidized subscriptions to CGM for only $6 a year if you're willing to put up with a little extraneous email from the subscription sponsor. CGM's chalk full of contributions from a lot of this board's most talented, so it reads sort of like a monthly Qt3 digest. Without having to filter Jose Liz and shit bonerz posts.

Finally: since sigining up, I have noticed neither a significant change in volume nor a reduction in quality in the "Se.x.ually X.plicit" and "E.nlarge Ur Pen||s!" emails from my friends like Ubiquitous Q. Lucidly and Timescales A. Rockiness, so the subsidy spam hasn't created any additional administrative distress. I highly encourage everyone to check it out.

[Miramon]

Not that I'm a reviewer.... But I'd like to reserve the highest ratings for really amazing things. IMO, a typical best movie in a typical year shouldn't get the highest rating possible for movies, same for books, games, restaurants, or anything else which is rated -- this is because the typical best of the year in these categories just pales in comparison to the best of all time or the best of the last 10 years, or whatever. And if you are going to keep all things of a type on a reasonable comparative scale, it's sad to rate the best of all time and the best of the year at the same level, when the former is really 10 times as a good as the latter.

If, I don't know, Lost In Translation (a deeply flawed movie that was still one of the best of its year) gets 4 stars, then what do you give Casablanca (widely considered one of the best of all time)?

Did you, Tom, think that God of War was so good that nothing in your memory was really all that much better? If so, then I wouldn't complain about the 10/10 rating, but if you can reel off a number of other games from previous years which were superior, then I'd have to say that 10/10 was not really appropriate for the game.

I understand that everyone's scale is different, so if you want to define your scale as merely going up to "one of the best of the year" as the best possible rating, that's OK if it is made clear that's the meaning and the limit of the scale, but I'd rather see a scale that puts that an 8 or a 9 out of 10, to give room for the truly superlative achievements that don't come along so frequently. For that reason, I think it would be a good idea to provide a key to scale ratings along with the rating itself for every review, so you can be reminded that 8="you'll love this game", 9="best of the year" and so on, so that you don't just sneer at some relatively lower rated but still-excellent game....

[Kalle]

Fuck Review Scores

[Hans Lauring]

Originally Posted by Miramon

Not that I'm a reviewer.... But I'd like to reserve the highest ratings for really amazing things. IMO, a typical best movie in a typical year shouldn't get the highest rating possible for movies, same for books, games, restaurants, or anything else which is rated -- this is because the typical best of the year in these categories just pales in comparison to the best of all time or the best of the last 10 years, or whatever. And if you are going to keep all things of a type on a reasonable comparative scale, it's sad to rate the best of all time and the best of the year at the same level, when the former is really 10 times as a good as the latter.

If, I don't know, Lost In Translation (a deeply flawed movie that was still one of the best of its year) gets 4 stars, then what do you give Casablanca (widely considered one of the best of all time)?

Did you, Tom, think that God of War was so good that nothing in your memory was really all that much better? If so, then I wouldn't complain about the 10/10 rating, but if you can reel off a number of other games from previous years which were superior, then I'd have to say that 10/10 was not really appropriate for the game.

I understand that everyone's scale is different, so if you want to define your scale as merely going up to "one of the best of the year" as the best possible rating, that's OK if it is made clear that's the meaning and the limit of the scale, but I'd rather see a scale that puts that an 8 or a 9 out of 10, to give room for the truly superlative achievements that don't come along so frequently. For that reason, I think it would be a good idea to provide a key to scale ratings along with the rating itself for every review, so you can be reminded that 8="you'll love this game", 9="best of the year" and so on, so that you don't just sneer at some relatively lower rated but still-excellent game....

This one goes to eleven.

[Jason McMaster]

This all comes down to the whole scale system and blah blah blah. If he thinks it deserves a 10 then it deserves a 10, it's his level of suggestion for the product.

[Wholly Schmidt]

Miramon, if you spend any length of time agonizing over whether game A with a score of 10 is exactly as good as game B with a score of 10, your reviews will be forever crippled by your desire to be fair. You just can't anticipate what's going to happen in the future and you can't give scores based on an exact comparative ranking to games in the past either unless you rank them on a constantly changing scale of 1-X, X being the number of games released so far. At some point, you've got to just suck it up and say "This is a damn fine game. It gets a 10."

[TomChick]

if you spend any length of time agonizing over whether game A with a score of 10 is exactly as good as game B with a score of 10, your reviews will be forever crippled by your desire to be fair.

Which is one reason that I think the 7-9 ratings systems are so silly, and the percentile Gamespot/PC Gamer style ratings systems are particularly silly. That level of granularity just leads people to assume you've got a fine-tuned x is better than y system.

It's even more ridiculous when all your ratings are piled at one end of the scale and you've got no room to move. As far as I'm concerned -- as a reader and a writer -- the coarser the scale the better. Ah, the salad days of Daily Radar's hits and misses!

-Tom

[Miramon]

I don't want to anticipate the future so much as I want to accommodate the past.

Say I think Planescape: Torment from 5 years ago (or whatever it's been now), is much better than Jade Empire. Then, despite thinking Jade Empire is the best RPG of the year, I don't want to give Jade Empire a 10, because then my scale doesn't say what I want it to say. If I had reviewed games for a number of years, I would want to be able to look back and see that my rating of Planescape was higher than my rating of Jade Empire, because I think the first game is much better than the second.

If there is any point to numeric ratings at all (and Kalle has a point) then I think numeric ratings should distinguish games that have very different quality and enjoyment levels. I hope it's not somehow snobbish or elitist to say that, instead of just saying that all games that are "good enough" get the top rating despite some being much better than others.

[Troy S Goodfellow]

Originally Posted by Miramon

Say I think Planescape: Torment from 5 years ago (or whatever it's been now), is much better than Jade Empire. Then, despite thinking Jade Empire is the best RPG of the year, I don't want to give Jade Empire a 10, because then my scale doesn't say what I want it to say. If I had reviewed games for a number of years, I would want to be able to look back and see that my rating of Planescape was higher than my rating of Jade Empire, because I think the first game is much better than the second.

But doesn't this lead to infinite regression? You couldn't give Planescape a ten, because what if a much better game came out the next year? So you hedge and give it a 9. Or maybe even an 8.5, since you have to leave room for Bioware's next gem.

You've just lost yourself 1 or 2 points on your scale. Because if you have to make sure that all your ratings line up like ducks in a row - you're just asking for trouble.

Troy

[Miramon]

Originally Posted by TSG

You've just lost yourself 1 or 2 points on your scale. Because if you have to make sure that all your ratings line up like ducks in a row - you're just asking for trouble.

Troy

This is a good point. I'm not sure what to say except I don't feel any need to accommodate future games, I just want to make sure that my theoretical system handles the older ones that I think may be better than the current ones. Maybe Tom is right and there should be a courser scale with just *s or just a yes/no, and I can reserve a special super-ultimate-huzzah rating for the games which really stand out and not assign that except when it is really warranted. Fortunately I'm not a reviewer so I don't really have to address this issue in practice.... But I still want to know from a reviewer that a game is really great as opposed to "it's good enough for my top rating, but so are a half-dozen other games this year".

[Troy S Goodfellow]

Well, if you don't make room for future games, your list will never line up right. "Civ 2 is the best game ever! 10/10!" "Damn. I like SMAC a lot more. But I've used my 10."

I get your point though, and I agree that "perfect scores" should be measured out with coffee spoons.

Still I gave Darwinia 5 stars at DIY because I think it's the most interesting strategy game I played this year. I caught some flak for that from readers, but I think it's a great way to draw attention to a title a lot of people might miss. Well, maybe not DIY readers, but I was genuinely excited by the game.

I also think that Rome and Pirates fit my definition of 5 star games.

Now if you cornered me and asked me whether they would all get 10s, maybe not. Certainly not 100s.

Troy

[Wholly Schmidt]

Originally Posted by Miramon

This is a good point. I'm not sure what to say except I don't feel any need to accommodate future games, I just want to make sure that my theoretical system handles the older ones that I think may be better than the current ones. Maybe Tom is right and there should be a courser scale with just *s or just a yes/no, and I can reserve a special super-ultimate-huzzah rating for the games which really stand out and not assign that except when it is really warranted. Fortunately I'm not a reviewer so I don't really have to address this issue in practice.... But I still want to know from a reviewer that a game is really great as opposed to "it's good enough for my top rating, but so are a half-dozen other games this year".

You keep talking about best games of the year as well, and I don't think that was even Tom's point. He's not giving Kirby the 10 because it's the best game he's played this year, that's still putting a random and arbitrary restriction on your score. He's giving it a 10 because it's a really awesome game. See how the sentence ends right there?

If he's playing through a super awesome game and maybe thinks "you know, this could've been better here, but overall it's pretty dang cool" maybe it gets a 9. And maybe the part he found lacking was only because of his experience with other games, knowing it could've been done better. That's about as far as direct comparisons should go as they relate to the numerical score. Other more specific comparisons may be apt when dealing with a couple games trying to do almost the same thing, like your Forzas and Gran Tourismos for example, but that's the kind of thing you read the rest of the review to find out.

[Wholly Schmidt]

Originally Posted by TSG

Well, if you don't make room for future games, your list will never line up right. "Civ 2 is the best game ever! 10/10!" "Damn. I like SMAC a lot more. But I've used my 10."

I get your point though, and I agree that "perfect scores" should be measured out with coffee spoons.

Troy, did you read the first post? The bit it's linking to? The whole point is that it's not a perfect score. There shouldn't be a score that means "this is flawless." Certainly it'll take a lot for some people to wrap their heads around it based on the 7-9 scale they're all used to, but with Tom's explanation I think it's clear. 10/10 doesn't mean it's the best game ever. 10/10 means it's a fantastic game. I understand if you want to say that's impractical as far as getting people to understand it when they're used to the Gamespots and IGNs and whatnot, but from the sound of things, you missed the point too.

[Troy S Goodfellow]

Yeah, I read it. Which is why I put "perfect scores" in quotation marks. It's not about perfection. "Perfect scores" is good shorthand though. Maybe we should use "top marks" instead.

Though it would be great if there were 70 10/10 games every year, the score loses all purpose if it is used too much. I am not arguing that Tom is doing that. In fact, I would take a single one of his 8 or four star games over anything that IGN gave a 9.6. And a lot depends on how many games the reviewer plays. I stick pretty much to a couple of genres, so if I hand out 20 top marks in a year, people will look at me funny. Someone with more versatility could easily find that many by finding two awesome games in every major genre.

The thing is, most games are average - but still better than most stuff made ten years ago. Which is why I like your characterization of a game review as "How could this have been done better?". So long as the answers are realistic and not dependent on dinosaurs with lasers, the length of your list is a pretty good barometer of where the game should be on a rating scale.

Troy

[GenericID]

Originally Posted by Miramon

This is a good point. I'm not sure what to say except I don't feel any need to accommodate future games, I just want to make sure that my theoretical system handles the older ones that I think may be better than the current ones. Maybe Tom is right and there should be a courser scale with just *s or just a yes/no, and I can reserve a special super-ultimate-huzzah rating for the games which really stand out and not assign that except when it is really warranted. Fortunately I'm not a reviewer so I don't really have to address this issue in practice.... But I still want to know from a reviewer that a game is really great as opposed to "it's good enough for my top rating, but so are a half-dozen other games this year".

There is really simply no way to equate any 10/10 score given right now compared to one dished out a couple of years ago. In that sense, yes, it's a sliding scale in which the reviewer's preferences change with age and experience. A 10 from a guy on one site is definitely not the same as a 10 from another reviewer on the same site. How does a strat game 10 compared to a FPS 10? On that note, retroactively going back to change all your previous scores because you now think X is better than Y a few years ago is an exercise in complete futility. And isn't any review a contextual affair where there are always comparisons with current and previous products in the same field? It's hard to write a review in an absolute vacuum as defined only by the score number. With that in mind, my only expectation for the reviewer is to grade it quantitively by the scoring methodology listed on the corporate website. Even if it's a non-linear scale with component weightages I find ridiculous. :)

Of course, it does help that 10/10 (or 5 stars and whatnot) be handed out less often. A 9.5/95%/4.5 stars would probably work the same impression for most games.

[wildpokerman]

It's good to see someone is giving games high scores lately, look at what gamespot is doing:


http://www.gamespot.com/reviews.html

Now sort by score and you'll see every game they have rated since the beginning of the year (two exceptions as of now) has been scored lower by Gamespot than by the readers. Now change the timeframe to 2 years and sort by GS score again, it's an even distribution of higher and lower scores and most are within 2/10ths of a point.

It's like from December on they've become jaded game haters, either that or gamers have relaxed their standards since then.

[GenericID]

Originally Posted by yurislave

It's good to see someone is giving games high scores lately, look at what gamespot is doing:


http://www.gamespot.com/reviews.html

Now sort by score and you'll see every game they have rated since the beginning of the year (two exceptions as of now) has been scored lower by Gamespot than by the readers. Now change the timeframe to 2 years and sort by GS score again, it's an even distribution of higher and lower scores and most are within 2/10ths of a point.

It's like from December on they've become jaded game haters, either that or gamers have relaxed their standards since then.

Is the stable of reviewers the same since last year? If so, it might be a conscious editorial decision to score games more critically.....perhaps because of threads like this. :)

[Wholly Schmidt]

Hasn't Gamespot been giving out reviews for a lot longer than they've been keeping track of user rankings as well? Is it as recent as sometime in that two year frame that they started user rankings? If so, that could explain some of the differences.