scales, continued

[Shmtur]

I didn't want to add more 1-10 = 7-9 scale discussion to the *ahem* other thread that had it, but I do recall someone asking why a middling/poor game would get a 5-6 over a 3-5. So I decided to come up with my own interpretation of the 1-10 scale.

1-3 = game has severe technical issues, and when it does work is, well, awful
3-6 = game generally works, but is still quite awful/pretty bad (depending on where it falls in the 3-6 scale)
7-9 = game is decent to good, this can be dependant on a few different things.
9-10 = great, mindblowing (and the free tacos the devs bought me were excellent).

This isn't to say this is how a 1-10 scale should be used; it's just a general interpretation of it. My interpretation, I suppose. Anyone care to revise it more?

[Alan Au]

Anyone care to revise it more?
No, not really.

- Alan

[Equisilus]

I'd say a game that was "quite awful/pretty bad" should never be rated above a 4, for sure, which is on the lower side of neutral ground. It's that middle range of ratings that people don't seem to get quite right. I'd put a range of 1-4 (even 0) for games on the low end of your scale (the 0-1 range for games that are entirely unplayable, 1-4 for playable but awful), with ranges from 4-6 for neutral (not bad but not good), 6-9 for good to great, and 9-10 for amazing games that will become classics or must-haves for their genres.

That's generally how I rate games (if I was to use a number system at all). I understand that folks generally don't care for ratings much, but I like using them for my own personal lists to get an idea of where games I've played measure up to each other. Note: the lowest rated game I've played was "Crystal Key 2: The Far Realm" (4.5) and the highest was "The Longest Journey" (9.3). The lack of low rated games is due to the fact that I usually do some research beforehand and avoid games that would fall in that range (CK2 came in a pack with "The Omega Stone", which is why I played it).

[Edit] To Alan: I'm anyone!

[Moore]

I'd prefer 'it's ok, buy it' and 'complete shit' to be the only permissable ratings, myself.

[SwampIrish]

I'd prefer 'it's ok, buy it' and 'complete shit' to be the only permissable ratings, myself.

How about this:

Complete Shit
Steal it
Borrow it
Rent it
Buy it

[Dave Long]

I've always felt there should be a "Break the rental copy you took out so no one else has to be subjected to that shit!" rating.

[EvilIdler]

Complete Shit
Steal it
Borrow it
Rent it
Buy it
I'd use that scale, and add a top rating: "Worship it and write fanfic" :)

[Troy S Goodfellow]

I'd use that scale, and add a top rating: "Worship it and write fanfic" :)

One should never encourage fan fiction - even in jest.

Troy

[steve]

How about this:

Complete Shit
Steal it
Borrow it
Rent it
Buy it
I love it! Now if only we could represent it graphically, subbing in something like stars...

[Moore]

You could just have a happy grinning face, a face with a little smile, a neutral face, a shifty eyed face, and a really sad face covered with excrement.

[Rob Beschizza]

Let's return to when ACE magazine had scores out of 1,000 with graphs to show you how interested you would be in the game over the course of a year.

[fuzzyslug]

I think I'm going with the rating system proposed by Matt in that other thread (http://www.quartertothree.com/game-talk/showpost.php?p=641653&postcount=94).

8, 9, or 0. Seems pretty clear cut to me.

[jfletch]

Eurogamer has a very solid, well-explained methodology page (that I cant find right now), but most importantly, they actually use it in their reviews. I couldn't tell you how many times I have read an IGN review and found the score doesn't match the text, or the score matches the text but doesn't match what their methodology page says, or the scoring on the PS2 page feels totally different from the Xbox one, etc.

[Charles]

RIP Daily Radar.

[Justin Fletcher]

Eurogamer has a very solid, well-explained methodology page (that I cant find right now), but most importantly, they actually use it in their reviews. I couldn't tell you how many times I have read an IGN review and found the score doesn't match the text, or the score matches the text but doesn't match what their methodology page says, or the scoring on the PS2 page feels totally different from the Xbox one, etc.
The problem, my handsomely-named friend, is that no one cares about methodologies. Referring to the dreaded 7.6 in the other thread, Gamespot has repeatedly made the point that 7-ish games are considered good and worthwhile of purchase. Nevertheless, people are losing their minds over it because of what the general perception of 7.6 means. Even abstracts like stars are converted to letter grades in most people's minds. I could use a rating system of Red, Q, ~, and Zaftig, and it would still be Gamerankings-ified into percentages.

[Angrycoder]

Fuck You
Shit Bonerz
Maybe
Fanfuckingtastic

[Bill Dungsroman]

RD
TFANRF
WFT10PD
WFTB
WFTP
CPSHST
NC
WMC
WC
BB
DNPO20
DNPO10
WD
CWBIFYYSE



That is:

Release Day
Time for a New Rig, Fool
Wait for the $10 Price Drop
Wait for the Bundle
Wait for the Patch
Console Port, Should Have Stayed There
Niche Crowd
Wal-Mart Crowd
Weak Clone
Bargain Bin
Do Not Pay Over $20
Do Not Pay Over $10
W4r3z D00d
Consider Whomever Bought it for You Your Sworn Enemy

[steve]

So could there be half-ratings, like "Release Day-Time for a New Rig, Fool." or "Do Not Pay Over $15" or "Warez for Your Sworn Enemy?"

[Bill Dungsroman]

So could there be half-ratings, like "Release Day-Time for a New Rig, Fool." or "Do Not Pay Over $15" or "Warez for Your Sworn Enemy?"
Totally!

[Raife]

One should never encourage fan fiction - even in jest.

http://johnfreemanjohnfreemanjohnfreeman.ytmnd.com

[BobJustBob]

Here's my idea for a rating system: no numbers, but rather you list two similar games, one better, one worse.

For example: Full Auto. Better than Need for Speed: Most Wanted, worse than Burnout Revenge. New Super Mario Brothers: better than Kirby Canvas Curse, worse than Yoshi's Island. PGR3: better than Gran Turismo 3, worse than Kirby Air Ride.

Like that. Over time you could have a map built up, where you find a genre game you like and see what's considered better, and know those are games you should try out. And if you didn't know either of the games being compared to, you'd still have the text of the review.

The only real downside here is that the subjectivity of the reviewer would be a much greater factor.

[steve]

The only real downside here is that the subjectivity of the reviewer would be a much greater factor.
Tom Chick would always have Flying Heroes in the "better" column, and Deus Ex in the "worse" one. And that would be for every game in every genre.

[Troy S Goodfellow]

The only real downside here is that the subjectivity of the reviewer would be a much greater factor.

There's also the fact that I'm not always sure that what I like in a game can be mapped that transitively, even within certain genres. I'm still not sure if I like Rise of Nations more or less than Age of Mythology - a debate I should have settled by now. Moving outside of genres would make this sort of thing impossible.

Plus, the extremes are the easy part. BfME2 is better than Cossacks II. This just gets muddy in the middle.

Troy

[jfletch]

For example: Full Auto. Better than Need for Speed: Most Wanted, worse than Burnout Revenge. New Super Mario Brothers: better than Kirby Canvas Curse, worse than Yoshi's Island. PGR3: better than Gran Turismo 3, worse than Kirby Air Ride.


GMR did that in each of their reviews... but they didn't last long. Anecdotally I dont think gamers liked it much (I did).


The problem, my handsomely-named friend, is that no one cares about methodologies.

The only problem I ever have with these sites is methodologies and I have embraced Eurogamer since I learned they have a good one. If nobody cares about methodologies, why do people care about scores? To bitch and yell at each other?

[Troy S Goodfellow]

If nobody cares about methodologies, why do people care about scores? To bitch and yell at each other?

Nobody wants to see hot dogs being made, but they damn sure want ketchup on it.

Troy

[fuzzyslug]

Here's my idea for a rating system: no numbers, but rather you list two similar games, one better, one worse.

For example: Full Auto. Better than Need for Speed: Most Wanted, worse than Burnout Revenge. New Super Mario Brothers: better than Kirby Canvas Curse, worse than Yoshi's Island. PGR3: better than Gran Turismo 3, worse than Kirby Air Ride.

Like that. Over time you could have a map built up, where you find a genre game you like and see what's considered better, and know those are games you should try out. And if you didn't know either of the games being compared to, you'd still have the text of the review.

The only real downside here is that the subjectivity of the reviewer would be a much greater factor.

I don't think that translates well for review purposes but would sure be fun on a selective basis as a forum topic.

[steve]

The only problem I ever have with these sites is methodologies and I have embraced Eurogamer since I learned they have a good one. If nobody cares about methodologies, why do people care about scores? To bitch and yell at each other?
What's so interesting about Eurogamer's methodology? I'm looking at the site and seeing, um, a 10-point rating system. Is there some description of it somewhere that I'm too stupid or lazy to find?

Our scale is pretty complex:
5-stars: Really, really good
4-stars: Really good
3-stars: Sorta good
2-stars: Not so good
1-star: Really not so good

[fuzzyslug]

What's so interesting about Eurogamer's methodology? I'm looking at the site and seeing, um, a 10-point rating system. Is there some description of it somewhere that I'm too stupid or lazy to find?

Our scale is pretty complex:
5-stars: Really, really good
4-stars: Really good
3-stars: Sorta good
2-stars: Not so good
1-star: Really not so good

If you describe this scale in these exact words within the magazine's covers, you've got yourself another subscriber.

[jfletch]

What's so interesting about Eurogamer's methodology? I'm looking at the site and seeing, um, a 10-point rating system. Is there some description of it somewhere that I'm too stupid or lazy to find?

Our scale is pretty complex:
5-stars: Really, really good
4-stars: Really good
3-stars: Sorta good
2-stars: Not so good
1-star: Really not so good

I found it: http://www.eurogamer.net/scoring_policy.php

[steve]

If you describe this scale in these exact words within the magazine's covers, you've got yourself another subscriber.
That's the exact wording we use in our review section.

I should note it has changed before. A year ago, in our April issue, we changed our ratings to:

5-stars: The Bee's Knees
4-stars: The Cat's Pajamas
3-stars: Copacetic
2-stars: Horsefeathers
1-star: Hooey!

(edit: I wish I was the clever person who came up with this, but the idea and verbiage was courtesy of the lovely and talented Sparky.)

[steve]

I found it: http://www.eurogamer.net/scoring_policy.php
Good lord.

I'm going to go out on a limb and say that if you think your scoring system needs that much description, there's something wrong with your scoring system. I do like the rhyming, however.

This is amusing: "At the other end of the spectrum you've got marks out of three or five: the kind of scoring systems that newspapers use to give a general guide, but hopelessly inexact for specialist publications trying to score things right on the money."

Um, our "hopelessly inexact" system is identical to theirs, since we, like most people that use the 5-star system, use half stars too. So 3.5=7, 2.5=5, etc.

[Charles]

http://johnfreemanjohnfreemanjohnfreeman.ytmnd.com


I can't stop laughingcrying.

[Hans Lauring]

I found it: http://www.eurogamer.net/scoring_policy.php

You gotta be kidding me! That's not methodology, that's a writer in dire need of an editor... the rhyming is ok, though.

I'll go with Steve's... actually considering that no Danish newsagents carry his publication, I think I'll go check out Zinio to see if his mag is as good as his posts here leads me to believe.



Here's my idea for a rating system: no numbers, but rather you list two similar games, one better, one worse.


Actually the quite good (but not as good as the quite similar Edge) British magazine Games uses this, as well as a simple 1-10 scale. They have to pictures and the legend 'better than' and 'worse than' - the first is sometimes substituted with 'just like' and the latter with 'As good as' (but never both of course)
So Outrun 2006: Coast 2 Coast gets an 8/10 and "Just like: Outrun 2" and "Worse than: Burnout Revenge".
I like it, but most importantly Outrun 2006, which is on the cover and which they really like gets four full pages and Me and My Katamari for the PSP, which is merely ok, gets half a page - it would still work without the score. Count me in the 'nevermind how they score, as long as they can write'-camp.

[Chris Nahr]

I do have to say, I'm disappointed that Eurogamer couldn't keep up the rhyming for all of the subsections. For example, how about "Three -- not for me, thank thee!" Or perhaps "Ten -- not a question of if but of when!"

[Rob Beschizza]

I've got a fantastic idea! They should rate games with a segment of descriptive prose English about 500 words long that details the qualities and shortcomings of the game!

[steve]

I've got a fantastic idea! They should rate games with a segment of descriptive prose English about 500 words long that details the qualities and shortcomings of the game!
Whoa, that's a radical idea. Can I use it?

[fuzzyslug]

I've got a fantastic idea! They should rate games with a segment of descriptive prose English about 500 words long that details the qualities and shortcomings of the game!

Crazy shit. It'll never work.

[Justin Fletcher]

GMR did that in each of their reviews... but they didn't last long. Anecdotally I dont think gamers liked it much (I did).
As did I.


The only problem I ever have with these sites is methodologies and I have embraced Eurogamer since I learned they have a good one. If nobody cares about methodologies, why do people care about scores? To bitch and yell at each other?
To clarify, most people don't seem to care about other people's methodologies:

"Gamespot says 7.6 is a good game? Bullshit! 7.6 is a "C" and everyone knows it! Gamespot's score is WRONG!"

[Desslock]

Bullshit! 7.6 is a "C" and everyone knows it!

It's a B+ in most of the world, actually.

[Mark Crump]

It's a B+ in most of the world, actually.

I always though I B+ was 85+

[Tom Chick]

I don't know about this "most of the world" Desslock is talking about, but in the good ol' US of A, a 7.6 ain't no B I've ever heard of. An A is a 90-100, a B is 80-89, a C is 70-79, etc.

Of course, I've been out of school for a while, so maybe they changed it when I wasn't looking.

-Tom

[Ben]

I LOVE the "better than" "worse than" idea. Everyone should use it immediately.

Remember, y'all might think you are humorists or journalists or whatever in your flights of fancy, but when it comes down to it you're telling me what I should buy. So just cut through the crap and do exactly that:
Game A is a better use of $50 than Game B if you are the sort of person that liked Game C. Repeat as necessary.

[steve]

So just cut through the crap and do exactly that:
Game A is a better use of $50 than Game B if you are the sort of person that liked Game C. Repeat as necessary.
It's great if every reader has played Game B or C. If not, I'm not sure how useful it is.

Such a system makes it even even more difficult to maintain any sort of consistency when you use dozens of different reviews, all with different--and potentially contradictory--choices for B and C.

All of this can be better handled in the text when relevant, because that would allow some explanation and nuance. But I think it's somewhat lazy to just fall back on comparisons for reviews. You're always better off focusing on the strengths and weaknesses of the title you're reviewing.

[Marcus]

I don't know about this "most of the world" Desslock is talking about, but in the good ol' US of A, a 7.6 ain't no B I've ever heard of. An A is a 90-100, a B is 80-89, a C is 70-79, etc.

Of course, I've been out of school for a while, so maybe they changed it when I wasn't looking.

-Tom

All I know is I wish that I would have gone to Desslock's school! Man I woulda had straight A's all the way through! Man thats some shit right there.

[Desslock]

I don't know about this "most of the world" Desslock is talking about,

Anywhere other than the USA,

[steve]

Anywhere other than the USA,
So Gamespot and PC Gamer are using European school scoring?

How sophisticated of them.

[Mark Crump]

Actually, according to the PC Gamer scale, 50% is "average."

[Bill Dungsroman]

Yeah, are you sure you're not confusing it with the 4.0 scale, where a 2.7 is a B?

[Hans Lauring]

It's great if every reader has played Game B or C. If not, I'm not sure how useful it is.

Such a system makes it even even more difficult to maintain any sort of consistency when you use dozens of different reviews, all with different--and potentially contradictory--choices for B and C.

All of this can be better handled in the text when relevant, because that would allow some explanation and nuance. But I think it's somewhat lazy to just fall back on comparisons for reviews. You're always better off focusing on the strengths and weaknesses of the title you're reviewing.


As an editor, you must know that readers read box quotes, headlines, the text under the pictures and everything like that before going into a long read (or it can be considered another entry into a long read) - Games is merely using several different ways of presenting the same picture, they're not relying on it (in that issue there was four or five racing games reviwed and they were all 'Worse than: Burnout Revenge' - pretty consistent).

I know it's not what you mean... at least I hope so, still haven't found your mag, but everything can be presented better in the main text, so we could just do away with all the little boxes containing additional information. I'd personaly love to read a four page review of Outrun 2006 done as one solid block of text...

[Wholly Schmidt]

Since we're all just weighing in, I like the 1-5 scale, numbers, stars, whatever. I keep a running list of movies I've watched, basically just for my own amusement, and use that list.

5 stars, buy it, recommend it as often as your friends will tolerate
4 stars, recommend it when someone asks "What should we watch?"
3 stars, go along with it when someone asks "What should we watch?"
2 stars, protest when someone suggests it
1 star, if they pick it, go home.

I sometimes try to more objectively qualify it, but in the end it's not like I need to. Like:

5 - Without any real flaws
4 - Either a couple minor flaws, or something very specific. Like, the best movie ever about some subject with such a limited appeal that you still can't recommend it without at least a little qualification.
3 - Uninspired, but functional. Nothing technically wrong with it, but nothing to elevate it above every other competantly made movie. You could watch it, or you could pick from hundreds of others just like it. So why bother when there are 4 and 5 stars out there?
2 - Sort of the opposite of a 4, or maybe the inverse, derivative, or compliment. I don't know, 1-5 sequentially is as far as my math goes. It's got maybe one or two redeeming features. Maybe a great actor stuck in a horrible movie, or a killer soundtrack, or a double-bladed lightsaber. If that's your thing, you suffer through it, but you keep it to yourself.
1 - Everyone is worse off for this movie existing. Realistically, it won't show up in my ratings much because I won't see it in the first place.

[steve]

As I]Games[/I] is merely using several different ways of presenting the same picture, they're not relying on it (in that issue there was four or five racing games reviwed and they were all 'Worse than: Burnout Revenge' - pretty consistent).
Oh, I know. It's a nifty idea, in theory. But it falls apart under two scenarios: One is if I've never played Burnout, which makes the comparison meaningless. The other is if I did play Burnout and hated it, in which case I'll assume I'll hate all of those other games. In either case, the full text should give me the information I need to make a more informed choice, but if sidebars and boxes and such are designed to get me to read the text, they've done a poor job.

Of course I'm looking at this backwards, for the reasons not to do it. You can make just as compelling a case for why this is a good idea. But I'm generally not to keen on ways to keep people from having to read a short article. I still won't go as far as remove the ratings, because I know people dig those too much.

When CGW dropped its ratings, people made a fairly big deal about it. It's a fine idea, but they're still using a summary box and "Editor's Choice," which still gives people a reason not to read the text. Instead of saying, "I can't believe you gave it 4-stars," they'll throw your summary text or Editor's Choice back at you, still ignoring the main body.

Or worse, they won't even talk about your reviews anymore. They'll be even more irrelevant.


I'd personaly love to read a four page review of Outrun 2006 done as one solid block of text...
Yikes, I sure wouldn't. 2000-2500 words on Outrun... well, I'd like to see someone try to find that much to say about a game like that, but I sure wouldn't want to be the person writing the article.

[steve]

Since we're all just weighing in, I like the 1-5 scale, numbers, stars, whatever.
The best thing about it is that you don't need to define it. Everyone gets it.

Letter grades are equally good for this reason.

[Rob Beschizza]

Whoa, that's a radical idea. Can I use it?
Awesome! :)