Glad Avault that cleared this up for us!

QuarterToThree Message Boards: Free for all: Glad Avault that cleared this up for us!
Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Friday, June 22, 2001 - 04:33 pm:

Adrenaline Vault Announces New Overall Ratings

The Adrenaline Vault has updated the Overall ratings for its PC and Console reviews to more clearly indicate which titles are recommended for purchase. While the Criteria ratings remain unchanged, the Overall ratings feature new descriptions that match the numerical ratings. This means that Half-Life, released in 1998, still earns a 5-star, or “Incredible!” rating for its visuals, among other scores for additional criteria, but that its Overall rating of 5-stars now means, “An Instant Classic! Must Own.” On the other hand, the underwhelming Blaze & Blade, released in 2000, still earns a solitary star, or a “Terrible!” rating, for its graphics, but is now deemed “Not Worthy of Being a Coaster!” with its one-star Overall rating.

Brian Clair, publisher of the Adrenaline Vault, said of the new method, "This change in the Adrenaline Vault's Overall rating is one of the biggest we've made to our system since it was created. Our hope is that this alteration will give readers a more crystal clear understanding of our feelings toward a game as a whole."

The updated Overall ratings fit both new and archived reviews. The Overall rating refers to the Adrenaline Vault reviewer's final thoughts about a game. Should you spend your money on it? That's what the Overall rating helps to answer. For a complete explanation of the new system, go to the Adrenaline Vault’s PC Review Criteria page.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Michael Murphy (Murph) on Friday, June 22, 2001 - 04:49 pm:

Wow. I'm certainly glad they published the thorough explanation. That whole five-star system has always confused me. Is one star better than two? Should I buy a five-star game? The reviewer had very good things to say about the game, but only gave it four stars! What's up with that?

Sheesh.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Dave Long on Friday, June 22, 2001 - 05:01 pm:

There are a significant number of people out there who think Adrenaline Vault is the best game review web site there is.

*shakes head*

--Dave


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Friday, June 22, 2001 - 05:15 pm:

1. I apologize for mangling the subject header.

2. I love this part:
"On the other hand, the underwhelming Blaze & Blade, released in 2000, still earns a solitary star, or a “Terrible!” rating, for its graphics, but is now deemed “Not Worthy of Being a Coaster!” with its one-star Overall rating."

Obviously Avault is now trying to "outdo" the hallowed CGW rating system which calls "terrible" "1-star" games "coasters". See? Avault is braver. Avault is MORE CRITICAL.An Avault One Star review is "NOT worthy of being a coaster"! Therefore an Avault reviewer presumably has more rings on their wooden tables while CGW staffers do not.

Brilliant!

-Andrew


Top of pagePrevious messageNext messageBottom of pageLink to this message  By David E. Hunt (Davidcpa) on Friday, June 22, 2001 - 05:24 pm:

I was also confused by that news item on Adrenaline Vault. I don't exactly understand what they changed or how it improves a reader's understanding of the rating.

Their game reviews may not be top notch, but I do like their hardware reviews. Paul D. Sullivan generally provides through and understandable hardware reviews. He also reviews quite a number of products each month.

-DavidCPA

PS...It may simply be an error, but when I flipped over to avault to check the spelling of Paul's last name, he was not listed as an editor in their "About Us" section anymore. He even has a review posted today.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Michael Murphy (Murph) on Friday, June 22, 2001 - 05:26 pm:


Quote:

This change in the Adrenaline Vault's Overall rating is one of the biggest we've made to our system since it was created.




This line just cracks me up! Anybody else enjoy that as much as I do? If this is the biggest changes they've ever made...
Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Friday, June 22, 2001 - 05:35 pm:

It's filled with good lines, but I'm still stuck on the "Coaster" thing. When, exactly, did the term "coaster" become the worst metaphor for "bad game"? I mean, outside of CGW.

I can imagine Avault's staff sitting there in an earnest "Biggest Overall Ratings Change Meeting" saying... "What can we use to elaborate on 'terrible'?" Finally, at 4am one of them says "Why, this game is so bad it isn't even useful as a COASTER!" (I imagine him saying that while stashing a copy of CGW hurridly under his chair.)

It just lacks imagination.

-Andrew
PS: Next up for Avault, a legless "Adrenaline Monkey" mascot.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By David E. Hunt (Davidcpa) on Friday, June 22, 2001 - 05:49 pm:

I am a blind man. Avault has a posting today for a new hardware writer - Doh!

-DavidCPA


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bill McClendon on Friday, June 22, 2001 - 07:39 pm:

What's ironic about the whole coaster thing is that CDs make really crappy coasters. They have no rim, aren't absorbent at all, and you end up with a pile of water in your lap.

Maybe somebody needs to be creative, and call it somethinb different. Give it the "Thank God For Generous Return Policies" award or something.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Coconut Funky on Friday, June 22, 2001 - 09:18 pm:

"PS: Next up for Avault, a legless "Adrenaline Monkey" mascot."

"I would gladly boot you in the man pouch, but I have no legs."


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Jason Levine on Friday, June 22, 2001 - 09:57 pm:

"What's ironic about the whole coaster thing is that CDs make really crappy coasters. They have no rim, aren't absorbent at all, and you end up with a pile of water in your lap."

Not to mention that rather inconvenient hole in the middle.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Tom Ohle on Friday, June 22, 2001 - 10:55 pm:

I use old games as coasters... they work well enough.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Michael Murphy (Murph) on Friday, June 22, 2001 - 11:00 pm:

I don't use coasters, those I used to use one of those AOL free trial offer CDs as one...


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Frank Greene (Reeko) on Friday, June 22, 2001 - 11:46 pm:

"I don't use coasters, those I used to use one of those AOL free trial offer CDs as one..."

I currently use one as a coaster for my coffee mug. I get a kick out of people asking what it is. I get to explain to them how AOL offered me 500 hours of free internet time in one month, and how I would have to spend over 15 hours a day trying to use them.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Aszurom on Saturday, June 23, 2001 - 08:10 pm:

Actually, if you get a small wheel of cork and glue it onto the bottom of the CD, it works BEAUTIFULLY. Ideally, make a sandwich with two CD's.

Hrm... I could start turning a profit from those AOL CD's.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Jason_cross (Jason_cross) on Sunday, June 24, 2001 - 12:40 am:

Oh YEAH? Well WE have had a 5-star rating like FOREVER! =P

We call one star "contemptible." Beat THAT!

Seriously though, I think the 5-star rating is probably best. It's very self explanatory, and intuitively obvious. It's what lots of reviewers use, like those for hotels and restaurants and crash saftey tests and stuff. Movies generally use 4 stars (some use letter grades). The 10-point scale is hard... is 7 average? Is it like grades in grade school? What if there are half points?

The 100-point scale is needlessly complex. Could someone please explain to me what EXACTLY is the difference between a game that gets an 82 and one that gets an 85? Is 50 average, or is it "failing" like if you got a 50 on a test in school?

I think if you were going to use a 10-point scale (particularly with half-points) or a 100-point scale, you'd be better off just saving everyone the trouble and going with school letter grades. If a game gets a B+, I know how good it is.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Sunday, June 24, 2001 - 01:15 am:

No - Jason - wait.
AVault has ALWAYS had a Five Star Rating.

The news release is pointing out that they are "naming" the stars now.

1 star = Not even coaster material
5 stars = Aeron Chair.
It's the funky fresh and witty names that are the new thing!

---
Also Jason, you are aware that your CG 5 point system includes halfs, therefore it's a 10 point system? Right?


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Mark Asher on Sunday, June 24, 2001 - 12:08 pm:

"Also Jason, you are aware that your CG 5 point system includes halfs, therefore it's a 10 point system? Right?"

It isn't the same, though. Five stars is the top rating, not a ten. There is no seven or 70%, so there is nothing that obviously correlates to a school grade scale, which is the problem that 10 point and 100 point systems have. For those, anything below a 7 or 70 seems like a crappy game. A 3.5 star game on the other hand is just a star-and-a-half away from the top score. That's a big difference.

There are no 10 or 100 point rating systems I know of where a 5 or 50 truly represents a score for an average game. You'll see a lot more 2.5 star ratings with a 5 point scale than you'll see 5/10 or 50% ratings with the other scales.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Mark Asher on Sunday, June 24, 2001 - 12:11 pm:

The other thing that's nice about a 5 point system is that it's really easy to label.

1 star = terrible
2 stars = below average
3 stars = average
4 stars = above average
5 stars = fantastic

Or something like that. It's much more intuitive.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Dave Long on Sunday, June 24, 2001 - 01:09 pm:

I've always used 2 1/2 stars as the middle. I thought that was the generally accepted rule. Am I wrong? For example, I gave Rail Empires: Iron Dragon a 3 star review because I thought it was slightly above average. How do the rest of you handle the five star scale?

--Dave


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bruce Geryk on Sunday, June 24, 2001 - 02:48 pm:

"Also Jason, you are aware that your CG 5 point system includes halfs, therefore it's a 10 point system? Right?"

Actually, since there is no "zero-star" or "1/2-star" rating, it's a nine-point scale, not ten.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Sunday, June 24, 2001 - 03:05 pm:

Mark,

I've never really looked at 10 points systems and thought "school" as you seem to. In fact, PCGamer (a 1-100 system) calls 50% the 'average' not 70%. Whether or not readers care, or think of school, is unknown. Even on a 1-10 scale with 5 being average I'd call 6.5-below unrecommendable. Maybe you're right and the majority do think about the A-F system when looking at percentages.

"Actually, since there is no "zero-star" or "1/2-star" rating, it's a nine-point scale, not ten."

Thank you Bruce! Now when did the "New Millenium" start again?

(CG has given a half-star before. It was my SkyDive! review as I recall)

-Andrew


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Supertanker on Sunday, June 24, 2001 - 03:59 pm:

I think normalized grading in law school ruined my thinking of percentage-based summaries, because I think of 77 or 78 as the mean, below 72 failing, and anything 85 or above to be outstanding.

Reasonable minds can differ, so none of these systems mean anything without a scoring key. The reader shouldn't assume the editors are thinking the same way as the reader (though I have no faith in humanity and think most readers do make this assumption). For example, PCG says a score of 70% is "Good. These are pretty good games that we'd recommend to fans of the particular genre, though it's a safe bet you can probably find better options." so I need to put aside my initial feeling that 70% is a dead bang failure. Are three stars in CG the same as three stars in CGW?

Because of all of these unanswered questions and assumptions, if I am actually interested in a game, I will read the whole review and ignore the score.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Steve on Sunday, June 24, 2001 - 06:12 pm:

Ratings are meaningless without the context provided by the text, so while I can make what I think is a convincing argument that the 5-star system is the only one that's worth using, it's ultimately irrelevant.

The text can stand on its own but the rating can't. For that game, the text serves as the definition for that rating.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Mark Asher on Monday, June 25, 2001 - 01:24 am:

"The text can stand on its own but the rating can't. For that game, the text serves as the definition for that rating."

Thing is, if you use a 10 or 100 point scale and argue that a game is average and give the game a 5/10 or a 50%, many readers will feel that the score and the review don't match. Average seems to be a 7/10 or so in the minds of many readers on those kinds of scales.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Jason_cross (Jason_cross) on Monday, June 25, 2001 - 01:39 am:

>The text can stand on its own but the rating can't.

But for a lot of people, in a lot of different mediums, it does. There are books full of hotel listings that show ratings but no "review text." Movies, too.

By the way, our scale goes all the way down to zero. It's hard to get half a star or zero stars--your game has to basically suck AND be so horribly broken that it doesn't work. A well-functioning piece of crap will usually get at least one star (which is still thoroughly "contemptible"). In fact, I don't know if we've ever given anything zero stars (Steve?).


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Michael Murphy (Murph) on Monday, June 25, 2001 - 09:32 am:

I got a copy of the Writer's Guide for CGM recently and it stated that a game should only be given a zero star rating if it fails to run, but anything that even runs should get at least one star.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Desslock on Monday, June 25, 2001 - 12:50 pm:

>normalized grading in law school ruined my thinking of percentage-based summaries, because I think of 77 or 78 as the mean, below 72 failing, and anything 85 or above to be outstanding.

Cripes, what law school did you go to? A lot of law schools in Canada mark on a "C" curve, so that the mean is about 65. Most people get predominately scores in the 60 and 70s, heh. The top 1% have a good mix of B's to go with A's. 71% failing? Sheesh, and they complain about gaming rating systems being poorly utilized.

>>The text can stand on its own but the rating can't.

>But for a lot of people, in a lot of different mediums, it does. There are books full of hotel listings that show ratings but no "review text." Movies, too.

I think a rating does stand out its own -- obviously, it's nowhere near as meaningful or descriptive as the text of the review (but then again, a 50 word review isn't as meaningful or descriptive as a 1300 word review).

You should be able to give a one word review, or just a rating, of any product - including a game. It won't be as helpful an assessment as a more substantive review, but it can still be meaningful, particularly if readers have prior knowledge of your previous ratings (to give them context when compared to the likes/dislikes of the reader).

Stefan


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Jason Levine on Monday, June 25, 2001 - 01:44 pm:

"Cripes, what law school did you go to? A lot of law schools in Canada mark on a "C" curve, so that the mean is about 65. Most people get predominately scores in the 60 and 70s, heh. The top 1% have a good mix of B's to go with A's. 71% failing? Sheesh, and they complain about gaming rating systems being poorly utilized."

Everyone who isn't a lawyer should be loving this thread by now. Anyway, Stefan, grade inflation has struck U.S. law schools as much as other areas of education. Up until last year, at the law school where I teach, we were instructed to grade on a B- curve. The result was that our students were suffering in the job market because of overall lower GPAs than comparable schools. So last year we were directed to bump it up to a B curve.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Monday, June 25, 2001 - 02:27 pm:

"Cripes, what law school did you go to? A lot of law schools in Canada mark on a "C" curve, so that the mean is about 65."

I think we've been through this before. Canada doesn't HAVE an "educational" system. Hell, even my undergraduate Communications BA required a 75% average, and that was a State School. My wife, going for her MA, is supposed to maintain an 80% average or she's out of Marquette. I'm actually afraid to learn what Bruce Geryk's Dr. Dr. thing requires....

-Andrew


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Jason Levine on Monday, June 25, 2001 - 02:34 pm:

"I'm actually afraid to learn what Bruce Geryk's Dr. Dr. thing requires...."

The empirical evidence would suggest that it requires a psychotic fear of elves. But I suspect it's a bit more than that.


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Desslock on Monday, June 25, 2001 - 03:25 pm:

>Canada doesn't HAVE an "educational" system.

What the heck does that mean?

>Hell, even my undergraduate Communications BA required a 75% average, and that was a State School

Those thresholds aren't particularly meaningful if you have criteria that's skewed towards giving a mean average higher than that. If the "bell curve" is 85%, then people who get more than half of an exam incorrect (and should, therefore, get a mark less than 50%) could get a 75%.

Similarly, with a 65% bell, people who would have gotten in the 80s could get dropped down, although in my experience with law students, that'd be really unlikely. Heck, there's more truth to the "highest F" gets and "A" dicta due to bell curves.

Stefan


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Bub (Bub) on Monday, June 25, 2001 - 03:50 pm:

Me: Canada doesn't HAVE an "educational" system.

Stefan: What the heck does that mean?

Me: It means you probably needed this after it ;-)

My wife reports that Marquette's Advanced Nursing program doesn't use a Bell Curve. The 80% or better she needed on her Pharmacology tests required an 80% average (or better) on each test. Needless to say a Communications BA also wasn't curved.

But in the US in all levels of schooling, the scale goes like this:

90-100%=A
80-89%=B
70-79%=C
60-69%=D (D generally = Not Passing)
~59%=F

Apply that to PCGamer as you whilst.
-Andrew


Top of pagePrevious messageNext messageBottom of pageLink to this message  By Steve on Monday, June 25, 2001 - 04:24 pm:

>>I got a copy of the Writer's Guide for CGM recently and it stated that a game should only be given a zero star rating if it fails to run, but anything that even runs should get at least one star.

That probably shouldn't be stated in such absolute terms.

Granted, to go below 1-star you have to be truly awful... and not even fun awful. Just awful awful.


Add a Message


This is a public posting area. If you do not have an account, enter your full name into the "Username" box and leave the "Password" box empty. Your e-mail address is optional.
Username:  
Password:
E-mail:
Post as "Anonymous"