Hyped game + bad review = reduced clicks?

[Doug Erickson]

Asking the website crewmen here: does giving a hotly-anticipated game, such as a Nintendo marquee title, an "unexpected" review/score actually reduce views of your site? For example, if someone were to give, say, Metroid Prime, or the next GTA a low score, do you think that the amount of people visiting your site might decrease?

I had this discussion with Jason McCullough at lunch, suggesting that perhaps website editors score big releases higher and are more forgiving out of fear; that a backlash to Mario Sunshine getting a 4/5 or less might lead to reduced clicks and thus less revenue. Any honest editors want to admit to a fear of the will of the majority, or am I just being an ass?

Just wondering. Is credibility-to-clicks something that nags the web reviewer during the review process, or is purely a credibility thing?

[Mark Asher]

I have no idea. My guess is a review that goes against the trend gets more attention. Whether this translates into a change in traffic over time is anybody's guess.

[Anonymous]

It's tough to say. When it comes to website viewers there's more than a few different camps. The people who visit your site because they like it. The people who visit your site because they hate it. The people who might potentially visit your site by positive word of mouth. Those who will visit by negative word of mouth. The Hardcore crowd and the Casual crowd...etc..

Chances are neither of us are a member of all the groups listed above, so there is nooo way we can accurately gauge this sorta thing. From what I have noticed tho, negative reviews tend to only really piss off people who already own the game. People who don't own the game yet usually, and responsibly, file it as reference.

There is of course the whole snowball effect that comes with a bunch of people who own the game not liking the review, creating a bad feeling which some of the lesser involved groups pick up on. Soooo I dunno.

I figure it this way, why do people bother reading reviews after the fact? Validation of course, but it's still dumb to get upset. They made the purchase, chances are they like it or have fooled themselves into thinking so. Why can't they just leave the negative review alone?

WHY ARE PEOPLE DUMB?!?!?!?

[Anonymous]

I figure it this way, why do people bother reading reviews after the fact? Validation of course, but it's still dumb to get upset.

WHY ARE PEOPLE DUMB?!?!?!?

Because "most people" who "get mad" about reviews are kids.

[Anonymous]

DUMB kids!

[Doug Erickson]

Yeah, but I'll also bet that most of the clicks on a review site are kids, which begs the question: did Dungeon Siege get great reviews because the reviewer wanted to meet expectations and satisfy his ad-clicking audience, or did it get great reviews because the reviewer simply wanted a boost to his "credibility" by meeting these same expectations?

I've heard a few reviewers say to me, in person, that sometimes it's easier to give a flawed but hyped game a great score, especially after it has received high scores from other sites, just to avoid hate mail. Truth?

[Mark Asher]

I don't know if most clicks come from kids. Traffic tends to nosedive on the weekend, which makes me think most traffic comes from people surfing the web during work.

[balut]

I've heard a few reviewers say to me, in person, that sometimes it's easier to give a flawed but hyped game a great score, especially after it has received high scores from other sites, just to avoid hate mail. Truth?

Well I guess that would keep the idiots appeased. The site would have lost any credibility with the non-idiots, but since they're such a minority, that really wouldn't matter.

The problem is that most people are dumbasses. How else do you explain "Carrot-Top's" existence?

- Balut

[asspennies]

Didn't ....well, now I forget where it was, but Tom's review of Deus Ex - wasn't it so unpopular (and incidently, correct) that the site had to do damage control and write an unprecidented (I should hope) second review?

I wonder how many people who loved Deus Ex's slow psuedo-dungeon crawl swore off that site forever, all because of one bad review of a game they love?

I think the question has some merit. If you're looking for the most eyeballs, you try to appeal to the masses. And the masses are going to like most of the crap out there. So, they sell off a little journalistic integrity for some advertising dollars. It wouldn't be the first time.

[Anonymous]

Well, I can see how it would be easier to give the score the majority wants to see.

I guess we just need to remember that reviewers are people too, and are constantly doing this kind of stuff to seek a sort of acceptance or just to keep loud stupid people out of their hair.

You know tho, I remember one review I did for Sydney 2000 for Gaming Age. Eidos' PR actually e-mailed the site with:

"I saw your Sydney 2000 review :("

Tough to say if that was a joke or not, but is it right for PR to put guilt trips on reviewers like that? It wasn't like I cared that much, I can be sort of a jerk when it comes to bad games, and Sydney 2000 was a bad game.

Come to think of it, what was the last GOOD game based on the Olympics.

[chet]

A contrary to public opinion review will normally get mentioned more and drive more traffic. Fan boy sites will go crazy and put "mad links" to your review if it is negative.

We still get hunted down over some of our reviews from time to time when fanboys post in some obscure forum about their 5 year old game.

Chet

[Greg Kasavin]

I've tried to stop being ironic altogether, but it's difficult to respond to this thread in a straightforward manner. But I'm compelled to try so here goes: First of all, it's not like anyone would ever actually admit to acting out of "fear" in giving a game an inflated score, if in fact that's actually what happened. So Doug, if I were to flatly tell you that, no, this never actually occurs, there would be no reason for you to believe me, because for all you'd know I'm probably lying.

Secondly, and more importantly, pageviews no longer matter much to commercial sites. Pageviews and traffic don't pay the bills--they actually makes the bills go higher. There's no incentive for commerical sites to act purely in the interest of gaining more pageviews. It's a self-defeating battle that's killed off a lot of good sites over the years.

Third, as others have said, a polemical "off-kilter" review is more likely to generate more pageviews anyway--not that pageviews matter.

No one will deny the importance of credibility. And everyone would agree that being truthful and forthright is the ticket to gaining credibility. Overyhyping games or overinflating / deflating review scores doesn't help a publication's credibility. This is simple stuff.

Any decent reviewer doesn't go into an assignment with a lot of preconceived expectations, even if the assignment is a very highly anticipated game. Acknowledging that a game is highly anticipated isn't the same thing as expecting it to be good, and giving a highly anticipated game a high score doesn't automatically make you a sellout or a hack if the game itself is actually good.

My spider-senses tell me that this thread might venture into the topic of payola--the other explanation for why popular games get high scores. It's been weeks since that dead horse has been flogged here.

[asspennies]

I don't know if I'd blame payola so much as simply the apparent tendency for reviewers to buy the hype - no matter what the hype is. And it may be subconcious, as well.

The best recent example is the oft-discussed Black and White - no one today could argue that it deserved the majority of scores it got. Yet when it came out, the hype machine was rolling fast and furious, and many thought that if it plays like the best game ever for two hours, it must continue that way.

Was this sort of hype-following a cynical attempt to grab eyeballs? I'm not entirely sure, but certainly many sites had to be first to get the review out, accuracy be damned.

Will WarCraft 3 (which, for all its good points, certainly has faults) receive the same treatment? Some would say it already has. Would Daikatana have been reviewed as poorly as it was, and by as many publications, had it not had the "negative hype" it received? (Speaking as someone who actually played through Daikatana the entire way through, I certianly hope it would - the game was terrible.)

Maybe Greg is right about pageviews and their ilk not being the focus of websites anymore, but certainly there must be some advantage to having the review for the hot game (be it famous or infamous) first to get some sort of recognition. So the question remains - do some sites skim a few ounces off the proverbial quality bucket to make sure their review is the first in line?

[Jason McCullough]

Secondly, and more importantly, pageviews no longer matter much to commercial sites. Pageviews and traffic don't pay the bills--they actually makes the bills go higher. There's no incentive for commerical sites to act purely in the interest of gaining more pageviews. It's a self-defeating battle that's killed off a lot of good sites over the years.

Really. That's...completely insane. Is running a gaming website just completely unprofitable now?

[Tyjenks]

I am not sure if this factors into this discussion or not. It seems to me that every time there is a hyped game released, Gamespot and IGN come out with their reviews on the exact same day.

Maybe they have a gentleman's agreement to post 2 days after a game's release.

Maybe they do like in grade school and whisper to each other what they got for question number 34 and compare answers.

Maybe it is a total coincidence.

Maybe I am paranoid and imaginig things and the aliens are not going to invade in the year 2012.

Just wonderin' if anyone else had noticed.

[Doug Erickson]

Sorry, Greg, I'm not buying. I've written for a number of websites and magazines, and I know what a powerful force positive reader feedback is, especially to your average insecure reviewer.

That's assuming, of course, that the reviewer has played through the entire game and isn't building his review based on preview content, in which case regurgitating expectations is the safest route to take.

Like asspennies said, how are we, the readers, expected to explain away uniformly high ratings for Black and White, Turok 2, Dungeon Siege, Rogue Leader, and Metal Gear Solid 2? (the jury's still out on WC3, although MP has been around long enough for the problems to be apparent.) These games aren't necessarily bad, but they *are* demonstrably flawed in ways that are pretty damn self-evident. Shouldn't we at LEAST see a few breakout scores that call the games to task by these intellectually-honest reviewers? Every one of these games I've mentioned never makes any "Best Of" lists, and is often dismissed as mediocre or crap now that they've come and gone, although you wouldn't have guessed it by revisiting the reviews that appeared around their release.

They all have the following in common:

1. They had crazy levels of hype, supported with boatloads of fawning interviews and preview specials galore.
1a. Because they were made as part of a popular series, or by a popular developer.
1b. Or were part of a console launch library.

I don't buy the payola argument, unless we're talking about Adrenalin Vault. Payola cases are usually painfully obvious, and always involve cut-rate games who HAVE to have their reviews purchased. Peter Molyneaux doesn't have to buy anything; he's seen as a credible developer, with plenty of wide-eyed reviewers and critics willing to play the apologist role for a piece of his credibility pie.

You've told us you aren't guilty. So what's your explanation?

[Dave Long]

They probably hit the exact same day because they're both trying to get the review out day and date with the game's release or as close to it as humanly possible. That's the web for you. It's just a part of their MO.

It's hard to argue that it's bad because people want the review immediately if possible. Most readers probably also believe it should be there because, well, it's the web and it's an "I want it now" culture when it comes to the Internet because often you can get it now.

There's no great conspiracy of game reviewing. That much is certain. With so many freelancers and different internal reviewers at all these various publications, it'd be impossible for us all to get together and come to a conclusion. Payola wouldn't work either simply because no company could afford to pay us all, not even Microsoft.

Those of us that have written reviews all can ask you the same question time and time again... where's the proof? No one ever comes up with anything because it just isn't out there. We're not sitting around fixing scores and writing our reviews with the publishers looking over our shoulder. We're just sitting in a dirty, old computer room at a desk that probably looks a lot like your own making a judgement call on a game we got in the mail. It's really not rocket science. Some are better at defining their exceptions and accolades than others, but if someone's on the take, I've yet to meet him.

--Dave

[Anonymous]

Didn't ....well, now I forget where it was, but Tom's review of Deus Ex - wasn't it so unpopular (and incidently, correct) that the site had to do damage control and write an unprecidented (I should hope) second review?

No. Games Domain (www.gamesdomain.com) has done "second opinions" on games since it started back in, oh, 1995. Browse the site and see just how many "unprecedented second reviews" there are. It's a way for the site to review games from different perspectives. Some games with initially good reviews get bad second opinions, and vice versa.

[Doug Erickson]

I'm not implying a conspiracy. In fact, I don't believe anything of the sort.

Rather, I'm simply trying to figure out why hyped mediocre titles get incredibly high marks, and what little piece of reviewer psychology leads to that sort of result.

[Greg Kasavin]

You've told us you aren't guilty. So what's your explanation?

My explanation of what, the fact that I've given positive reviews to highly anticipated games? My reviews are my explanation--they stand for themselves. I've read your complaints about my Warcraft III review, and nevertheless feel that it holds up to intense scrutiny, same as all of my reviews. I put a lot of time into them up front so that I don't need to defend them later. This has proven to be a worthwhile strategy for me--I highly recommend it!

Black & White always comes up in these discussions, rightfully so. I should at least point out that it's not really a "recent example" as asspennies said, since it's a year-and-a-half old. I've acknowledged before that, of all the hundreds of reviews I've ever written, it's probably the one and only one that I wish I could go back and adjust... mostly just the high score I gave it rather than any of the points I wrote down. In hindsight I certainly don't think the game is "crap." But it probably deserved 8-range scores from people instead of 9-range scores. Do I regret recommending the game and possibly influencing people to buy it? No, I think Black & White was definitely an unusual experience and by all means worth playing. I felt a big part of the problem with the game was Lionhead's utter failure to produce a patch in a timely manner, something my review couldn't realistically have anticipated. Many players experienced technical issues that weren't resolved for months.

At any rate, it was a superficially impressive game that really caught some critics off guard. Maybe some of the critics bought into the hype (not Steve!), but I do think those that rated it highly were genuinely impressed with the game's originality and style. Yes, Black & White didn't end up having a ton of lasting value, so I think if the critics are to be faulted for any one reason, then it should be for the fact that most of them probably just didn't play it enough before reviewing it. But no, I don't think they were "afraid" to give it a low score. Go back and read the reviews and you'll see that people genuinely liked the game. They were charmed by it.

It's funny, because I've always felt that most all of Peter Molyneux's games give a great first impression but then taper off and get un-fun in a hurry, all of a sudden. Dungeon Keeper was very much that way for me--lots of novelty for the first 20 hours, and then all at once, I had the whole thing figured out and completely lost interest. While playing Black & White, I distinctly remember thinking, "No way is this game going to have that same problem." Oh well. My own experience with Black & White served as an effective reminder that I should always make double- or triple-sure that I've played the game I'm reviewing to the point of exhaustion.

[Anonymous]

how are we, the readers, expected to explain away uniformly high ratings for Black and White, Turok 2, Dungeon Siege, Rogue Leader, and Metal Gear Solid 2?

The thing is, the official review scores for these generally aren't too far out of sync with the reader scores.

(game: gamespot score/reader score)

Rogue Leader: 9.4/9.0
Metal Gear Solid: 9.6/9.1
Turok 2: 9.0/8.8
Dungeon Siege: 8.4/8.3
Black & White: 9.3/7.9

The biggest gap is for B&W, but even that's just a hair shy of "great" according to the polls. For a bigger reviewing travesty, Soldier of Fortune 2 received an 8.2 from Gamespot and a 6.0 from readers. I wrote the sof2 review, and I wasn't bought off and I can honestly say that my being scared only accounted for .4 of the overall score.

What's the point? Your guess is as good as mine. The gamespot readers aren't sheep, or else they would have backed me up a little on fucking sof2. I can only conclude that the reviewer's taste generally matched the taste of the readers in the examples you cited. Plus, though I hated MGS2 and would love to think that the scores were over-inflated, it received game of the year nominations all over the place.

[Doug Erickson]

Most of those so-called reader reviews are written the moment the review is posted, and are designed as nothing more than deliberate fodder in the console wars (espcially the Turok 2 and Rogue Leader reviews). If you actually read the reviews, you find perhaps one or two that are thoughtful and reflect some degree of playtime as well as gaming experience - the rest are all hyperbolic crap designed to boost the average. Besides, how many of the reviews written well after the game's release are in the upper range? Usually, none - most are far more thoughtful and often excuse themselves with "after all the hype..."

If you read a few message boards, and talk to folks who play games, most of them found the games I've listed to be spectacularly average, if not bad (Turok 2 is downright bad, period; the level design is criminal, and the PC version actually netted the title reviews that reflected such). I'd like to know why the disjoint between "early bird" review scores and the opinions post-hype for most of these titles is so, well, obvious.

[Jason McCullough]

My initial reaction - Greg, you think B&W was worthy of an 8? Has grade inflation gone that far?

[Dr Fear]

I'm not implying a conspiracy. In fact, I don't believe anything of the sort.

Rather, I'm simply trying to figure out why hyped mediocre titles get incredibly high marks, and what little piece of reviewer psychology leads to that sort of result.

Maybe the part of reviewer psychology that gets people excited about games, since as far as I know all reviewers are people except Steve Baumann. If you're excited about a game, you're probably going to like it better than some game you're not that excited about, at least if it doesn't completely suck. And since most big games on major sites and in the magazines are reviewed by people who are big fans of that game's genre, they're probably even more excited about the game than the average gamer. Because in the end, everybody writing these things is a gamer and is a human.

I don't know. Maybe you could argue that it is totally the opposite - that these "big fans" of the genre should be more demanding, and take more convincing than your average player. So that would be the exact opposite of the previous psychological profile. I am not a psychologist. And unless you do some kind of study with surveys and psychiatrists, you aren't going to get much of an answer, especially on a messageboard. Who cares, anyway? Should there be a game reviewing institute where only old men who hate games are kept away from all they hype so that they can be free to issue their hyper-critical reviews because they're always constipated? They are game reviews. jesus.

[Anonymous]

My initial reaction - Greg, you think B&W was worthy of an 8? Has grade inflation gone that far?

Yeah, well, why don't you just go head over to Quartertothree.com with all the other curmudgeons. Oh, wait.

[Greg Kasavin]

That was me; forgot to log in.

[Tyjenks]

Who cares, anyway?......They are game reviews. jesus.

I think the point is that someone was interested in this topic and started a thread. Others also found it interesting and added to the debate.

Unless in your Bizarro world the "Games" section of message boards are strictly for talk of urinary tract infections and diseases of the bladder, this "games" section of this discussion board is for talk of games, issues relating to games, and diseases of the bladder.

[Doug Erickson]

So why is it that great, UNHYPED games get these "balanced" reviews/scores - like Age of Wonders 2 and Disciples 2 with their apparent B&W-level quality play, according to Greg's revised score - but hyped games always get the benefit of the doubt? According to Doctor Fear, it's also because reviewers love games.

If I posit that both AoW2 and D2 might've received at least an extra .5 added to their scores if they'd had twice the previews and "Blizzard" on the box (along with the legion of attendant fans and their trigger-happy email urges), would anyone disagree?

(Not saying I disagree with the Disciples 2 and AoW2 scores, BTW, although they're both lightyears beyond B&W.)

If you love games so much you can let excitement add at least two points to a review score, then I guess I just can't compete in this little hippy commune. Me, I still think it's some sort of intellectual dishonesty - a desire to meet expectations and play it safe with the noisier internet element than actually call a spade a spade. Or a fatty a fatty - whatever gets Chet on my side.

[routlaw]

I only did reviwing for a short period of time (the demands of the real world and all of its horrors caught up to me), but I wanted to comment on a couple of points brought up in this thread, and then point to how they (possibly) might interrelate.

Jason's got a good point-much of the time the big-name games, no matter how poor they are when released, seem to be immune to getting average grades with reviews that gloss over the apparent flaws to back up the score. Certainly there's an argument to be made that big-name games generally are big-name for a reason-bigger budgets mean better art and graphics, longer development times lead to more polish, etc. Discouting that, there is still a large discrepancy between the scores given to a mediocre lower-cost game and a mediocre big-budget title.

If the game is medicore, I don't see why the improved production values warrant the 10-15% increase in the final score given in the review. Not surprisingly, that relatively small differential is located in the key 6/10-7/10 range, the psychological point at which a game goes from 'respectable, but flawed' to 'not ever worth thinking about again'. I know in high school that my friends who got a 6/10 or a 7/10 on their AP Lit essays felt much different about their scores-both were in the end middling, but the 6/10 barely passed and the 7/10 owner could wipe the blood of his lips and fight on in the next essay. It's a big difference, and I don't know if reviewers take it into account when scoring their games.

As for influences on reviewers, I'd say that falls into two camps: the fansite reviewer and the paid, commerical website reviewer. The former has a drive to keep in favor with the big publishers so that he can keep getting free swag and kudos from the fans that read his/her site, and the latter has a drive to keep in favor with his coworkers in the gaming industry, both in the publishing business and in the game development business. The paid reviewer working for a website has a lot of time working with people inside the industry, and working with people inside the industry. They see the games created from pre-annoucement to release, and talk to (or know someone on staff who talks to) the people working on it. I don't think it's a stretch to say that these relationships give the big-profile games a bit more leeway, both in criticism and reviews, than the games they get mailed by the lower-teir publisher, and then unceremoniously dropped on the co-op or newblood in the office. It only gets worse as you start to cloud the deal with big-name studios like Westwood and Blizzard, who seem above reproach based solely upon their past successes.

I realise that no real critical analysis on games can be done in a clean room, and that there will always be some external factors outside the quality of the game that will influence the reviewer. However, when there's such a lack of consistency in the reviews of the hype have's and have nots, you've got to ask if that room isn't more dirty than it is clean.

[Tyjenks]

If you love games so much you can let excitement add at least two points to a review score, then I guess I just can't compete in this little hippy commune. Me, I still think it's some sort of intellectual dishonesty - a desire to meet expectations and play it safe with the noisier internet element than actually call a spade a spade. Or a fatty a fatty - whatever gets Chet on my side.

I had not thought on this subject much until my much anticipated Heroes IV came out. A wonderful game which I am still playing that has many, many faults, unfortunately. I, of course, read all the reviews to see what folks were saying and waited a week to purchase the game. Many noted a few problems, but otherwise they loved it.

Jonah Falcon's review for one, a day or two before the game's release, gave it a ringing 5 out of 5 endorsement. Upon further investigation, I find what a rabid Heroes fan he is. Then, as some here can no doubt attest, when discussing games he cherishes on the CGOnline boards, he often seems a little, how shall I say, all over the place and hair triggered when it comes to his discussions/arguements. I cannot trust a reviewer who seems so totally biased. How can a game get a perfect score when there are soooo many valid and substantiated complaints?

There are other examples too, but this one just stood out.

I used to think I could read maybe 6 or 8 reviews from trusted and semi trusted sources and get a general opinion about a game. I am beginning, finally, to realize much can get glossed over or ignored. I believe some reviewers are just as guilty by omission as they are of, possibly, inflating a score ever so slightly.

[Jason McCullough]

Jonah Falcon's review for one, a day or two before the game's release, gave it a ringing 5 out of 5 endorsement. Upon further investigation, I find what a rabid Heroes fan he is. Then, as some here can no doubt attest, when discussing games he cherishes on the CGOnline boards, he often seems a little, how shall I say, all over the place and hair triggered when it comes to his discussions/arguements. I cannot trust a reviewer who seems so totally biased. How can a game get a perfect score when there are soooo many valid and substantiated complaints?


This gives me a perfect segue into something that's been annoying me lately. The definition of bias is:

http://dictionary.cambridge.org/define.asp?key=bias*1+0

'A tendency to support or oppose a particular person or thing in an unfair way by allowing personal opinions to influence your judgment.'

Is it even possible to declare a game "fun" based on something other than personal opinions? You're not reviewing bridge construction, where it's quite clear whether it falls down or not. It's an annoying tick that everyone constantly mentions "bias" when talking about gaming reviews, as if there's a logical equation out there somewhere that converts a game box into "fun" or "not fun". Sure, you can make a reasoned case as to why the game is fun, but you base that on, shock: personal opinions.

Even the most insanely wrong reviews I've seen (Falcon comes to mind) at least have reasons, albeit bad ones. The only other explanation I can think for this verbal tic is that the speaker thinks the reviewer has a financial interest in seeing the game succeed, but that doesn't sound right either.

Maybe political argumentation in the United States has debased the language enough that "bias" translates to "disagreeing with me?" I mean, Falcon isn't "biased", as in "giving you an opinion based on other than the merits," he's just wrong. His review makes an argument that "you'll enjoy HOMM4," proceeding with arguments from whatever his premises are. That either the premises or arguments are wrong/inconsistent/completely kooky doesn't make the review "biased."

[Doug Erickson]

Reviews are subjective, agreed. BUT: there are certain elements we can largely discuss that are common to good game design.

Graphics: glitchy graphics and artifacts are bad.Constant rendering errors are bad. Design elements that obscure your ability to recognize and effectively engage the game's mechanics are bad.

Interfaces are easy to criticize in pseudo-objective terms as well: too many layers of navigation, or strictly defined paths to accomplish common tasks are bad. A lack of hotkey options is bad. Illegible fonts, cryptic glyphs, and a lack of mouseovers are bad. A good interface allows for multiple nav paths, makes important information easily discovered and read, and tries to keep as much on the top level as possible. A good interface understands the game it represents and ranks its data accordingly.

The whole notion of "good gameplay" is a bit shakier, and it behooves the good reviewer to avoid meaningless nitpicks, or taking said nitpick and painting it in hyperbolic terms. A valid gameplay criticism is one that often requires a scenario, and that can easily be experienced to the annoyance of most players. Certainly, graphics and interface are part of "gameplay" to most people - if the graphic design is muddy and confusing; if the representation of the gameplay is imprecise or glitchy; if the interface causes too much frustration, it's safe to say that the gameplay is going to be significantly impacted.

But when we discuss the DESIGN of the game - the theme, the goals, the player objectives, how the individual atomic tasks and actors lay out - well, then things get a bit ooky.

Reviewers often walk a fine line between their expectations of a game within a genre, and what the game actually is. For example, I could review Warcraft 3 in the context of Warlords Battlecry 2, and while I'd have a number of valid criticisms to level at WC3 by comparison, it simply wouldn't be 100% germane, because "being better than WC3" was never an implied design goal. I'd probably cite WBC2 as an example of a great interface, or of options I'd like to see in all RTS titles of that flavor, but I'd never rank WC3 on that comparison alone.

Rather, when reviewing a game professionally and not bitching on a message board, I like to break the gameplay down into the discrete tasks I am expected to perform and judge exactly how well the game lets me pursue those tasks. The more frustration it incurs, and the less feeling of progress and development I get, the lower my score goes. It also behooves me, I feel, to clearly and expressly identify WHAT tangible elements of the game design are presenting the obstacles and incurring the frustration, and to determine whether or not these obstacles stem from deliberate design or through an implementation failure.When I identify these problematic elements, I must explain them to the reader in the language of the game itself.

If the design choice seems deliberate, I evaluate it in the greater context of the game - is having to arbitrarily micromanage spellcasting in Warcraft 3 something that makes the game a better experience? Or is it an unfun limitation designed to balance play? I consider the same for the twelve unit bandbox limit - when I'm regularly making MORE than 12 units, is this design decision a good one? Is the justification apparent, and how many intellectual hoops do I have to leap through before I can come up with a good excuse? Is my response to these limitations one of appreciation or of irritation? Good designers can and do make bad gameplay decisions, and sometimes, the effects of these decisions run pretty deep - often deeper than a simple design failure. In this case, the individual design choices just don't work within the context of the game overall; while they may be individually valid, they invalidate or contradict other goals, or they simply do not have the intended effect because game design/technology has moved onward and established more optimal solutions for the same design problem.

If the design "choice" seems to be a failure - usually when ambition exceeds the technology, or when the designer was simply incompetent - it's often easy to articulate. When a game revolves around playing bots and has a sloppy online component, you can bet that having crappy bot AI will be a major gripe. When you play a mission-based game like AquaNox, and the mission objectives aren't obvious on the screen or fail for reasons that do not seem to be apparent, that's a design FAILURE. The design goals of the game are very clear, and the implementation in no way reaches them.

Anyway. There's a language of game criticism that can be clearly communicated to the average reader, if the reviewer is willing to put in the time to analyze his idea of "fun". Just saying something "isn't fun" or "doesn't work" won't cut it with me.

[Brad Grenz]

The Idiot's Guide to Game Review
by Doug Erickson

Available this Fall at a Borders near you.

[Mark Asher]

"If the game is medicore, I don't see why the improved production values warrant the 10-15% increase in the final score given in the review."

So you're saying that production values don't count? If two games are otherwise equal, but one has better production values, it shouldn't get a higher score?

I'm finding that production values are more and more important to the enjoyment I get out of a game. I have less tolerance for poor art, a dated engine, poor voice acting, etc. Maybe that's just because as the industry ages games naturally become more derivative, so enhanced production values set them off from previous incarnations.

[Anonymous]

Unless in your Bizarro world the "Games" section of message boards are strictly for talk of urinary tract infections and diseases of the bladder, this "games" section of this discussion board is for talk of games, issues relating to games, and diseases of the bladder.

I had a urinary tract infection once -- I gave it a 2. Now, the ensuing kidney infection, that was at least an 8. Of course, I was paid off by the fine people at Bayer AG, who make Cipro, a fast-acting and delicious cherry-flavored antibiotic with markedly few side effects. Available at a pharmacy near you, ask for it by name.

[Anonymous]

I think the original post had a legitimate point. It made me think back to the pc gamer review of diablo 2, they sent a reviewer who had been anticipating the game for months. It seems impossible to me that a reviewer that is looking forward to a game as much as he had been would have a completly neutral bias toward the game. If it had been a total piece of crap I'm sure he would have said so. I would be willing to bet though that little flaws would have been overlooked. I have no idea if there were any little flaws in that example, just that the potential was there. At least in that case the bias was admitted up front. Anticipation and fandom has got to play a role in the review. Now whether the small .1 or .2 that it makes up really makes that much of difference is debatable, but you have to admit that it's there.

[Doug Erickson]

Well, if a game is highly anticipated by the reviewer (as with any gamer), the game only has to satisfy the minimum expectations of said reviewer to guarantee an uberscore.

If the game isn't hyped, it doesn't have the attention of the reviewer prepped and engaged, and hence must sell itself purely on its own terms, in a vacuum. That's why great games like Disciples 2, Age of Wonders 2, and WBC2 get "good scores - they're honestly good games - but they're lacking the hype that's needed for a reviewer to really become enamored of them and throw himself into them wholeheartedly and unreservedly. It's a completely dishonest approach, but it's forgiven because its done in the name of positivity. It's still scoring a game based on expectations, rather than a rational evaluation of the game itself.

Hype is often a self-fulfilling prophecy. Publications hype a game, getting themselves excited about it based on developer/premise, and eventually wind up overscoring the game simply because they're all too willing to spend time with it and excuse its shortcomings.

[Jason McCullough]

It seems impossible to me that a reviewer that is looking forward to a game as much as he had been would have a completly neutral bias toward the game.

Augh! He said it again! He said it again!

[Qenan]

I find it hard to imagine that reviewers in general do other than say what they think. There just isn't enough money at stake to be worth doing otherwise...

Social pressure to "go along" with the predominant view might have a minor chilling effect, but on the other hand, being different helps you stand out. I think it's a wash.

[wumpus]

Reviewing? It's very simple-- never be afraid to tell people when you think something sucks. For an example, you need look no further than Tom Chick. He's made an entire career out of this one simple rule.

[Tyjenks]

I mean, Falcon isn't "biased", as in "giving you an opinion based on other than the merits," he's just wrong. His review makes an argument that "you'll enjoy HOMM4," proceeding with arguments from whatever his premises are. That either the premises or arguments are wrong/inconsistent/completely kooky doesn't make the review "biased."

OK, you got me on semantics. I threw the word "bias" in, in haste and you sure nailed me on it. Bravo!! You shall not provoke me again, Mr. Sneakyman. Even if you were sorta right on this one.

How about this:

Me no likey his review.

[Bub, Andrew]

This gives me a perfect segue into something that's been annoying me lately. The definition of bias is:

http://dictionary.cambridge.org/define.asp?key=bias*1+0

'A tendency to support or oppose a particular person or thing in an unfair way by allowing personal opinions to influence your judgment.'

Is it even possible to declare a game "fun" based on something other than personal opinions? "

Jason, nice rant, but when people accuse game reviewers of "bias" (my favorite is when they say "YOU ARE BIAS!" all in caps) is that they think game reviewers are "biased" in favor of the game developer. Or in Doug Erikson's case, biased in favor of, um, "hype". My opinion on this whole debate can best be summed up by something really excellent Tom Chick wrote here at Quarter to Three very recently:

"And, of course, you're going to hear a lot of uncritical acclaim about how it's 'fun'"

Yes. Exactly. Uncritical acclaim. Well put! There are a lot of game reviewers, but there aren't a whole lot of critics.

[Murph]

Agreed. I think it is possible to be biased, even when it comes to something that is, ultimately, based on nothing other than your personal opinions.

Example: I gave Neverwinter Nights an extremely high score when I reviewed it for Gamersclick. Am I biased? I sure don't think so. I legitimately love the game. It does have problems, but to me, they're inconsequential. The important part is that I feel the text made that clear: I discussed the problems that the game had, but made it quite clear that I, personally, feel like the game is worth buying in spite of it.

But, I can't deny that I bought into the hype before it was released, and I can't deny that Bioware is one of my favorite developers. So, had I not mentioned the bad stuff, as well, and someone had called me biased, they would probably have been right.

Even when discussing something totally subjective, a certain amount of fairness is expected, I think. If you're not being fair, then you're being biased...Wouldn't you agree? Not mentioning the compatibility issues that plague Neverwinter Nights in an attempt to make the game look better would be just as unfair (and, thus, biased) as not acknowledging the robust character creation/customizability options that Bioware built in by incorporating the 3rd Ed. rules.

Just because the review is nothing other than your personal opinion doesn't make it impossible to be biased. You still have to be fair.

And, in spite of the problems, I think Neverwinter Nights is totally worth the money, so I think my score is justified.

[Jason McCullough]

'Even when discussing something totally subjective, a certain amount of fairness is expected, I think. If you're not being fair, then you're being biased...Wouldn't you agree? Not mentioning the compatibility issues that plague Neverwinter Nights in an attempt to make the game look better would be just as unfair (and, thus, biased) as not acknowledging the robust character creation/customizability options that Bioware built in by incorporating the 3rd Ed. rules.'

Ok, that makes sense. I've just never seen it applied that way by anyone; all I see is "U R BIAS" attached to perfectly fine reviews. ;0

[Brad Grenz]

There's also the inevitable, "All your BIAS are belong to us!"

[Erik]

Michael Murphy said:

I gave Neverwinter Nights an extremely high score when I reviewed it for Gamersclick. Am I biased? I sure don't think so.

From Michael Murphy's review:

Neverwinter Nights is, without a doubt, one of the most highly anticipated, eagerly discussed, most ambitious projects that has ever been undertaken.

This isn't just an extremely high score - it's the highest score ever given to any game. In fact, it's one of the highest scores ever given to any thing, especially when you consider the really pretty impressive scope of humans and their projects, for instance the highly anticipated Newton's Laws of Motion II. I'm not sure if this even counts as bias, because it's just insane.

[Murph]

Okay, so...Ummm...Crap. Unfortunately, neither the words "game" nor "gaming industry" made it into that statement. Which does, of course, make me look like a big idiot here and now.

That doesn't change the fact that very few people -- if anyone, ever -- would read that and not infer that I'm referring to the industry, and of course, Erik's just trying to make me look like a fool. Well...

Erik=1
Murph=0

Still, that doesn't totally invalidate everything I've said. I don't think.

I was trying to acknowledge the hype surrounding the game. I may forever lament my omission of the word "game" in the sentence Erik quoted.

[Tom Chick]

Is Neverwinter Nights not one of the most highly anticipated, eagerly discussed, most ambitious projects ever?

Sorry to pick on you, Murph, but Erik's point is well taken. NWN is just a game. When someone mentions "the most ambitious projects ever", I'm put in mind of stuff like the Allied landing at Normandy and sending a man to the moon.

However, rest assured that you're not alone. This sort of hyperbole is endemic in game journalism [sic]. You're fitting right in. :)

And I have to admit I just turned in a pretty enthusiastic review of NWN in which I rated it as comparable to a cure for cancer.

-Tom

P.S. Not really.

[Brian Rucker]

I was once asked to review a game and turned the gig down because I knew I was too biased. When I first started freelancing for Gamersclick I suggested I shouldn't probably do reviews because I have very definite and particular views about what makes games good - and I wasn't at all sure they meshed with the reviewing needs of mainstream readers. Someone has pointed out the difference between critics and reviewers. Both roles are valid. The reviewer keeps firmly in mind the expectations and experience level of his audience while looking at a title in the context of similiar offerings. The critic relies more on his personal experience with and theories about quality in anticipation of what makes for great games and measures new titles against those often dauntingly high, and extremely subjective, standards. This can damn entire genres that are otherwise playable and enjoyed by a huge audience.

Would a reflective music critic think much of Britanny Spears? Is there a huge audience that does have it's own standards in judging the work of Britanny Spears and comperable artists? Who serves this very real audience best? It's not the critic. Ultimately if Britanny Spears and her cohorts deliver entertainment to the audience and that audience has a roadmap they can relate to all is well. The best reviews will tend to look like each other if reviewers understand the subject and the audience. This doesn't have to do with collusion or herd mentality per se. It has to do with being a professional. The creme of reviewers can deliver that useful information to their audience with a unique spin and a bit more insight - verging on critical analysis without really stepping over the line into pure opinion.

On the other hand, without opinionated critics whose commentaries will rarely resemble each other (at least over time) the coverage of the form as a whole will lack greater context and encouragement for qualitative advancement.

[Mark Asher]

"Sorry to pick on you, Murph, but Erik's point is well taken. NWN is just a game. When someone mentions "the most ambitious projects ever", I'm put in mind of stuff like the Allied landing at Normandy and sending a man to the moon."

Of course, if he had simply written "most ambitious [game] projects ever?" I don't think he would have been out of line.

[Sam Jones]

Didn't ....well, now I forget where it was, but Tom's review of Deus Ex - wasn't it so unpopular (and incidently, correct) that the site had to do damage control and write an unprecidented (I should hope) second review?

Deus Ex was just absolutely awesome. I have rarely enjoyed a game as much as I did this. The only things I didn't like were minor interface concerns, like automatically picking up all objects off of a corpse.

[Jack]

I hate RTS games. Even the best RTS game leaves me frustrated and frantic -- not what I call a good gaming experience. I'm secretly hoping (not secret anymore) that W3 will fail and the RTS genre will begin swirling in the bowl.

I mention this because all this discussion/argument about bias is very amusing -- but it appears most posters are overlooking the nature reviewing any art: intrinsic, subconscious bias will always exist. I simply wish most reviewers were more up-front with such bias. Note Tom Chick's review of W3, when he says gameplay is a mess (I'm paraphrasing) and concedes that's what most RTS fans probably want; I found his admission refreshing. He's stating his bias and then deconstructs the game. (It was curious that Mark's review was partly a review of Tom's review, but that's another topic).

My point is that I could play the best RTS game in the world and still think it's crap. Yes, the game could be technically tight, but I wouldn't be able to give it stellar marks because I don't know what an RTS fan sees in those awful things. Think of the movies: I could see the best damn Julia Roberts chick flick of all time; it could be technically perfect. But I'd still leave the theater feeling a little queasy. No amount of technical perfection could swing my intrinsic bias.

To expect reviews to be unbiased is absurd. A review, by nature, is a personal opinion and that should be kept in mind by the reader.

Pre-release hype is blade that can cut both ways: a reviewer could be a happy drone and let himself be influenced by PR hype or he could be a anti-social bug and try to "de-throne" a hyped game. In either case, the personality of the reviewer comes into play.

There is no such thing as lack of bias in journalism -- and most of all in reviewing any art form. And games are a form of art that not only is a reflection of the designer, but also a personal reflection of the player. I think RTS games are a waste of code, but a fan would think they're the greatest things since the flush-toilet. Who's to say which opinion is wrong?

[Lurker]

"Sorry to pick on you, Murph, but Erik's point is well taken. NWN is just a game. When someone mentions "the most ambitious projects ever", I'm put in mind of stuff like the Allied landing at Normandy and sending a man to the moon."

Of course, if he had simply written "most ambitious 'game' projects ever?" I don't think he would have been out of line.

The context of the sentence in a computer game review on a computer game review site would suggest that when talking about "projects", they are, in fact, computer game projects. If he was writing for the entertainment/tech section of some mainstream mag, the dig would seem more appropriate.

Is there really not a review out there by Tom/Erik that also fails to put the words "game" or "game industry" into a sentence that, when pulled out of context, sounds like they've lost grasp of reality?

[asspennies]

From what I can tell, the principal problem here is that we're all elitists of sorts - you could say we want to have our cake and eat it too.

I think that more and more of us expect game reviewers to act like movie reviewers. With a market this big ($3 billion) there's a lot more games out there that are being produced. With this much money being thrown around, there are bound to be some well-made stinkers, some mediocre-but-playable games, some rough gems, some spectacularly ordinary games, and the all-too-rare absolute showstopper. A lot like the movies.

Invariably, the public likes more than most reviewers do. This is why I think Erik's point about the user ratings for games is invalid. Just see that Mr. Deeds made $80 million plus over the last weekend. Crap - especially well-advertised crap - still sells, and people will still buy it. Pick up Entertainment Weekly some time and take a look at their cinescope ratings. I don't think I've ever seen a rating lower than a C, even for movies that were universally panned. Many bad films get Bs or As.

No, what you see from movie reviewers is a much more cynical approach to the business. I think that most movie reviewers go into a film expecting NOT to like it, and enjoy being pleasantly surprised. Perhaps that comes from years of watching bad movies, although I imagine many a game reviewer has played their share of Deer Hunters. Less movie reviewers - at least the good ones - are willing to give movies the benefit of the doubt.

It always annoys me when friends of mine, or people I work with, opine that reading movie reviews somehow conflicts to their overall philosophy. Surely there are going to be the single review that you occasionally disagree with, but to constantly disagree with the majority of movie reviews is mostly a foreign concept to me. I suspect it is for most of the readers here, as well.

Ultimately, the problem may be that games are scored at all. Many movie reviews - the more respected ones, especially - don't list a star or grade rating. Perhaps a move towards that would help smooth over the problem; reviewers could praise a game for its style and ingenuity, and still not have to compare it to something else that's even better down the line which may get a worse "score"simply by default. I certainly don't imagine that listing the final score for a game is in any way an easy process.

[Bernie_Dy]

don't list a star or grade rating

In the old days, CGW didn't list review scores. I really liked that. You had to read the article to understand how the writer felt about the product. What a concept. Of course, that went away and in came a star rating. CGW editors said the market called for it, that readers want the quickly digestible stat.

Another problem about reviews is that the slate is wide open. You have to talk about the game, but what are the points of emphasis? What do you weigh more heavily, implementation or innovation? Graphics? What about the documentation? The product's resource usage? Interface? Game balance? Are the music and sound effects cute or are they also functional? Multiplayer? Co-op multiplayer? Stability? Extensibility?

By the way, you have to do this in less than 600 words, and no matter what you pick, there's always someone who will think you looked at the wrong priorities. Frankly, there's little room left to talk about the hype.

[Anonymous]

Jeez, you people are overthinking this. It boils down to this: if you disagree with a review, you look for some reason beyond "he liked it, I didn't" to explain why it doesn't match your own perspective. It's either bias, an attempt to get more/less clicks, blah blah blah. Or maybe that person is less discriminating, not as critical, or maybe you're just a cynical bastard who hates everything. It's generally a little "mine's bigger" contest to see who can be the most alpha-critical person on a message board. (Ooh, mixed bad metaphors. Rock on.)

I really like Dungeon Siege. I explained why in my review. Does this make me less discriminating and critical because some have decided it's this year's overrated whipping boy? I guess so. Hey, I'm still playing the game online with some friends. I still like the game. I don't take back that review. I didn't go into it with inflated expectations, or with the idea that it would suck.

Sure, some reviews really miss the mark when they don't mention some obvious flaws. But you can point-by-point debate me on any game if you'd like, and I'll definitely concede some of their weaknesses, but to look for anything beyond differing tastes to explain why I liked a game more than you did is sorta masturbatory.

[asspennies]

Steve, Steve. It always comes down to masturbation with you, doesn't it? ;)

The point that I was trying to get at is that it is harder and harder to trust game reviews these days. With so many sites trying to get the jump on others, trying to throw their reviews out the door as soon as possible, one is faced with the possibility that no review will provide a good idea of what game(s) to buy.

Everyone says "Find one reviewer and stick with his opinions" but no one says that for movie reviews. The fact is, like you said, taste is divergent. Like movies, I like to go for what the majority of good critics says is good. Unlike movies, I don't feel I can necessarily judge by that criteria in the gaming world. The fact that these are $50 expenditures and not $8 certainly plays a part.

While I don't think my suggestion of eliminating game scores is going to solve the problem (or even go over very well with the public) it may make reviewers less obligated to justify their scores, instead leaving them to focus on explaining their likes and dislikes of the game in a purely subjective way. Add a few of those reviews together, and you have something tangible to go on.

[Jason Cross]

I didn't read the whole thread, but...

It's been my experience that the hundreds of "Joe's Gaming Website" places out there have reviews that just parrot what everyone else says. Even if they're one of the first reviews, it's a plagarism of the "first impressions" articles and forum posts and stuff. It seems really obvious, looking at a bunch of these over time, that they're just afraid to say something outside the envelope of the other reviews. None of these websites wants to be the one to give Warcraft III a 7 out of 10, for instance.

I see some reviews that feel the same way on some of the big major sites as well. Reviews where they trumpet the same features and ignore the same problems.

There's definitely a bit of a "horde mentality" to the game press out there. Not every single one of them, granted, but the hobby as a whole. (I don't dare use the words "journalist" or "profession" - the professionals who make their living at it tend to act a little more like journalists and often have actual cirtical skills. They are in the minority.)

At least we're not the movie press. What a bunch of freakin' quote-whores.

[Anonymous]

Steve, Steve. It always comes down to masturbation with you, doesn't it? ;)
But of course. Self-love is the most gratifying of all.


The point that I was trying to get at is that it is harder and harder to trust game reviews these days. With so many sites trying to get the jump on others, trying to throw their reviews out the door as soon as possible, one is faced with the possibility that no review will provide a good idea of what game(s) to buy.
That's a different problem, though.


Everyone says "Find one reviewer and stick with his opinions" but no one says that for movie reviews.
Ooh, I completely disagree. The review in the Village Voice is radically different from the one from quotewhores like Joel Siegel. Some people love Ebert and stick with him, others hate him and think he's biased, blah blah blah. I had some e-mail exchanges with him a few years ago and we talked about that kind of thing. He gets the exact same accusations we get, only more of them.


While I don't think my suggestion of eliminating game scores is going to solve the problem (or even go over very well with the public) it may make reviewers less obligated to justify their scores, instead leaving them to focus on explaining their likes and dislikes of the game in a purely subjective way. Add a few of those reviews together, and you have something tangible to go on.
You explain your likes and dislikes TO justify your score. My reviews, and I suspect the vast majority out there, would be identical whether or not there was a rating.

[Ben Sones]

Everyone says "Find one reviewer and stick with his opinions" but no one says that for movie reviews.

Huh? They don't? You're kidding, right?

I always read all of Michael Wilmington's reviews (Chicago Tribune) because he rarely steers me wrong, but I'm a little wary of John Caro's reviews (ditto Tribune), because he likes a lot of movies that I hate. I used to like Roger Ebert's reviews, and I still like some of them (and his writing, which is unpretentious and straightforward), but he's become much more uneven over the past year or so. When I was living in Vermont, I ignored all reviews in the Burlington Free Press, because their reviewers couldn't write their way out of a wet paper bag.


The fact is, like you said, taste is divergent. Like movies, I like to go for what the majority of good critics says is good. Unlike movies, I don't feel I can necessarily judge by that criteria in the gaming world.

If you are going to lend equal weight to every review, then you are going to have problems. It doesn't matter if it's a game review, or a movie review, or a book review, or whatever. Some publications have higher standards and are more professional than others. If you are going to lump the web into the equation, the variance is extraordinarily wide. I'd trust a review from the Tribune before I trust one from Joblo's Movie Emporium, just as I'd give a lot more weight to a review from Gamespot than I would to a review from Drunk Gamers. If you aren't going to be discerning, then yeah--reviews are very hit and miss. Which is why it's good to be discerning. There's a reason why Rotten Tomatoes has a "Cream of the Crop" section.


While I don't think my suggestion of eliminating game scores is going to solve the problem (or even go over very well with the public) it may make reviewers less obligated to justify their scores, instead leaving them to focus on explaining their likes and dislikes of the game in a purely subjective way. Add a few of those reviews together, and you have something tangible to go on.

I'm all in favor of eliminating scores, but too many people like them. That's why most movie reviews have them, too. I'd even be happy with a simpler system ("recommend" or "don't recommend"--thumbs up or thumbs down), which at least doesn't try to make the scoring process seem more exacting and scientific than it really is. But I don't think that writers typically write a review to justify their score. Or at least, I don't. I rarely even assign a final score until after I've written the review.

[asspennies]

Huh? They don't? You're kidding, right?

...I used to like Roger Ebert's reviews, and I still like some of them (and his writing, which is unpretentious and straightforward), but he's become much more uneven over the past year or so. When I was living in Vermont, I ignored all reviews in the Burlington Free Press, because their reviewers couldn't write their way out of a wet paper bag.

Nope, not kidding. I think you've proven my point right there. Reviewers, even the best, are inconsistant. They also tend to agree with you about one genre but perhaps not another. Usually I find that an amalgam of reviews is much more telling then sticking with a chosen few. Rotten Tomatoes, you're my hero.

I still say you can't do that with games. I know all about GameRankings but therein lies the problem, right? Again we're back to rankings.


If you are going to lend equal weight to every review, then you are going to have problems. It doesn't matter if it's a game review, or a movie review, or a book review, or whatever. Some publications have higher standards and are more professional than others.

Surely. I really enjoy reading GameSpot. Since I'm more of an FPS fan, I have grown accustomed to Erik's reviews, and usually consider them a great resource. But I think relying on him as my only resource would be a mistake.


But I don't think that writers typically write a review to justify their score. Or at least, I don't. I rarely even assign a final score until after I've written the review.

I'm glad to hear that this is the case. I still think that sometimes the score tends to be more of a distraction than anything else. If you can boil your entire review down into a percentage and not miss anything, you didn't write a very good review. IMO. (I'm not suggesting that this is what any of you do.)

[Ben Sones]

Nope, not kidding. I think you've proven my point right there. Reviewers, even the best, are inconsistant. They also tend to agree with you about one genre but perhaps not another. Usually I find that an amalgam of reviews is much more telling then sticking with a chosen few. Rotten Tomatoes, you're my hero.

The reason I like Michael Wilmington is that I can usually get a good feel for a movie, even if I don't necessarily agree with his opinion. He'll review Movie X and love it, and I will read the review and say "well... that doesn't sound like something that I'd like." You don't have to rely on finding a reviewer whose tastes match yours perfectly. Which is good, since you never will. But the best reviewers are helpful even when their tastes don't match yours.

The reason I've grown more wary of Ebert's reviews is the fact that they have become inconsistent. He'll praise one movie for one reason, then damn another for the same reason. It makes it tough to gauge the film beyond his recommendation.

Review roundup sites can be just as misleading as review scores, too. I also like Rotten Tomatoes, but mostly because it gives you links to all the individual reviews, which I read. The "Tomatometer" score is as meaningless as any individual rating, however, and I wouldn't rely on it for movie recommendations.


I still think that sometimes the score tends to be more of a distraction than anything else. If you can boil your entire review down into a percentage and not miss anything, you didn't write a very good review.

True. The score is really just a summary, but no scoring system can do justice to well-considered criticism. One solution is to ignore scores. I try to, at least until after I read the review.

[Rob de los Reyes]

I'm inclined to agree with whoever thought these concerns were a bit overblown. Many reviews are poorly written, but still functional. Some beautifully written reviews are nonfunctional. I'm interested in, but care less about what a reviewer actually concludes than why he does so. If you tell me why you think what you think (even in horrible horrible sentences), then you have served your function.

I suppose, by the by, that "bias" could mean that people think the reviewer is too friendly with a developer to give an honest review. I can't speak to whether that's the case. It does make it tough to write the "rude" style of negative review that some people enjoy, but if you don't subscribe to the ad hominem attack school of negative review writing, you don't run into as much trouble here.

The only "bias" I can knowingly claim is this. I want to like the games I play. Whether holding a review copy or one I purchased, I want to like it. I love games. I love to love them. In that sense, to me, all games are good before you load them up. What a disappointment when they turn out to be awful. I don't seek such disappointment, but I will share it when it happens.

[Alan Au]

*insert rant about numeric scoring systems*

- Alan

[Lee Johnson]

*insert rant about numeric scoring systems*
2.5 / 5

[Anonymous]

I still say you can't do that with games. I know all about GameRankings but therein lies the problem, right? Again we're back to rankings.
Okay, I think I'm following a bit better what you're getting at. You don't see enough variety in game reviews. There seems to be more consensus opinion in game reviews than in movies.

That's a legitimate criticism, assuming there isn't some actual level of consensus about a large number of games. There's certainly a herd mentality with the press, and I've often wondered if some reviewers merely summarize the findings of others instead of doing their own evaluation, but the other possibility is that more people like more games than you do.

Tom might be able to answer this better than I, but I suspect it's easier to do movie reviews than game reviews. Both require critical thought, clear writing, sold analysis, and a sense of history. But I know Ebert turned me down when I asked if he'd be interested in reviewing some of those old "movie" games way back when, saying it just took too long to play games. Also, everyone's machine is different so the experience differs slightly for everyone, there's little consensus on what constitutes proper "control" or interface, the importance or lack thereof of story affects everyone differently... there's so many more variables to deal with in a PC game review than a movie review that I'm not sure how many similarities can be drawn between the two.

We're also still in the early stages of "game criticism." I suspect the majority of movie reviews were positive all the way up until the 50s and 60s.

[Desslock]

>I really like Dungeon Siege. I explained why in my review. Does this make me less discriminating and critical because some have decided it's this year's overrated whipping boy?

Yeah, wumpus, etc. seems to be postulating the inaccurate perception that a lot of gamers think that Dungeon Siege is overrated, which I think is nonsense. It seems like most gamers agree that the game is solid -- a well polished action/RPG, with beautiful graphics but repetitive gameplay (like every game of its type). 4,000 readers have written reviews at GameSpot, and the average rating coincides almost exactly with the review rating I gave the game for GameSpot.

Sure the game doesn't deserve the crazy 95% ratings it's gotten from a bunch of the fanboy sites, but most of the reviews from the larger, more professional publications seem to have a pretty consistent "very good" rating.

[Erik]

Okay, I think I'm following a bit better what you're getting at. You don't see enough variety in game reviews. There seems to be more consensus opinion in game reviews than in movies.

I think another reason for this is that movie reviewers feel qualified to review basically any movie that comes out. Game reviewers, on the other hand, tend to specialize in a few categories of games that they're predisposed to enjoy. For instance, Ebert obviously doesn't like the Adam Sandler genre; He doesn't find Sandler's sense of humor funny and he doesn't enjoy the movies Sandler is likely to make. But that doesn't stop Ebert (or any other movie critic) from reviewing them. So if I were to adopt the work habits of a movie reviewer, I'd start reviewing flight sims, and I guarantee I'd pan every single one. That would break up the concensus a little, unless I happened to luck into reviewing a really bad flight simulator.

[Desslock]

>I think another reason for this is that movie reviewers feel qualified to review basically any movie that comes out. Game reviewers, on the other hand, tend to specialize in a few categories of games that they're predisposed to enjoy.

That's exactly right.

[Ben Sones]

It's more than just enjoyment--there is also a time issue involved. It takes roughly two hours to watch a movie. Since only a handful of movies are released each week, reviews can be handled by only one or two people at most publications, and writing the reviews constitutes the bulk of the effort. Game reviews are different--it takes weeks to review many games, and even "fast" reviews (shooters, adventures) typically can't be done in less than three or four days. Writing the review is about 1% of the work.

That's why publications that run movie reviews often have only one to three reviewers, while game publications rely on a vast pool of writers. And when you can't cover everything yourself, as a writer, you tend to gravitate towards the types of games that you are most familiar with. Readers, by the same token, probably find it difficult to identify with (or even remember) the tstes and style of any given reviewer, simply because there are so many different ones.

[asspennies]

Not that I'm trying to rag on you guys, because these certainly are some valid points.

But a good game is a good game is a good game, right? And a bad game by any other name is still Daikatana. The best reviewers should be able to see those aspects in the best games. Whether or not they'd be able to provide the best write-up and accounting of features is certainly a very real concern.

Reviewers may not always agree, but really, that's all I'm hoping for. That way, when a game gets universally lauded, you know it's good.

Not just a decent FPS or a nice RTS. But a good game.

[Ben Sones]

It would be nice if it were that simple, but I don't think it is. The Sims was universally lauded. I can even see why, but I didn't like it. Like I said, just because many people like something, that doesn't mean that you will. In the end, everyone's tastes are unique.

[wumpus]

Yeah, wumpus, etc. seems to be postulating the inaccurate perception that a lot of gamers think that Dungeon Siege is overrated, which I think is nonsense. It seems like most gamers agree that the game is solid -- a well polished action/RPG, with beautiful graphics but repetitive gameplay (like every game of its type).
At some point, you have to realize that life is a zero-sum game. We only have a limited number of hours for entertainment. As Jeff Green once said, "If we take it as a given that playing games is a waste of time, then what could be worse than wasting the time spent wasting time?" Unless you're a huge fan of that particular genre, your time is better spent with other, superior games. It's not that Dungeon Siege is a bad game, it's that so many other games are better. My shelf is stacked with great games I never got a chance to play due to lack of time.


4,000 readers have written reviews at GameSpot, and the average rating coincides almost exactly with the review rating I gave the game for GameSpot.
I'm totally unconvinced that the GS reader reviews carry any real meaning. For one thing, the crazy-ass score averaging/subscore thing, and second, the fact that these reviewers have likely just finished reading the 'official' GS review and are thus even more predisposed to parrot the party line.

I'd have a lot more faith in a meta-review site like rottentomatoes, which I guess would be gamerankings in this case.

[Murph]

Well, it seems like a non-issue now, and maybe it's cheating, but I did get that one offending sentence in my review changed, for the sake of those like Erik and Tom -- not to mention that, as Tom said, it's pointless hyperbole. The point of the sentence was to acknowledge the hype, so now it reads: "Neverwinter Nights is, without a doubt, one of the most highly anticipated, eagerly discussed, most ambitious games that has ever been attempted."

Hope you guys are happy. I don't know if it's cheating to change the text of a review after publication, but it was only a slight change, and...well, that's the beauty of reviews online rather than in a magazine, I guess. :)

Now, back to the topic at hand...Carry on, guys.

[Tom Chick]

"Neverwinter Nights is, without a doubt, one of the most highly anticipated, eagerly discussed, most ambitious games that has ever been attempted."

That's better. However, if I were your editor, I would trim it even more. 'Without a doubt' is probably superfluous. 'Highly anticipated' and 'eagerly discussed' are redundent. Economy of words, young Murph, economy of words.

Okay, that's enough pedantry from me for the evening. I'm going back to lovingly gazing at my Warlords Battlecry II CD.

-Tom

[Murph]

Points noted. For the sake of sparing the guy running the site the trouble, I'll probably leave it as is for now. But, I certainly appreciate constructive criticism, and as I hope to be doing more and more of this, it'll surely come in handy in the future. Thanks Tom.

[Anonymous]

'Highly anticipated' and 'eagerly discussed' are redundent.

I disagree. They are redundant, though.

[Jason Cross]

We're also still in the early stages of "game criticism." I suspect the majority of movie reviews were positive all the way up until the 50s and 60s.

Interestingly, movies were driven largely by technology until that time, too. Adding music, then real sound with talking actors, and better film, wider screens, color, "technicolor", stereo, blah blah blah.

We're out of touch with that now, but it's an interesting parallel to the games industry. In the theatres, technology sold for the first two or three decades. By that point, movie technology was pretty much a level playing field for everyone and it came down to ambition of what's ON the screen (Gone with the Wind, Lawrence of Arabia...). That started to come around again with Star Wars and the special effects battles, but that's not quite the same thing.

We're in that stage with games now, and I wonder to what degree it affects critical evaluation? And in 10 years, maybe game developers will have the technology available to display pretty much whatever they want to, and the content will be king. There will be better and worse looking/sounding games (just as with movies), but it will be because of the skills and tools and budget of the creators, not because of the game's or platform's tech.

How will that affect the critical evaluation of games?

[Bub, Andrew]

Murph, buy a copy of Strunk and White's Element's of Style. It's the best little writing book in the world. Alternatively you can buy the Chicago Style Guide, which is useful for writing and lethal in hand-to-hand combat.

[Jason Cross]

4,000 readers have written reviews at GameSpot, and the average rating coincides almost exactly with the review rating I gave the game for GameSpot.

I honestly wonder, with big releases like Dungeon Siege, if the average reader review rating would have still matched your score if you had made it a point lower? Or if they were required to write their reader reviews without ever seeing what you gave it or said about it?

[Murph]

Murph, buy a copy of Strunk and White's Element's of Style.

I'm pretty sure I've got a copy. Or at least I used to. Haven't seen it in awhile, though. Might be worth re-investing in, if it doesn't turn up. Thanks. :)

[Ben Sones]

Interestingly, movies were driven largely by technology until that time, too. Adding music, then real sound with talking actors, and better film, wider screens, color, "technicolor", stereo, blah blah blah.

Brian Moriarty used to talk about this subject a lot; I don't know if any of you ever caught his "Entrain" presentation at GDC (back when it was CGDC... I don't think he does it any more), but it covered this topic pretty handily. In the early days of cinema, people were so entranced by the novelty of the media that they spent all their time doing simple tricks, like having an actor fire a gun at the camera (the audience, not used to seeing recorded images as we are, would be scared out of their wits). Eventually people got bored with that, and realized that film had the potential for more than cheap parlor tricks and visceral thrills. Games will get there.

Slightly off topic, but in the early days of film, there were plenty of people that didn't consider film to be an art form, either. Which is why I always smile and shake my head when I see that argument bantered about with regard to games.

[Mark Asher]

Murph, buy a copy of Strunk and White's Element's of Style. It's the best little writing book in the world. Alternatively you can buy the Chicago Style Guide, which is useful for writing and lethal in hand-to-hand combat.

It's only because you're telling someone to get a book on writing that I'm doing this, but there's no apostrophe in "Element's" in the title. It's not a possessive. It's not a contraction for "Element is" or "Element was".

[Bub, Andrew]

I also recommend the "Idiot's Guide to Grammar and Style".
Ahem.

[mtkafka]

I read Murph's review... I think its fine. Its much better than most of the recent NWN reviews. Its accurate and gives an informative pros and cons of the game. It doesn't intercede with pointless humor or leettalk. Its fine. Plus it pretty much is true that the game was 'Highly anticipated' and 'eagerly discussed'. Like wasnt there countless threads on NWN eagerly discussed here?!? And I know alot of people were highly anticipating it! So whats the beef?

etc

[asspennies]

I also recommend the "Idiot's Guide to Grammar and Style".
Ahem.

I'd also put punctuation inside of quotation marks, but now we're just getting snippy. ;)

[Ron Dulin]

Interestingly, movies were driven largely by technology until that time, too. Adding music, then real sound with talking actors, and better film, wider screens, color, "technicolor", stereo, blah blah blah.

I would argue that films are still primarily driven by technology. Blockbusters, at least. And they always have been, with the brief exception of artistic/popular convergence in the seventies. But games are more intrinsically linked to their technology. There are few films we remember solely because of their technological advances, while with games it seems to be the opposite.


Games will get there.

Do you really think so? There were only 23 years between A Trip to the Moon and Battleship Potemkin. 16 more and you have The Maltese Falcon. Spacewar is 42 years old. Pong and the Magnavox Odyssey are both 30. Despite constant proclamations to the contrary, electronic games aren't in their infancy. Perhaps there is some great golden age around the corner, but it's difficult to convince the cynic in me that it will be more than just better-looking cranial chunks in whatever incarnation of Soldier of Fortune my grandkids will be playing.

-Ron

[Ben Sones]

I'd argue that we've had some small revolutions along the way already. One of the problems, however, is that there is a lot more room for technological progress in computer games than there was in film. Film had plenty of advances, too, but most of them didn't give filmmakers entirely new ways to make films. With games we've had a constant stream of platform improvements, each creating the opportunity for new cheap parlor tricks, and that serves as a pretty effective distraction for designers. It's going to plateau, however; the technology will reach points where further progress doesn't add significantly to the designer's toolbox, and during those times I think you'll see more creative thinking and less technological fetishism.

Give it time. As a medium for public consumption, electronic games are only twenty years old or so.

[Desslock]

>I honestly wonder, with big releases like Dungeon Siege, if the average reader review rating would have still matched your score if you had made it a point lower? Or if they were required to write their reader reviews without ever seeing what you gave it or said about it?

By no means do GameSpot's reader review scores always come as close to matching the site's official score. When there's only a small sample size, that could be manipulated, etc., but (as anyone who has studied statistics can confirm) 4,000 people is a HUGE group of samples. Most political polls sample a few hundred or 1,000 people. Any sample size of opinions that large is pretty telling.

Stefan

[asspennies]

When there's only a small sample size, that could be manipulated, etc., but (as anyone who has studied statistics can confirm) 4,000 people is a HUGE group of samples. Most political polls sample a few hundred or 1,000 people. Any sample size of opinions that large is pretty telling.

Anyone who's studied statistics can confirm that a voluntary poll on a website, no matter its nature or size, is statistically insignificant and cannot be considered a scientific poll. It's a far different cry from a random sampling, which is what political polls are.

[Doug Erickson]

Reader reviews are worthless, regardless of sample size - all a large number of reader reviews indicates is controversy. The only people compelled to post a reader review are those who feel strongly in some way about the game, usually in a positive sense, with a few detractors posting ridiculous scores to bring the average down.

Look at GameFAQs reader reviews - mediocre games have hugely inflated scores, because the only people "motivated" to write a review are the game's erstwhile champions.

[Desslock]

>a voluntary poll on a website, no matter its nature or size, is statistically insignificant and cannot be considered a scientific poll

Not random sampling, so its finding can't be extrapolated to purchasers of the game generally. But it provides a very significant expression of opinion of those gamers who are hardcore gamers enough to view gaming websites.

If the number of responses was relatively small (as it is for some games), then the results would be less persuasive, since they'd be more subject to manipulation by interested parties or fringe, extreme opinions. With a sample size that huge, it's very likely that the average opinion reflects the view of the audience.

[asspennies]

I hate to push on this point, but it's simply not statistically valid, no matter how you look at it. It's not, as you suggest, "a very significant expression of opinion of those gamers who are hardcore gamers enough to view gaming websites." It's a poll taken of people who felt the need to go to the website, view a specific page relating to the game in question, and from there, decided to mark their own opinion down.

You've just limited the sample size three times, all in a voluntary way. You've eliminated any basis for scientific analysis and are left with a worthless and neccesarily slanted poll. I don't know what safeguards GameSpot takes to curb re-posting (be it the easily surmounted cookie method, or the less likely IP method, or even worse, no method at all) but to presume that the opinions given are from 4000+ random individual gamers who were compelled to post their opinion is, to put it mildly, absurd.

[Desslock]

>It's a poll taken of people who felt the need to go to the website, view a specific page relating to the game in question, and from there, decided to mark their own opinion down. You've just limited the sample size three times, all in a voluntary way.

Every poll of an audience has the same limitations. Questioning people who just came out of a movie is limited by the views of "people who felt the need to go", to "view a specific" movie, and "mark their opinion down". Yet people use those sorts of exit polls to judge a movie's popularity all the time. Simiarly, a political poll taken at a shopping mall is limited by the views of "people who felt the need to go" to that mall, at that time of day, to go to that "specific" store location where the poll was occurring, and "mark their opinion down".

The results of that poll get less useful the more you try to extend its findings to a broader sampling of people, but getting the opinion of 4,000 people in an audience provides an extremely compelling view of the opinion of that specific audience.

Stefan

[Jason McCullough]

>It's a poll taken of people who felt the need to go to the website, view a specific page relating to the game in question, and from there, decided to mark their own opinion down. You've just limited the sample size three times, all in a voluntary way.

Every poll of an audience has the same limitations. Questioning people who just came out of a movie is limited by the views of "people who felt the need to go", to "view a specific" movie, and "mark their opinion down". Yet people use those sorts of exit polls to judge a movie's popularity all the time. Simiarly, a political poll taken at a shopping mall is limited by the views of "people who felt the need to go" to that mall, at that time of day, to go to that "specific" store location where the poll was occurring, and "mark their opinion down".

The results of that poll get less useful the more you try to extend its findings to a broader sampling of people, but getting the opinion of 4,000 people in an audience provides an extremely compelling view of the opinion of that specific audience.

Stefan

More accurately, in this case you're getting people who not only saw the movie, but cared enough afterwards to go home and post a comment about it on a message board. Sorry, it's complete bunk.

[Erik]

The only people compelled to post a reader review are those who feel strongly in some way about the game


It's a poll taken of people who felt the need to go to the website, view a specific page relating to the game in question, and from there, decided to mark their own opinion down


More accurately, in this case you're getting people who not only saw the movie, but cared enough afterwards to go home and post a comment about it on a message board. Sorry, it's complete bunk.

So does this mean that anyone who takes the time to express an opinion is automatically suspect? That externalizing a viewpoint invalidates it? I can't argue that the reader reviews are statistically sound, but it seems equally disingenuous to reject them entirely as any sort of useful metric for guaging the general opinion of Gamespot's audience. For instance, using Doug's expressed on the Internet and therefore suspicious opinion that the reader reviews attract a bunch of obsessed boosters and a few token malcontents, we'd expect to see a lot of 10's and some 1s or 2s. Instead, the (written) review scores for Dungeon Siege are as follows: 7.1,7.5,8.0,8.3,8.3,8.5,9.8.

[asspennies]

So does this mean that anyone who takes the time to express an opinion is automatically suspect? That externalizing a viewpoint invalidates it? I can't argue that the reader reviews are statistically sound

That's my argument in its entirety. That the reader reviews statistics are not statistically sound. Once you start bandying around unscientifically gathered scores as some sort of telling statistic, you're already twisting the argument into fallacy.


we'd expect to see a lot of 10's and some 1s or 2s. Instead, the (written) review scores for Dungeon Siege are as follows: 7.1,7.5,8.0,8.3,8.3,8.5,9.8.

Well, you're certainly using a MUCH smaller sampling then the 4000+ Desslock was referring to. Yeah, out of 7 people who took the time to write a full review, it's not surprising that their scores would vary. But then you start dealing with the casual Dungeon Siege fan who likes the game so he goes to every website he can find and puts in a 10. I suspect there are quite a few of those, and more than a few vote more than once.

So no, I really don't trust any online, click-a-number-and-leave poll of any sort. It's nothing more than psuedo-fantasy statistics.

[Erik]

Doug Erickson wrote:


If you read a few message boards... most of them found the games I've listed to be spectacularly average

If you don't trust people simply because they sought out the Gamespot review in order to express their opinion, why the hell would you trust someone who seeks out a message board?

I don't want you to think I'm picking on you, because I like you in the manly I'm going to kill you last sort of way. That said, I don't know how you can be so disappointed with Warcraft 3, yet use Disciples 2 as an example of an under-reviewed game. Arguably, Warcraft 3 didn't revolutionize the genre, but D2 didn't even manage to innovate D1. D1 was an excellent game, but if there was ever an a case of a sequel primarily adding nothing but prettier graphics, D2 is it. At least W3 is a lot different than W2.

[wumpus]

For instance, using Doug's expressed on the Internet and therefore suspicious opinion that the reader reviews attract a bunch of obsessed boosters and a few token malcontents, we'd expect to see a lot of 10's and some 1s or 2s. Instead, the (written) review scores for Dungeon Siege are as follows: 7.1,7.5,8.0,8.3,8.3,8.5,9.8.
But the readers didn't assign the score "7.1" to the game. It's an average of five or six different sub-scores, right? So if I gave the gameplay in Dungeon Siege a "1" and the graphics in Dungeon Siege a "10", that means the game scores a 5.5. Awesome. Awesomely bogus, that is.

[Doug Erickson]

Yeah, but Disciples' mechanics, simplistic as they were, don't feel meaningless like the old-school RTS unit babysitting and peon management one finds in the Warcraft titles. There's no good reason NOT to fix these flaws, unless you're a smarter-than-thou Blizzard developer who thinks that limiting a bandbox selection to twelve units magickally balances play, all the while ignoring the fact that even in the beta, it was orcs uber alles. If the bearded ivory tower types at Microsoft could figure it out with AoE2, then I don't see why Blizzard can't uphold the status quo by giving us a decent interface and more context-sensitive clickery, at the very least. Or just let us bandbox all our units at once. Yes, that issue bugs the hell out of me. If I'm gonna play with Koreans, at least let me play like one and use the bandbox-mapclick approach as effectively as possible. Maintenance clickery is a fucking chore.

Disciples 2 is definitely not innovative - and I'm not arguing that it didn't get the score it deserved. That said, it did bring a lot of new features that Disciples fans wanted, including better combat and AI. If Disciples 2 played like old-school HoMM, you might be able to knock it on the same grounds as I'm knocking WC3, but it's also incorporated a lot of the good ideas that have been circulating in the tea parties that all turn-based gamers attend where they discuss Peter Frampton and PBEM rivalries. We get simultaneous turns, inmproved inventory management, persistent heroes, and better balance. D2 also added defense, better unit variety, better mission variety, a vastly improved combat interface, and a map editor - D2 took D1 and overhauled it into a playable modern game, all the while retaining what D1 fans liked (which would be "the art" and the weird Canadian-to-English story, I guess). I sure as hell wouldn't call it innovation, but they definitely took pains to advance and improve the game, which is why it scored well.

With WC3, we get the same interface we had with Starcraft along with the same micromanagement issues. I see NEW gameplay elements, like the broken "creep" factor and heroes, but I don't see BETTER gameplay elements, and with as much competiton as Blizzard has, there's no reason why they can't upgrade to match the status quo. Like I said, this isn't innovation; this is simply recognizing that other people have done things such as the all-important interface better and found superior solutions to problems that plague the genre. All of these non-WC3 RTS games I'm romancing took away Blizzard's good ideas and tossed the chaff; what's wrong with expecting Blizzard to learn, too?

That said, the two games aren't particularly comparable, since D2 has very little pointless busywork, thanks to its simplistic mechanics - it isn't subtle enough to have the sort of issues an RTS title might. I suppose if I had to manually feed my heroes, or give them marching lessons, I might have an analogue for the tedious resetting of control-number groups or multi-click spellcasting found in WC3. If D2 had those sort of features carried over from D1, you can sure as hell bet I'd be kvetching about it. But with WC3, there's no damn good reason for not abstracting the maintenance busywork out of the gameplay- hell, if a nancy lad like Garriot can figure out that it's not fun to feed the Avatar and Iolo, then Blizzard can damn well figure out that it isn't fun to click on your little hero, click to open a spell menu, and then try to click a moving target - regardless of how the nation of Korea might feel on the issue.

D2 improved D1. Some people might think that the WC style of play doesn't need improving, but when I see other developers implementing methods that would obviously improve the WC approach without compromising its style, I'm just not sold. Very few non-Final Fight games get it right the first time, especially complicated ones like RTS titles.

What were we arguing about, again?

[Doug Erickson]

Ah, yes.

In Erik/Desslock's defense, I *will* say that to some readers, scoring a game 8/10 might as well be calling it "average", since very few sites use the full scale effectively, although GS is better than most.

However, and I may just be saying this to get a dev spot on Majestic 2, I feel that the almost anyone who posts a reader review has an agenda. It might be as simple as an overwhelming, irrational urge to champion an underappreciated title (see: the joint marriage between me, Tom Chick, and WBC2), or as part of a backlash against know-it-all snob critics (see: me, Warcraft 3), or simply because they're 13 and its the first decent game they've played in ages. Like Jason said: they post because they have some sort of overriding reason to make that post. If everyone else who purchased Dungeon Siege shared that strong of a sentiment, you'd probably see quite a few more reviews. Hence, as I read it, saying that seven people felt strongly about DS is damning it with faint praise.

And, yes, quite a few folks love WC3. That doesn't make WC3 an instant 9/10 or better, at least from my friendless critic's perspective. I expect a more balanced perspective from a reviewer than I do the average Hyung Park Sixpack, whose last game purchase was probably Brood War and doesn't know what he's missing.

[Ron Dulin]

But the readers didn't assign the score "7.1" to the game. It's an average of five or six different sub-scores, right? So if I gave the gameplay in Dungeon Siege a "1" and the graphics in Dungeon Siege a "10", that means the game scores a 5.5.

if you're talking about the gamespot system, which is weighted (and explained on the rating page, I believe) that's incorrect. A 1 for gameplay and a 10 for graphics would, assuming these are the only two scores and they are weighted at approx. the same ratio as the GS scale, result in around a 3.5 or so.

[wumpus]

if you're talking about the gamespot system, which is weighted (and explained on the rating page, I believe) that's incorrect. A 1 for gameplay and a 10 for graphics would, assuming these are the only two scores and they are weighted at approx. the same ratio as the GS scale, result in around a 3.5 or so.
Not so. Check the WC3 user reviews. I found one that was 1/10/10/10/1 and the resulting score was 4.6. According to the description popup, only the reviewer's tilt and gameplay scores are "weighted", with no further description given. And those two happen to be the sections that this reviewer scored 1 and 1. Mathematically it should be 1 + 10 + 10 + 10 + 1 / 5 or 6.4. That's not much of a weighting.

Do you want to play a game that got the lowest possible score for gameplay and reviewer's tilt? I'm not sure I want to play a game that got a 5 on the gameplay scale-- and that could easily end up with a final score of 8 or 9.

[Qenan]

I hate RTS games. Even the best RTS game leaves me frustrated and frantic -- not what I call a good gaming experience. I'm secretly hoping (not secret anymore) that W3 will fail and the RTS genre will begin swirling in the bowl.

I mention this because all this discussion/argument about bias is very amusing -- but it appears most posters are overlooking the nature reviewing any art: intrinsic, subconscious bias will always exist.


Well, at least you have the right biases. :D

[Jason Becker]

"There's no good reason NOT to fix these flaws,"

Allot of people don't see those as flaws though. Maybe the peon managment in some ways, but not the combat units.

[Jason McCullough]

"There's no good reason NOT to fix these flaws,"

Allot of people don't see those as flaws though. Maybe the peon managment in some ways, but not the combat units.

Ok, then: what's fun about individually ordering hundreds of combat units around? I honestly can't see it.

[Murph]

Allot of people don't see those as flaws though. Maybe the peon managment in some ways, but not the combat units.

Personally, I kinda like peon management.