Hyped game + bad review = reduced clicks?

[Doug Erickson]

Asking the website crewmen here: does giving a hotly-anticipated game, such as a Nintendo marquee title, an "unexpected" review/score actually reduce views of your site? For example, if someone were to give, say, Metroid Prime, or the next GTA a low score, do you think that the amount of people visiting your site might decrease?

I had this discussion with Jason McCullough at lunch, suggesting that perhaps website editors score big releases higher and are more forgiving out of fear; that a backlash to Mario Sunshine getting a 4/5 or less might lead to reduced clicks and thus less revenue. Any honest editors want to admit to a fear of the will of the majority, or am I just being an ass?

Just wondering. Is credibility-to-clicks something that nags the web reviewer during the review process, or is purely a credibility thing?

[Mark Asher]

I have no idea. My guess is a review that goes against the trend gets more attention. Whether this translates into a change in traffic over time is anybody's guess.

[Anonymous]

It's tough to say. When it comes to website viewers there's more than a few different camps. The people who visit your site because they like it. The people who visit your site because they hate it. The people who might potentially visit your site by positive word of mouth. Those who will visit by negative word of mouth. The Hardcore crowd and the Casual crowd...etc..

Chances are neither of us are a member of all the groups listed above, so there is nooo way we can accurately gauge this sorta thing. From what I have noticed tho, negative reviews tend to only really piss off people who already own the game. People who don't own the game yet usually, and responsibly, file it as reference.

There is of course the whole snowball effect that comes with a bunch of people who own the game not liking the review, creating a bad feeling which some of the lesser involved groups pick up on. Soooo I dunno.

I figure it this way, why do people bother reading reviews after the fact? Validation of course, but it's still dumb to get upset. They made the purchase, chances are they like it or have fooled themselves into thinking so. Why can't they just leave the negative review alone?

WHY ARE PEOPLE DUMB?!?!?!?

[Anonymous]

I figure it this way, why do people bother reading reviews after the fact? Validation of course, but it's still dumb to get upset.

WHY ARE PEOPLE DUMB?!?!?!?

Because "most people" who "get mad" about reviews are kids.

[Anonymous]

DUMB kids!

[Doug Erickson]

Yeah, but I'll also bet that most of the clicks on a review site are kids, which begs the question: did Dungeon Siege get great reviews because the reviewer wanted to meet expectations and satisfy his ad-clicking audience, or did it get great reviews because the reviewer simply wanted a boost to his "credibility" by meeting these same expectations?

I've heard a few reviewers say to me, in person, that sometimes it's easier to give a flawed but hyped game a great score, especially after it has received high scores from other sites, just to avoid hate mail. Truth?

[Mark Asher]

I don't know if most clicks come from kids. Traffic tends to nosedive on the weekend, which makes me think most traffic comes from people surfing the web during work.

[balut]

I've heard a few reviewers say to me, in person, that sometimes it's easier to give a flawed but hyped game a great score, especially after it has received high scores from other sites, just to avoid hate mail. Truth?

Well I guess that would keep the idiots appeased. The site would have lost any credibility with the non-idiots, but since they're such a minority, that really wouldn't matter.

The problem is that most people are dumbasses. How else do you explain "Carrot-Top's" existence?

- Balut

[asspennies]

Didn't ....well, now I forget where it was, but Tom's review of Deus Ex - wasn't it so unpopular (and incidently, correct) that the site had to do damage control and write an unprecidented (I should hope) second review?

I wonder how many people who loved Deus Ex's slow psuedo-dungeon crawl swore off that site forever, all because of one bad review of a game they love?

I think the question has some merit. If you're looking for the most eyeballs, you try to appeal to the masses. And the masses are going to like most of the crap out there. So, they sell off a little journalistic integrity for some advertising dollars. It wouldn't be the first time.

[Anonymous]

Well, I can see how it would be easier to give the score the majority wants to see.

I guess we just need to remember that reviewers are people too, and are constantly doing this kind of stuff to seek a sort of acceptance or just to keep loud stupid people out of their hair.

You know tho, I remember one review I did for Sydney 2000 for Gaming Age. Eidos' PR actually e-mailed the site with:

"I saw your Sydney 2000 review :("

Tough to say if that was a joke or not, but is it right for PR to put guilt trips on reviewers like that? It wasn't like I cared that much, I can be sort of a jerk when it comes to bad games, and Sydney 2000 was a bad game.

Come to think of it, what was the last GOOD game based on the Olympics.

[chet]

A contrary to public opinion review will normally get mentioned more and drive more traffic. Fan boy sites will go crazy and put "mad links" to your review if it is negative.

We still get hunted down over some of our reviews from time to time when fanboys post in some obscure forum about their 5 year old game.

Chet

[Greg Kasavin]

I've tried to stop being ironic altogether, but it's difficult to respond to this thread in a straightforward manner. But I'm compelled to try so here goes: First of all, it's not like anyone would ever actually admit to acting out of "fear" in giving a game an inflated score, if in fact that's actually what happened. So Doug, if I were to flatly tell you that, no, this never actually occurs, there would be no reason for you to believe me, because for all you'd know I'm probably lying.

Secondly, and more importantly, pageviews no longer matter much to commercial sites. Pageviews and traffic don't pay the bills--they actually makes the bills go higher. There's no incentive for commerical sites to act purely in the interest of gaining more pageviews. It's a self-defeating battle that's killed off a lot of good sites over the years.

Third, as others have said, a polemical "off-kilter" review is more likely to generate more pageviews anyway--not that pageviews matter.

No one will deny the importance of credibility. And everyone would agree that being truthful and forthright is the ticket to gaining credibility. Overyhyping games or overinflating / deflating review scores doesn't help a publication's credibility. This is simple stuff.

Any decent reviewer doesn't go into an assignment with a lot of preconceived expectations, even if the assignment is a very highly anticipated game. Acknowledging that a game is highly anticipated isn't the same thing as expecting it to be good, and giving a highly anticipated game a high score doesn't automatically make you a sellout or a hack if the game itself is actually good.

My spider-senses tell me that this thread might venture into the topic of payola--the other explanation for why popular games get high scores. It's been weeks since that dead horse has been flogged here.

[asspennies]

I don't know if I'd blame payola so much as simply the apparent tendency for reviewers to buy the hype - no matter what the hype is. And it may be subconcious, as well.

The best recent example is the oft-discussed Black and White - no one today could argue that it deserved the majority of scores it got. Yet when it came out, the hype machine was rolling fast and furious, and many thought that if it plays like the best game ever for two hours, it must continue that way.

Was this sort of hype-following a cynical attempt to grab eyeballs? I'm not entirely sure, but certainly many sites had to be first to get the review out, accuracy be damned.

Will WarCraft 3 (which, for all its good points, certainly has faults) receive the same treatment? Some would say it already has. Would Daikatana have been reviewed as poorly as it was, and by as many publications, had it not had the "negative hype" it received? (Speaking as someone who actually played through Daikatana the entire way through, I certianly hope it would - the game was terrible.)

Maybe Greg is right about pageviews and their ilk not being the focus of websites anymore, but certainly there must be some advantage to having the review for the hot game (be it famous or infamous) first to get some sort of recognition. So the question remains - do some sites skim a few ounces off the proverbial quality bucket to make sure their review is the first in line?

[Jason McCullough]

Secondly, and more importantly, pageviews no longer matter much to commercial sites. Pageviews and traffic don't pay the bills--they actually makes the bills go higher. There's no incentive for commerical sites to act purely in the interest of gaining more pageviews. It's a self-defeating battle that's killed off a lot of good sites over the years.

Really. That's...completely insane. Is running a gaming website just completely unprofitable now?

[Tyjenks]

I am not sure if this factors into this discussion or not. It seems to me that every time there is a hyped game released, Gamespot and IGN come out with their reviews on the exact same day.

Maybe they have a gentleman's agreement to post 2 days after a game's release.

Maybe they do like in grade school and whisper to each other what they got for question number 34 and compare answers.

Maybe it is a total coincidence.

Maybe I am paranoid and imaginig things and the aliens are not going to invade in the year 2012.

Just wonderin' if anyone else had noticed.

[Doug Erickson]

Sorry, Greg, I'm not buying. I've written for a number of websites and magazines, and I know what a powerful force positive reader feedback is, especially to your average insecure reviewer.

That's assuming, of course, that the reviewer has played through the entire game and isn't building his review based on preview content, in which case regurgitating expectations is the safest route to take.

Like asspennies said, how are we, the readers, expected to explain away uniformly high ratings for Black and White, Turok 2, Dungeon Siege, Rogue Leader, and Metal Gear Solid 2? (the jury's still out on WC3, although MP has been around long enough for the problems to be apparent.) These games aren't necessarily bad, but they *are* demonstrably flawed in ways that are pretty damn self-evident. Shouldn't we at LEAST see a few breakout scores that call the games to task by these intellectually-honest reviewers? Every one of these games I've mentioned never makes any "Best Of" lists, and is often dismissed as mediocre or crap now that they've come and gone, although you wouldn't have guessed it by revisiting the reviews that appeared around their release.

They all have the following in common:

1. They had crazy levels of hype, supported with boatloads of fawning interviews and preview specials galore.
1a. Because they were made as part of a popular series, or by a popular developer.
1b. Or were part of a console launch library.

I don't buy the payola argument, unless we're talking about Adrenalin Vault. Payola cases are usually painfully obvious, and always involve cut-rate games who HAVE to have their reviews purchased. Peter Molyneaux doesn't have to buy anything; he's seen as a credible developer, with plenty of wide-eyed reviewers and critics willing to play the apologist role for a piece of his credibility pie.

You've told us you aren't guilty. So what's your explanation?

[Dave Long]

They probably hit the exact same day because they're both trying to get the review out day and date with the game's release or as close to it as humanly possible. That's the web for you. It's just a part of their MO.

It's hard to argue that it's bad because people want the review immediately if possible. Most readers probably also believe it should be there because, well, it's the web and it's an "I want it now" culture when it comes to the Internet because often you can get it now.

There's no great conspiracy of game reviewing. That much is certain. With so many freelancers and different internal reviewers at all these various publications, it'd be impossible for us all to get together and come to a conclusion. Payola wouldn't work either simply because no company could afford to pay us all, not even Microsoft.

Those of us that have written reviews all can ask you the same question time and time again... where's the proof? No one ever comes up with anything because it just isn't out there. We're not sitting around fixing scores and writing our reviews with the publishers looking over our shoulder. We're just sitting in a dirty, old computer room at a desk that probably looks a lot like your own making a judgement call on a game we got in the mail. It's really not rocket science. Some are better at defining their exceptions and accolades than others, but if someone's on the take, I've yet to meet him.

--Dave

[Anonymous]

Didn't ....well, now I forget where it was, but Tom's review of Deus Ex - wasn't it so unpopular (and incidently, correct) that the site had to do damage control and write an unprecidented (I should hope) second review?

No. Games Domain (www.gamesdomain.com) has done "second opinions" on games since it started back in, oh, 1995. Browse the site and see just how many "unprecedented second reviews" there are. It's a way for the site to review games from different perspectives. Some games with initially good reviews get bad second opinions, and vice versa.

[Doug Erickson]

I'm not implying a conspiracy. In fact, I don't believe anything of the sort.

Rather, I'm simply trying to figure out why hyped mediocre titles get incredibly high marks, and what little piece of reviewer psychology leads to that sort of result.

[Greg Kasavin]

You've told us you aren't guilty. So what's your explanation?

My explanation of what, the fact that I've given positive reviews to highly anticipated games? My reviews are my explanation--they stand for themselves. I've read your complaints about my Warcraft III review, and nevertheless feel that it holds up to intense scrutiny, same as all of my reviews. I put a lot of time into them up front so that I don't need to defend them later. This has proven to be a worthwhile strategy for me--I highly recommend it!

Black & White always comes up in these discussions, rightfully so. I should at least point out that it's not really a "recent example" as asspennies said, since it's a year-and-a-half old. I've acknowledged before that, of all the hundreds of reviews I've ever written, it's probably the one and only one that I wish I could go back and adjust... mostly just the high score I gave it rather than any of the points I wrote down. In hindsight I certainly don't think the game is "crap." But it probably deserved 8-range scores from people instead of 9-range scores. Do I regret recommending the game and possibly influencing people to buy it? No, I think Black & White was definitely an unusual experience and by all means worth playing. I felt a big part of the problem with the game was Lionhead's utter failure to produce a patch in a timely manner, something my review couldn't realistically have anticipated. Many players experienced technical issues that weren't resolved for months.

At any rate, it was a superficially impressive game that really caught some critics off guard. Maybe some of the critics bought into the hype (not Steve!), but I do think those that rated it highly were genuinely impressed with the game's originality and style. Yes, Black & White didn't end up having a ton of lasting value, so I think if the critics are to be faulted for any one reason, then it should be for the fact that most of them probably just didn't play it enough before reviewing it. But no, I don't think they were "afraid" to give it a low score. Go back and read the reviews and you'll see that people genuinely liked the game. They were charmed by it.

It's funny, because I've always felt that most all of Peter Molyneux's games give a great first impression but then taper off and get un-fun in a hurry, all of a sudden. Dungeon Keeper was very much that way for me--lots of novelty for the first 20 hours, and then all at once, I had the whole thing figured out and completely lost interest. While playing Black & White, I distinctly remember thinking, "No way is this game going to have that same problem." Oh well. My own experience with Black & White served as an effective reminder that I should always make double- or triple-sure that I've played the game I'm reviewing to the point of exhaustion.

[Anonymous]

how are we, the readers, expected to explain away uniformly high ratings for Black and White, Turok 2, Dungeon Siege, Rogue Leader, and Metal Gear Solid 2?

The thing is, the official review scores for these generally aren't too far out of sync with the reader scores.

(game: gamespot score/reader score)

Rogue Leader: 9.4/9.0
Metal Gear Solid: 9.6/9.1
Turok 2: 9.0/8.8
Dungeon Siege: 8.4/8.3
Black & White: 9.3/7.9

The biggest gap is for B&W, but even that's just a hair shy of "great" according to the polls. For a bigger reviewing travesty, Soldier of Fortune 2 received an 8.2 from Gamespot and a 6.0 from readers. I wrote the sof2 review, and I wasn't bought off and I can honestly say that my being scared only accounted for .4 of the overall score.

What's the point? Your guess is as good as mine. The gamespot readers aren't sheep, or else they would have backed me up a little on fucking sof2. I can only conclude that the reviewer's taste generally matched the taste of the readers in the examples you cited. Plus, though I hated MGS2 and would love to think that the scores were over-inflated, it received game of the year nominations all over the place.

[Doug Erickson]

Most of those so-called reader reviews are written the moment the review is posted, and are designed as nothing more than deliberate fodder in the console wars (espcially the Turok 2 and Rogue Leader reviews). If you actually read the reviews, you find perhaps one or two that are thoughtful and reflect some degree of playtime as well as gaming experience - the rest are all hyperbolic crap designed to boost the average. Besides, how many of the reviews written well after the game's release are in the upper range? Usually, none - most are far more thoughtful and often excuse themselves with "after all the hype..."

If you read a few message boards, and talk to folks who play games, most of them found the games I've listed to be spectacularly average, if not bad (Turok 2 is downright bad, period; the level design is criminal, and the PC version actually netted the title reviews that reflected such). I'd like to know why the disjoint between "early bird" review scores and the opinions post-hype for most of these titles is so, well, obvious.

[Jason McCullough]

My initial reaction - Greg, you think B&W was worthy of an 8? Has grade inflation gone that far?

[Dr Fear]

I'm not implying a conspiracy. In fact, I don't believe anything of the sort.

Rather, I'm simply trying to figure out why hyped mediocre titles get incredibly high marks, and what little piece of reviewer psychology leads to that sort of result.

Maybe the part of reviewer psychology that gets people excited about games, since as far as I know all reviewers are people except Steve Baumann. If you're excited about a game, you're probably going to like it better than some game you're not that excited about, at least if it doesn't completely suck. And since most big games on major sites and in the magazines are reviewed by people who are big fans of that game's genre, they're probably even more excited about the game than the average gamer. Because in the end, everybody writing these things is a gamer and is a human.

I don't know. Maybe you could argue that it is totally the opposite - that these "big fans" of the genre should be more demanding, and take more convincing than your average player. So that would be the exact opposite of the previous psychological profile. I am not a psychologist. And unless you do some kind of study with surveys and psychiatrists, you aren't going to get much of an answer, especially on a messageboard. Who cares, anyway? Should there be a game reviewing institute where only old men who hate games are kept away from all they hype so that they can be free to issue their hyper-critical reviews because they're always constipated? They are game reviews. jesus.

[Anonymous]

My initial reaction - Greg, you think B&W was worthy of an 8? Has grade inflation gone that far?

Yeah, well, why don't you just go head over to Quartertothree.com with all the other curmudgeons. Oh, wait.

[Greg Kasavin]

That was me; forgot to log in.

[Tyjenks]

Who cares, anyway?......They are game reviews. jesus.

I think the point is that someone was interested in this topic and started a thread. Others also found it interesting and added to the debate.

Unless in your Bizarro world the "Games" section of message boards are strictly for talk of urinary tract infections and diseases of the bladder, this "games" section of this discussion board is for talk of games, issues relating to games, and diseases of the bladder.

[Doug Erickson]

So why is it that great, UNHYPED games get these "balanced" reviews/scores - like Age of Wonders 2 and Disciples 2 with their apparent B&W-level quality play, according to Greg's revised score - but hyped games always get the benefit of the doubt? According to Doctor Fear, it's also because reviewers love games.

If I posit that both AoW2 and D2 might've received at least an extra .5 added to their scores if they'd had twice the previews and "Blizzard" on the box (along with the legion of attendant fans and their trigger-happy email urges), would anyone disagree?

(Not saying I disagree with the Disciples 2 and AoW2 scores, BTW, although they're both lightyears beyond B&W.)

If you love games so much you can let excitement add at least two points to a review score, then I guess I just can't compete in this little hippy commune. Me, I still think it's some sort of intellectual dishonesty - a desire to meet expectations and play it safe with the noisier internet element than actually call a spade a spade. Or a fatty a fatty - whatever gets Chet on my side.

[routlaw]

I only did reviwing for a short period of time (the demands of the real world and all of its horrors caught up to me), but I wanted to comment on a couple of points brought up in this thread, and then point to how they (possibly) might interrelate.

Jason's got a good point-much of the time the big-name games, no matter how poor they are when released, seem to be immune to getting average grades with reviews that gloss over the apparent flaws to back up the score. Certainly there's an argument to be made that big-name games generally are big-name for a reason-bigger budgets mean better art and graphics, longer development times lead to more polish, etc. Discouting that, there is still a large discrepancy between the scores given to a mediocre lower-cost game and a mediocre big-budget title.

If the game is medicore, I don't see why the improved production values warrant the 10-15% increase in the final score given in the review. Not surprisingly, that relatively small differential is located in the key 6/10-7/10 range, the psychological point at which a game goes from 'respectable, but flawed' to 'not ever worth thinking about again'. I know in high school that my friends who got a 6/10 or a 7/10 on their AP Lit essays felt much different about their scores-both were in the end middling, but the 6/10 barely passed and the 7/10 owner could wipe the blood of his lips and fight on in the next essay. It's a big difference, and I don't know if reviewers take it into account when scoring their games.

As for influences on reviewers, I'd say that falls into two camps: the fansite reviewer and the paid, commerical website reviewer. The former has a drive to keep in favor with the big publishers so that he can keep getting free swag and kudos from the fans that read his/her site, and the latter has a drive to keep in favor with his coworkers in the gaming industry, both in the publishing business and in the game development business. The paid reviewer working for a website has a lot of time working with people inside the industry, and working with people inside the industry. They see the games created from pre-annoucement to release, and talk to (or know someone on staff who talks to) the people working on it. I don't think it's a stretch to say that these relationships give the big-profile games a bit more leeway, both in criticism and reviews, than the games they get mailed by the lower-teir publisher, and then unceremoniously dropped on the co-op or newblood in the office. It only gets worse as you start to cloud the deal with big-name studios like Westwood and Blizzard, who seem above reproach based solely upon their past successes.

I realise that no real critical analysis on games can be done in a clean room, and that there will always be some external factors outside the quality of the game that will influence the reviewer. However, when there's such a lack of consistency in the reviews of the hype have's and have nots, you've got to ask if that room isn't more dirty than it is clean.

[Tyjenks]

If you love games so much you can let excitement add at least two points to a review score, then I guess I just can't compete in this little hippy commune. Me, I still think it's some sort of intellectual dishonesty - a desire to meet expectations and play it safe with the noisier internet element than actually call a spade a spade. Or a fatty a fatty - whatever gets Chet on my side.

I had not thought on this subject much until my much anticipated Heroes IV came out. A wonderful game which I am still playing that has many, many faults, unfortunately. I, of course, read all the reviews to see what folks were saying and waited a week to purchase the game. Many noted a few problems, but otherwise they loved it.

Jonah Falcon's review for one, a day or two before the game's release, gave it a ringing 5 out of 5 endorsement. Upon further investigation, I find what a rabid Heroes fan he is. Then, as some here can no doubt attest, when discussing games he cherishes on the CGOnline boards, he often seems a little, how shall I say, all over the place and hair triggered when it comes to his discussions/arguements. I cannot trust a reviewer who seems so totally biased. How can a game get a perfect score when there are soooo many valid and substantiated complaints?

There are other examples too, but this one just stood out.

I used to think I could read maybe 6 or 8 reviews from trusted and semi trusted sources and get a general opinion about a game. I am beginning, finally, to realize much can get glossed over or ignored. I believe some reviewers are just as guilty by omission as they are of, possibly, inflating a score ever so slightly.