How should budget games be reviewed?

[TurinTur]

How about; instead of scores on titles you give them a.. value.
"this game plays like it was worth 15$"

That's the idea CLWheeljack mentioned above, and it has some supporters.

In fact sometimes I have used it informally, on forums. "this game is pretty decent, with nice moments but also have x y z flaws, buy it when the price falls, around 30$".

[Jon Shafer]

"As such?" Wait, how the first sentenced is linked to the second? There is no cause and effect there.

Reviews are always subjective, true. Which means you can put perfectly a score there. The number, the score, is also subjective, like the text.

Numbers can also be subjective, you know.

Fair enough. I should have noted that the real problem is that everyone's scoring scale is also subjective. See: all of the 7 to 9 jokes. There's almost no context provided when someone just slaps a "7" on a game. People spend as much time debating what a 7 should mean as they do the content of a review. The inability for everyone to agree on a single scoring metric gets in the way of having any meaningful discussion about the actual game/review. A simpler system like the old Crispy Gamer model dodges the semantic disagreement and immediately gets at what's important.

Just look at the review Tom put up for Lollipop Chainsaw yesterday. One of the first comments on the page:

reading the other reviews, I think your full of crap. With all the 9s this game is getting, I have to say that you are not only a piss poor reviewer, but a piss poor gamer as well.

The very next comment:

A 1 out of 5? Seriously? Even if you didn't like it to give it a 1 out of 5 is ludicrous. Does it have bad graphics, sound, and is unplayable? Then maybe. Otherwise STFU.

Then a few comments after that:

1 out of five? Oh wait the site's name is Quarter to Three nuff said.

You get the picture. The general fixation on what "score" a review gets completely pushes all other discussion aside.

- Jon

[TurinTur]

Well, it wouldn't be that different if there was only text.

If he put a 1/5 to the game, I suspect he greatly disliked it. Knowing Tom style, the text of the review surely won't be... nice. So without score, they would complain, and would complain of the same: of how Tom thinks the game sucks soundly and they think it's great. In the end it's all around that.

[Giaddon]

Without the score, Tom couldn't be aggregated on Metacritic, which I think I remember him saying he wanted to be.

[Telefrog]

Plus, scores are fun! They make people angry and that's always entertaining.

[TurinTur]

About reviews and price, in this review of Out of Eight of Sins Rebellion he put two scores:

I own previous Sins games: 4/8
I’ve never played Sins before: 7/8

In a way he is making reference to the price issue. The game is the same old Sins with a few new added features (subfactions, titans, victory conditions, a few new units).

For the people who are paying money for only the new features (previous Sins owners) is a 4/8, but for the people who are paying for the entire package (they didn't have nor have experienced the rest of the content) is a very good 7/8.

[djotefsoup]

I think in the age of expensive SSDs, they should give you an gameplay vs install size value, which is more or less the same thing as being expensive. And also how many billable hours of your time it may take up. Also they should write a paragraph on the important question of whether a a game flips back around to being good value because you won't risk getting bad grades from playing it too much, thus ruining an expensive education.

[Teiman]

Imho, reviews sould ignore the price thing until the last paragraph, where the reviewer can adjust his opinion about the game based on price. "Sure, is a multiplayer only game ,and only has 4 maps, and get repetitive very quick, but is only 10$ and you will sure have at least 20 hours of fun, and has this redeaming feature, so I say is a good deal". Or something like that. A very low price in a turd of game that have other redeamin features is still a "buy it".

Since most people don't pay for the videogames, talking too much about the price is like spam, or something.

[Pogue Mahone]

Plus, scores are fun! They make people angry and that's always entertaining.

And that's really the long and short of it for me. I could give a damn about a game's score, but I am having so much fun reading the comments attached to Tom's Lollipop Chainsaw review on the front page.

[Dan_Theman]

Yeah. 3/8's of a review score being based on price is, imho, a pretty bad way to score a game and leads to internal inconsistency (does that mean that a cruddy game which sells for $1 should at least get a 3?).

It's that level of granularity which I can't stand and I don't think reviewers should really have to hold themselves to. But they're expected to, and placing a number out there essentially stakes a claim to such an attempt. I take them all (Tom's included) with a grain of salt, but when I hear about publisher giving bonuses based on metacritic scores then my mind drifts to the annual thread here on how bad games journalism has become.

[scharmers]

After trolling through the Internet waste spew that is the Lollipop Chainsaw review comment section, I don't think that any games, budget or otherwise, should be reviewed at all. In fact, I think I am going to go back in time and inform my past self that the Internet now basically consists of nothing but the worst part of "Letters to the Editor" sections of niche enthusiast magazines content-wise.

[Telefrog]

And that's really the long and short of it for me. I could give a damn about a game's score, but I am having so much fun reading the comments attached to Tom's Lollipop Chainsaw review on the front page.

People really do get bent out of shape about low review scores. They take it personally which is weird because unless you're the developer or publisher the low score isn't hitting your paycheck. Why do you care so much?

[Jon Shafer]

Well, it wouldn't be that different if there was only text.

I disagree with your assertion here. Read some of those comments again. On the first page alone there's a half dozen commenters that only talk about the score. If it was just Tom ripping the game a large number of them would have just shrugged and gone away. As soon as you put a number on a game though, that becomes the story.

If he put a 1/5 to the game, I suspect he greatly disliked it. Knowing Tom style, the text of the review surely won't be... nice. So without score, they would complain, and would complain of the same: of how Tom thinks the game sucks soundly and they think it's great. In the end it's all around that.

No doubt, if you erased the number from the page there would definitely still be some people that stop by to complain and only say that Tom is dumb. But a much larger percent of the total comments would focus on what he actually said. And if folks come to argue that Tom's points instead of what number he gave the game that seems like a huge step up from what you see right now.

- Jon

[tincrow]

IMHO there's no real problem with the 7-9 scale, which really is more of a 6-10 scale with most scores falling from 7-9. Sure it's a little goofy but it's also well understood what each value means among gamers. I think anyone not using that scale is making metacritic a less useful tool, which perhaps is their goal :).

There are lots of decimal points between 7 and 9 if you want more specificity, just pretend 1-5 don't exist.

[tincrow]

In terms of games which come out at a lower price point, they barely even need reviews at all, just:
1. What are they all about
2. Are they worthless

From that you probably know at what price point you're willing to throw a couple of dollars and a couple of minutes into checking it out.

[Tyjenks]

The whole Lollipop Chainsaw ongoing tirade is awesome. I wish people would never stop posting their whiny comments about Tom's reviewing skillz and how people that do not agree with their take that he has made an "egregious" error belong to the bearded-twatted Cult of Tom members.

It reenforces the fact that no one reads the text before zooming to the score and then evaluates the text on what they have already predetermined is a good or bad review.

Scoring should be completely separate from price. I say reviewers just add, "This is the best $10 7-rated game I have ever played".

[Rod Humble]

Put me in with the "I dont like numbers" camp. If I want a number I can just go meta, look at user ratings if I care (and I usually dont).

What I want from a full review is a well thought out opinion that ideally will heighten my enjoyment of a game if I play it.

It seems to me there is a pretty big opportunity right now for a deeper level of video game reviews which (imho) are now pretty far behind boardgame reviews in terms of quality.

Someone like Calandale not only takes the time to play a boardgame and give a thoughtful review the whole affair is enjoyable and illuminating by itself. If you dont want to watch the playthrough just skip to the review but you know he has put in the time and you can see how he arrived at his conclusions.

http://www.youtube.com/watch?v=nnxPEaKU53M&feature=plcp

Of course, doing what he does, is hard and time consuming. But imho the result is well worth it.

[Shimarenda]

It's good that you're not, but other people are worried about money and how much they spend on games. As such, I think a mention of value is a service to readers, especially when it's far to one side of the bell curve. If a game costs $20 and it'll provides 100 hours of gameplay, that's something I want to know. Conversely, a 4 hour game that costs $60 is worth mentioning as well.

I didn't say "not;" I said "less." So I don't really disagree with the rest of what you wrote. It also helps that I don't buy a lot of games compared to a lot of internet people.

[Alstein]

People really do get bent out of shape about low review scores. They take it personally which is weird because unless you're the developer or publisher the low score isn't hitting your paycheck. Why do you care so much?

Low scores= people avoid a game= less chance of future content or a smaller MP community.

The 7-9 score makes sense to me, if I add a 0 to the end and assume it's a grading scale like your typical college course.

[Nesrie]

I am more forgiving with independent and lower priced games... forgiving in areas like graphics, for example, or a strong soundtrack. At any price, I am not especially forgiving with terrible UI or gameplay that isn't fun. A buggy mess is a buggy mess, any price point. Cost is a factor, but I don't know I could easily quantify it, so i don't expect reviewers to do that.

I appreciate it when a reviewer mentions they think content is a bit thin for the price point, or that DLC is priced too high. I've seen reviewers mention expansion packs were priced too high and had content at a more DLC level.

[SamS]

Low scores= people avoid a game= less chance of future content or a smaller MP community.

But people are actually talking about negative scores affecting developer's livelihoods.

Think of the children Tom, the children!

[jpinard]

Consider the scenario:
Diablo 3 (60$-ish?) review vs. Torchlight 2 review (20$-ish?).

This a thousand times over. There should be a different scale for small teams vs. mega-teams & money. Otherwise the inequity of scale can almost always bury smaller efforts.

What the dev's have done with Torchlight 2 is amazing compared with the gobs of cash thrown at Diablo 3 and if you scaled down per-person, per-dollar Torchlight 2 would blow away Diablo 3.

But as it will turn out, Diablo 3 will average a 9, while Torchlight 2 will average something south of that which (I believe) belittles Runic's efforts. I could make the same arguments for Dungeons of Dredmor, Legends of Grimrock etc. You cannot and should not use a straight scale and not compensate for budget or team size.

[Tom Chick]

Should I review a $30 hardcover copy of Fifty Shades of Grey differently from a $10 paperback copy? What about if I get it from the library for free? Should I review Avatar differently based on whether I saw it opening night, rented it from a Redbox machine, or just watched it as part of my monthly Netflix subscription? What if I watched a friend's copy over at his house?

Furthermore, a reviewer has no business assuming he knows the value of five, fifteen, or fifty dollars to someone else. If there's one thing more variable, subjective, and ultimately meaningless than a reviewer talking about "fun", it's a reviewer assuming he knows the value of someone else's money.

Price should never enter into the substance of a videogame review. A budget game should be reviewed no differently from any other game.

-Tom

[Thongsy]

Instead of a 1-10 review scale, somebody should do reviews where the score is how much the game is worth paying for.

The reviews on CAG do that. I think there is a five star system and he also has a recommended buy price too.

[Greybriar]

How should budget games be reviewed?

Like all other games.

[malkav11]

I think there's some argument to be made for factoring in the budget the game was made on, but there's really no need to factor in the price it's being sold for.

[tincrow]

Should I review a $30 hardcover copy of Fifty Shades of Grey differently from a $10 paperback copy? What about if I get it from the library for free? Should I review Avatar differently based on whether I saw it opening night, rented it from a Redbox machine, or just watched it as part of my monthly Netflix subscription? What if I watched a friend's copy over at his house?

Furthermore, a reviewer has no business assuming he knows the value of five, fifteen, or fifty dollars to someone else. If there's one thing more variable, subjective, and ultimately meaningless than a reviewer talking about "fun", it's a reviewer assuming he knows the value of someone else's money.

Price should never enter into the substance of a videogame review. A budget game should be reviewed no differently from any other game.

-Tom

I think it depends what kind of reviewer you want to be. If I'm buying a car, I'll read Consumer Digest, if I want car-based entertainment I'll watch Top Gear. I'd be sad to be without either of them. Movie reviewers have a luxury: every movie costs essentially the same amount, so they get to be two-in-one.

What is someone doing when they read reviews? Probably one of:
1. Making a purchasing decision
2. Looking for other interpretations and insights into a game they enjoy

Metacritic is aspiring to solve number 1. Entertaining reviewers work on number 2. I certainly wouldn't ask Jeremy Clarkson to recommend a car to me, but I sure watch his show.

[Nesrie]

Should I review a $30 hardcover copy of Fifty Shades of Grey differently from a $10 paperback copy? What about if I get it from the library for free? Should I review Avatar differently based on whether I saw it opening night, rented it from a Redbox machine, or just watched it as part of my monthly Netflix subscription? What if I watched a friend's copy over at his house?

Furthermore, a reviewer has no business assuming he knows the value of five, fifteen, or fifty dollars to someone else. If there's one thing more variable, subjective, and ultimately meaningless than a reviewer talking about "fun", it's a reviewer assuming he knows the value of someone else's money.

Price should never enter into the substance of a videogame review. A budget game should be reviewed no differently from any other game.

-Tom

While I understand what your trying to say, I think. There is nothing wrong with someone saying that a movie isn't bad but for the majority, it's probably worth a redbox rental instead of a price of a couple of movie tickets. They used to say the same thing for years about renting games vs. buying them... problem is the industry systematically destroyed renting games on some platforms and continues to do so on others.

The entertainment industry certainly is price sensitive when it comes to reviews... not so much a movies budget, but people ask all the time if 3D is "worth" it in this movie or the next. This isn't a unique issue to gaming.

[rezaf]

I think Brad Wardell wrote in the Warlock thread that full- and budget-price games should basically be reviewed/evaluated the same way, but a budget-game ought to be forgiven a lack of features. I think I can get behind that.
But basically, what Tom said - price shouldn't play a role, at least not for the game's score.

I mean, if price is a factor, why not time/$? For example, CoD games are usually raved by the gaming press, and they're exciting, action-packed thrillrides, but they also can be finished in one sitting of only a few hours - unless you're into multiplayer or replaying (with exactly the same story/progression), possibly on a higher difficulty - that's possibly $15/hour.
Strategy games or even old shooters will often be closer to $1/hour.
Why doesn't that play a role in the score?
_____
rezaf

[razarok]

I think Brad Wardell wrote in the Warlock thread that full- and budget-price games should basically be reviewed/evaluated the same way, but a budget-game ought to be forgiven a lack of features. I think I can get behind that.
But basically, what Tom said - price shouldn't play a role, at least not for the game's score.

I mean, if price is a factor, why not time/$? For example, CoD games are usually raved by the gaming press, and they're exciting, action-packed thrillrides, but they also can be finished in one sitting of only a few hours - unless you're into multiplayer or replaying (with exactly the same story/progression), possibly on a higher difficulty - that's possibly $15/hour.
Strategy games or even old shooters will often be closer to $1/hour.
Why doesn't that play a role in the score?
_____
rezaf

1. People bring up anectodal evidence in the form of "game x was only y hours long and was the best game I have ever paid. I would have easily paid 100$ for it".

2. People bring up the question of what constitutes gameplay. I personally really like (grindy) jrpgs, but some/many would not want to count the grind as actual gameplay for the $/hour calculation.