Having "no problem" and feeling like it accurately represents a true feeling of the review are two different things. One does not mean the other.Originally Posted by DKDArtagnan
So, you're saying Tom is quite ok with 3 = 60, even if it doesn't accurately represent "the true feeling of the review"?
Why would he be ok with that?
Also, how do you know these things? Is there a guide to the star rating I can check out somewhere?
Oh, here it is (found it):
** (2 stars)
I didn’t like it
So, Tom didn't actually like the game? It also seems to be a fully subjective score - with no interest in trying to be objective. Then the score makes sense.
Quality of response = 2/5.
;)
My own opinion is that reviews give visibility. "Everyone is talking about it" put the game on the list. If you are ignored, nobody will buy you. Once you are on the list, is your own merits that has to put you near the top, or the float to the bottom. It must depend largely IMHO in worth of mouth. Lots of negative reviews can paint a negative image for a game "Everyone is talking bad about it", so can probably sink a game. But a single negative review can't do that.
The solution to get better scores is to make better games, imho.
I unsubbed. I got my money's worth out of it, but I can't see spending money monthly when I haven't logged in in a week now. I just hit a point where it was no longer grabbing me, and I had little desire to continue the (essentially single-player) experience. It's an intriguing and in many ways very cool game, and I may return sometime in the future, but for now, so long and thanks for all the deep ones.
The problem you're having is you're thinking of the "standard" review scale as being from 1-100. The problem is, it isn't. Games are rated like an American grade school, that is 60% or below is a major failure of a game that doesn't even run half the time. A 70% from a typical review shop is a poor game. So if you need to take a math approach, the correct scale would be to convert Tom's out-of-five-stars review system to the standard 7-9 scale which would make a 2 star game be somewhere around a 75% or so. Which... is right where Metacritic is sitting right now.
All three.
Namely, when I checked Metacritic after seeing Funcom's writeup, there were only 5 reviews listed, yours being one. I didn't realize they had filtered out 40 others. Mea culpa.
I still think your review was unfair, even at the time and if you want to be listed on Metacritic you should, you know, use a Metacritic-compatible rating system (or the reverse, use your own system and be dropped from Metacritic). Regardless of the actual mathematic impact, your own intent, what someone would derive from actually reading the review, etc - by using a non-standard review scheme and having it plugged into Metacritic, you are on *their* record as saying TSW is as horrible a game as, say, Sword of the Stars 2, which until recently wouldn't actually run without crashing messily, is still trying to patch in actual gameplay a year after release, and has an aggregate rating four points higher than what you gave TSW.
Given that the game industry and Metacritic are now locked in a toxic embrace - it's a problem.
I do use a Metacritic-compatible ratings system. As I've said numerous times, I spoke specifically with the folks who run Metacritic and Gamerankings before they listed Quarter to Three. I made very clear what ratings system I would use. The people from both sites were very supportive. I'd hardly call that "incompatible".
A "non-standard review scheme" is a scheme that uses the entire scale? Uh, okay. I guess I'm non-standard. Personally, I wouldn't have it any other way.
I also can't believe you rolled out that old trope that "only buggy messes should be rated below 50%", or whatever you're trying to assert by dragging Sword of the Stars II into the equation. I can think of plenty of reasons to not like a game other than whether or not it's technically polished. You're doing a disservice to anyone who talks about games in a meaningful way when you try to limit criticism to technical issues like whether something crashes.
-Tom
It certainly sounds like a toxic embrace, one of several the industry is locked into simultaneously.
This Metacritic issue is symptomatic of the general immaturity and ineptness of both the industry itself and even more so the remora-like market analysis firms that circle it.
In this industry reviewer ratings are notoriously arbitrary and valueless, and there is an industry history of incompetence and bribery to boot. If Metacritic is really a principle metric for judging a company's success -- as opposed to, say, financials or the results of primary-source satisfaction surveys -- then there really is something seriously wrong with both the industry and the analysts.
I heard that this Tom Chick guy has the power to tank a game all by himself! He's like some kind of videogame Osiris, weighing the heart of games and tossing them into the devouring pit of Metacritic if they are found wanting.
So we do agree that, in Metacritic terms, you rated TSW the equivalent of (if you really had to put a value to it) a less than average 40% (which some here contested)?
The fact that other sites would use a 6 or 7 out of 10 to say the same thing is neither here nor there. As I mentioned earlier, you have other sites on Metacritic using the whole scale. So you have to stand by your whole scale ratings.
The 40% or 2 stars seems a bit harsh, but your review text made it clear what your gripes were at the time. You have also already discussed the fact you wouldn't necessarily mind a chance to update your review a few patches in. And Metacritic is the problem that it is.
But this whole discussion started with some people contesting what the 2 star rating even meant.
As for your review alone being enough to harm TSW sales and subs. Sales? Possibly maybe if we accept the hypothesis that your written word carries that much weight (in other words, I love your reviews but I don't think this is all your fault :) ). But subs from players who got to try the game for themselves? Definitely not.
Wendelius
Last edited by Wendelius; 08-13-2012 at 09:31 AM. Reason: Clarity. Grammar.
My hesitation about buying The Secret World had nothing to do with its 72 Metacritic score (or Tom Chick's review) and EVERYTHING to do with lessons learned buying MMOs at launch over the years. I decided that TOR (which I really enjoyed for a few months) was going to be the last subscription MMO that I paid $50 or $60 for at launch.
It just doesn't make sense to do so when time and again "premium" MMOs go free to play in a year. Why would I subsidize the game's development (and inevitable free to play customers) by paying for the client AND signup to pay $15 a month knowing that's the end game? The answer is: I won't.
I'm thinking that soft sales of The Secret World are just another vote in the referendum on the $15 per month MMO that's been taking place over the last couple of years. Honestly, I'm hoping that the $15 subscription model is over and done with. I'm not sure it ever really made sense from a consumer point of view, but I'm fairly confident its day is done. The accusation that Metacritic scores killed sales of The Secret World tells me that Funcom just isn't able to read the writing on the wall.
I thought metacritic was more of a place where you could get all the reviews, then CLICK AND READ THEM to understand why they gave what they did.. not to just see the value and decide "oh, that sucks".
The problem is something that Amiga Power alluded to all those years ago, where they decided to use the 0-100% scale... ALL OF IT. Even back then it was a problem that most game magazines gave 60% to something that was shit, and most games were around 75-85% (and too many above 90%). What is the point of using a scale that goes to 100 if you are not going to use all of it.
Edit:
Or maybe I am thinking about gamerankings... which is probably why I visited that a lot more often. Not sure I ever visited metacritic outside of the links from Steam.
The Secret World's issues came from a combination of factors. Mediocre scores, Conan's performance, broken quests at launch, $15 a month subscription, odd subject matter, and a lack of good marketing.
The idea that Tom's review somehow scuttled Funcom's budget is ridiculous.
That's exactly how I use it. It's easier to let them collect links to various reviews instead of jumping around sites myself. For that it's an excellent repository and tool. I've never found the Metacritic average number associated with a game to be very useful.
Same here... when I remember to use it. I tend to visit the sites I trust to look for reviews instead. But if Metacritic has any use for me, it's to help me find additional reviews to read.
Wendelius
You do realize it will be close to impossible for Funcom to create monthly content if they don't have a constant subscription inflow. So get rid of subscription rates, you get rid of the ability to create new monthly content.
Personally I have no problem paying $15 for new content each month.
For me the game is well worth the monthly sub. I bought the lifetime sub though and I don't regret it at all.
What they should do--and this thread is completely shot now--is to normalize the scores of each reviewer relative to their own score history, THEN aggregate them. That would remove the quirks of individual review sites. So if playmoargames.com gives an average score of 8 and Tom gives an average of 2, when they both scored a game at their average, it would still mean the same thing. But I don't see that actually happening, because it would be more work and confuse the hell out of people.
I agree that Metacritic is a great place to find reviews in one spot. I do make reference to the overall number sometimes if I'm in a hurry. So, if I'm in the store and see something cheap, I might check on there. If I see something in the 50s, I'm probably not picking it up.
Honestly, I find most reviews to be not that helpful. Reviewers tend to spend far too much time on whether they liked a game, and I don't care whether or not they liked it. I care whether or not *I* will like it, and that is seldom addressed. My favorite review scale is a 3 point Likert. 1=poor, avoid. 2=solid, definitely check this out if you enjoy other games of this type. 3=good/great, must for fans of the genre, good possibility for those with passing interest
Last edited by Misguided; 08-13-2012 at 10:19 AM.
I tend to play a new MMO for three months or less. I want to see the story content, but don't care about PvP or grinding up tokens to get Superman's special armor or whatever. I'd be fine paying for mission packs that I thought were worth buying and sounded interesting. What I'm not interesting in is paying $50 for a client that's inevitably going to be free and then paying $15 per month for the pleasure of accessing the content in that client.
It feels to me like MMO launches are the most cynically predatory in gaming. Forget Project $10 and all the kerfuffle about used game sales. MMO game companies take advantage of folks that are excited about a new MMO by extracting $50 or $60 from them up front and $15 a month for however long they play, knowing all the while that the client will be free in 12 to 18 months and that a free to play option will be implemented around the same time.
Let's just say that the content patches are worth $15 per month. That'd be fine if the game itself was made with fun as its top priority. Unfortunately, and this is patently obvious in every MMO I've ever played, the priority is on keeping the player playing for as long as possible so that as much money as possible can be extracted from that player. So things that are trivial and take little time in other games (like traveling) are big time sinks in MMOs. Want that new armor? In most SP games you play and find or buy armor. Sure, in MMOs you can do that same, but how much longer does that drag out? And for no other good reason than the MMO company wants another $15 per month. In my mind that's just no sustainable and I think the soft numbers on The Secret World and other MMOs are bearing that out.
Normalizing scores per source would be easy enough to do, but even more confusing. Really the problem is the importance placed upon the metascore, not the aggregate itself. But we've had that discussion many times before.
I think it should not only possible but strongly encouraged to update score for games that significantly change like MMOs. There should be a release score, a "3 month in" score, and then a re-evaluation with each major content patch or critical functionality patch.
Scores as snapshots of a point in time are a disservice to readers intending to use them as a buyer's guide.
If a game is buggy as hell on release, its score should drop right in the toilet, so consumers know not to buy it. If the developer fixes all those issues in a patch a month or two later, they should be rewarded for their good behavior with a re-evaluation.
Normalizing scores per source would be easy enough to do, but even more confusing. Really the problem is the importance placed upon the metascore, not the aggregate itself. But we've had that discussion many times before.
I think it should not only possible but strongly encouraged to update scores for games that significantly change, like MMOs. There should be a release score, a "3 month in" score, and then a re-evaluation with each major content patch or critical functionality patch.
Scores as snapshots of a point in time are a disservice to readers intending to use them as a buyer's guide.
If a game is buggy as hell on release, its score should drop right in the toilet, so consumers know not to buy it. If the developer fixes all those issues in a patch a month or two later, they should be rewarded for their good behavior with a re-evaluation.
This assumes that the Metacritic average has any inherent value at all and that any of that would be worth doing. These companies ascribe the Metacritic number so much more weight than it deserves in my mind. The Metacritic number is a simple average of the scores given to a particular game by many unrelated reviewers, each subjectively evaluating the product. I just don't understand why anyone gives a shit about it.
Does/will Funcom have enough subscription money on a continuing basis to provide new content in the amount and quality I want? I really doubt it does.
I'm doing what most people who enjoy a game but are bored with the current content do - stop paying for a few months and resub later if there's enough new stuff to keep them busy for another month at least. If Funcom ponies up some good new content I'll be happy to resubscribe, or even buy an expansion, but I'm not going to pay for a game I don't want to play.
There's nothing very social about TSW, it's got nothing to do but consume content, and unless I want to do the same stuff over and over over (I do not), there's no reason for me to stay subscribed while they make new stuff. If Funcom wants my subscription money they need to give me a reason to pay them.
This is really it in a nutshell.
If metacritic scored things the way Rottentomatoes scored film critics, you'd have game criticism actually incentivized to use the full ratings scale. In a case like The Secret World, a Rottentomatoes-like list of scores would have been more representative of the game's quality.
I agree too much weight is put on the Metacritic score, but the Metacritic score could be more useful and informative than it currently is. *shrug* the fact that it causes these sorts of stock fluctuations is kind of crazy.
Brilliant! I am inspired
The funny part about the stock price is that they're really just back to where they were 2 years ago. I'd say the climb into the 20s this year is the interesting and most likely irrational part of the graph.
I decided to buy after the free weekend. I'm sure this has all been said before, but I'm weirdly surprised how single-player minded it all is. I've been playing for about 25 hours, now and have seen exactly two group quests, both of which were instances. I hardly ever see players in groups larger than 3, and the local 'say' chat is dead silent, not to mention #qt3, unless I'm missing something. Despite all that, or maybe because of it, I'm having an absolute blast. The idea of using real world locations as a kind docking station for all these supernatural and horror tropes has made for a really excellent and atmospheric ride. I'm charmed to find that most of my motivation for continuing has little to do with gear, and a lot to do with getting to the next area or cut-scene. The writing has made me laugh a few times now, and while the voice work is on the spotty side, it's never been south of tolerable. I really hope this game can maintain this consistency after I'm out of Solomon Island.
Also I'll quickly say that blaming Tom for somehow throwing the Metacritic rating, among other things, rests on the assumption that the 7-9 'standard' is either correct, unchangeable, or both. I find that a bit sad.
Blame me, I voted Nader. :)
Posting Permissions
Reply With Quote