A Review of Video Game Reviews
Saturday, September 8th, 2007 at 1:15pm by Staff
Sykil from the Infendo Forums shares his review of video game reviews:
The Wii is quite the case study. More than any system before it, people are conflicted by how to approach reviews of its games. Games that are fun to play either get wildly mixed or universally moderate reviews based on elements that are specific to Wii hardware. I think these problems have been present in our review system for ages; the Wii has just brought them to our attention.
Part of the problem with numbered review systems is that reviewers sometimes use it to prioritize all gamer interest rather than to objectively quantify the quality of a game with respect to its genre and outstanding limitations. A ten is more worthy of your money than a seven, and RPG-centric and “hardcore” games tend to operate on the higher end of the scale (so long as they don’t screw anything up) than equally well-produced games of “lower-class” genres, such as sports, kart racing, and brain games.
There’s the snag; not everyone enjoys RPGs or RPG-centric games. Zelda’s courtesy ten means nothing to gamers who play sports games exclusively, and there are many gamers to whom that applies. As a result, that gamer feels shafted by the gaming media, who usually limit sports games to the 7.5-8.5 range regardless of their merits as sports games. The idea of a courtesy score isn’t limited to the 9.5-10 for Final Fantasy or Zelda: it extends to the “lower-class” genres that seem stymied in lukewarm water. Most reviewers obviously don’t care for them, but they throw those fans a bone anyway by choosing a moderate review score.
People know what games interest them, so they get huffy when their new favorite game is only high for that genre and not high for that scale. Games without definite genres fall victim to the same condition. A recent and vivid example is Elebits. What is it? … Exactly. There’s no basis for comparison, so reviewers assign it a lukewarm, generic, nondescript 7.5. Many have raved about the game, complaining that it deserves an 8.0+. Supposing the game did receive an 8.0+, a set of “hardcore” gamers would complain about it encroaching upon the territory of vastly superior games that adhere to classical gaming traditions. In this case, numerical systems of reviewing can’t win for losing.
What, then, is the solution? If gamers are perfectly capable of prioritizing their own interest, how should reviewers assess games? Show them what they need to know: who is it for, and is it worth buying? Genres are not good enough when it comes to telling who a game is for, so these things to convey themselves within the review. Now we arrive at the fault of review readers: most of the time, they don’t read the review, but that’s often because they’re distracted by the score itself. Without a score, and with a more concise review, I think there would be a much better response to gaming journalism.
This is palpable in the blogging trend, and I think most people have a positive opinion of gaming blogs as compared to professional gaming media. There are various reasons for this, but one of them, I think, is that blog items are usually conveyed in blurbs. It’s more personable, and their assessments of games seem more honest, conveying the merits and misses of the game without imposing a score upon it.
Some sites have used a similar system to review Virtual Console releases because a numerical value would inevitably be compared to their reviews of current games despite the fact that they’re obviously reviewed according to very different criteria. Time is a big enemy of the numerical system because scores don’t seem to mean as much after the game’s been out for ten years, or perhaps the game has taken on a new perception. That’s what people want to know when they prepare to throw some cash at a VC game: is it still fun (was it ever fun?), and is it still worth my money? I don’t see why it can’t be the same for new releases.
Now, we know what a game is. We also know that all games are different (more or less). Let’s make a parallel. We know what a car is. We also know that all cars are different (more or less): a Toyota Prius, a Ford F-350, and a Dodge Caravan. Would it make sense to simply assign numerical values to these vehicles? The Prius is a 9, say, because it’s got some sense of style, looks modern, and has nice features. The F-350 is an 8 because it’s an okay-looking workhorse. The Caravan is a 7 because it’s dead-beat ugly and cheap.
No, it makes no sense it all. They’re very different vehicles for very different people. Of course, these scores would be accompanied by an article explaining all of these things, but the score is still operating across all fronts, suggesting that we should all be driving Priuses. Why, then, is this system of reviewing so ubiquitous in our reviews of all entertainment mediums?
I give this system of reviewing a 2/10.*
* Kidding.
September 8th, 2007 at 2:04 pm
Dude, you are totally right, as an example, graphics mean nothing in Wii Sports since what you get is perfect for that kind of game, why should someone lower the score based on that aspect when the game is insanely fun? I myself don´t trust review scores anymore.
As a tip, maybe some of you´ve heard of wwwthewiire.com? Their review system is Wii crafted, and IMHO, in most case those reviews are really spot-on.
September 8th, 2007 at 2:17 pm
I’ve heard of it, but I never took the time to visit it. Now that I have, I think I like it. They have a very interesting review system.
As for Wii Sports, yeah, there were tons of things wrong with its reviews. I think it was another case where they didn’t know how to treat its score, so they just threw it in lukewarm territory: 7.5. Turns out it ended up being the most fun game of 2006.
September 8th, 2007 at 2:28 pm
the most unfair wii reviewer is EGM and gameinformer, oops did i say that? its true. read the reviews for mario strikers then check em out on nintendo power and gamepro. i usally hate gamepro but i liked thier review.
September 8th, 2007 at 3:41 pm
The review system could be better if they had experts rank them from genre to genre.
Like say, for example, one person reviews all sports games. The ratings for those sports games don’t apply to the other genre.
That could work in theory.
September 8th, 2007 at 4:04 pm
It could work for games that are faithful to those genres, yes. In fact, I was thinking about that when I was writing my review. The IGN Nintendo Wii team is comprised of two people: Matt Casamassina and Mark Bozon. Not to knock them, but they can’t be into everything, right?
September 8th, 2007 at 4:04 pm
I disagree. Scores give reviews context among other games. You want reviews to say if a game is worth our cash. But which game is worthier of our cash? We still need to rank them. Does changing to an ordinal system really improve anything?
Also, it sounds like the biggest problem with the rating system is that people complain about specific ratings. I’m OK with that.
You think wii sports was the most fun in 2006. It bores me after 20 minutes. The score worked for me. But Max Payne 2 was fantastic, while Fable was a letdown. Metacritic scores them the same.
I’m OK with having different opinions than the masses. Reviews are not empirical fact. People who get upset have forgotten this.
September 8th, 2007 at 4:08 pm
Nice review. I give it a 8.0 =p
September 8th, 2007 at 4:09 pm
Yeah, just as music magazines don’t have punk rock fanatics reviewing the latest recording of a Russian tone poem, maybe game review magazines and web sites shouldn’t have “hardcore” shooter and RPG fanatics reviewing puzzle games. I think the Wii and DS have actually improved this somewhat by demonstrating that there are enough people who want to play something besides shooters, RPGs and sports games that they need to have reviewers whose minds are a little more open.
But to keep the music magazine analogy going, there are many magazines who don’t assign numeric or letter scores to the music they review. I gravitate towards the ones who do, because I can incorporate their opinion more easily into a survey of all the reviews of a given CD. (That, and the no-scoring magazines’ reviewers tend to be frustrated writers or something; they often don’t even get around to telling you what they thought of the tunes.)
And I would do the same with gaming; in fact, I already do, by visiting gamerankings and similar sites. I think most people would do the same, and that’s why quantified ratings are here to stay.
September 8th, 2007 at 5:11 pm
deepthought: I agree that thumbs up/thumbs down-esque system doesn’t give much context among other games, as you say, so games that deserve the extra recognition might not get it in the overt manner that a score provides. However, these things should come across within the review itself.
I don’t think the biggest problem with our ratings system is that people complain about ratings. I think the biggest problem is that certain types of games get pigeonholed into certain score territory.
Of course, switching to a non-score-based rating system won’t stop review sites like 1up from writing incredibly ignorant and useless reviews.
raindog: On your first point, yeah, that would work, but what of Elebits? I go about the same process when looking into games and music. With music, though, there’s less risk because music is easily acquired and/or tested to see if one likes it. Gaming’s different. Not all games are available to test at a kiosk, and games are much more expensive. That puts a great responsibility on reviewers, and I think that readers are too prone to take ratings on face value alone.
September 8th, 2007 at 5:15 pm
I think the compromise is to use a point system with minimal increments not based on a 10 point (100%) scale, i.e. a max of five stars for a great game.
Five out of five stars denotes a very good game without the pretense of a prefect 10 (even though mathematically it’s the same thing).
September 8th, 2007 at 6:19 pm
I like the 4-star system that some movie reviewers use. It gives enough space for giving special merit to outstanding games while keeping things pretty fair otherwise.
September 8th, 2007 at 8:19 pm
Yeah, I’m running into an increasing number of blog articles criticizing the typical review format, and I tend to have criticisms of my own as well. I think it really says a lot that people tend to just skip the entire text of the review and just go straight to the numerical score in the overwhelming majority of situations. On one hand, it makes you wonder if there’s any point to the text if no one’s going to read it, on the other hand, it makes you want to strip out the scores to force people to read the text before forming an opinion. However, magazines have tried leaving out scores, and they always just end up with a bunch of hate mail and dropping sales because of it.
I think part of the problem is that the content of the review text is often dubious. Too much time is wasted on mere description of the game which could be gleaned from the back of the packaging. Then you tend to have entire paragraphs droning on in praise of the visuals, audio , and other technical aspects, when everyone just wants the writer to cut to the chase and tell them whether the game is fun to play. Opinion, analysis and discussion always seems to be short in supply and relegated only to the concluding paragraphs of the story. So really, who can be blamed for skipping the text and looking for the bold numbers, when the text mostly comes across as filler?
I also feel that the 10 point system is a huge waste in most cases. Few games ever go below a 7…even a 6 is fairly rare. Anything below tends to be relegated to the “Barbie Adventure” fare which is usually considered not even worth including in the first place. Even if it is included, readers will probably disregard the game unless it is a big name. As such, I feel a 5 point scale would be more telling.
Then there’s the whole “courtesy review” thing…if every Zelda and Final Fantasy is going to get a good review based on its name alone, then what’s the point? Furthermore, history has shown that these games can still be hit-or-miss…after all there has been a detracting faction that hated and criticized almost every Final Fantasy game since VII - VIII was too “emo”, IX was too “kiddy”, X-2 was fan service…etc. Similar criticisms have wrung off among fans for titles like Mario Sunshine and Zelda Windwaker, and yet judging by the reviews and review scores found in magazines you would have no inkling that anyone ever thought each of these games was ever less than perfect.
September 8th, 2007 at 8:51 pm
Personally speaking, the greatest way to judge a game is based on the body of review text. If the guy or gal is talking about how fun it is, how much they enjoyed playing it or how great the game feels to play then pick it up. This goes double if you like the genre.
In these cases, where an entirely positive written review is followed by a score of 7.0 or 8.0 they are usually marking down on presentation \ graphics, if it is similar to the last iteration of the franchise and writer personal preference (ie. I cant give it a nine even though I had loads of fun playing it because its just a silly non-game, its not hardcore enough to get a nine).
The best modern example of this is Mario Strikers, a really fun game and definitely worth your cash if you like football games or the mario sports games. However it gets an 8.0 everywhere because it doesnt have the greatest graphics of all time, it’s not as hardcore as fifa or pro-evo and the online isn’t as seamless as live.
Only one of those complaints is valid, and I dont care about online play as I generally play multiplayer with friends in the same room. So for myself I would pick that game up.
If I didn’t use this technique i would have missed out on classics such as Fallout, ICO, REZ, Viewtiful Joe, Astroboy Omega Factor, Phoenix Wright 1 & 2, Gradius V and Ikaruga. All of which were not heralded as the next big thingand given in the 7.0 - 8.5ish range, which most people just discount.
So people read those reviews that’s attached to the score, please I emplore you. And dont discount something with a 7.0, you never know you may be pleasantly surprised.
September 9th, 2007 at 1:15 am
A lot of your points are pretty spot on… however, the car example highlights a certain flaw in your thinking. Cars *are* rated that way quite often, and indeed very usefully, but they’re rated that way on several different criteria, and more importantly, they’re rated on those criteria *individually*. For instance, if you look at the safety rating on a bunch of cars, you’ll get a numeric value for them, and they’re comparable directly. You can do similar things for comfort, features, reliability, and what not. The problem only comes in when you take all of those numbers, average them (even using a weighted average) and give a final number WITHOUT giving the individual numbers as well. As long as you can see the specifics, most of the problems you mention go away.
So using Wii Sports as an example, breaking it into something like, Gameplay, Graphics, Sound, and Value for Money, you get very high values in the first and last, and middling numbers in the Graphics/Sound ones. This lets people prioritize their own wants and needs into the game. You obviously wouldn’t recommend Wii Sports to someone looking for some mindblowing audio-visual experience… but if you want good fun gameplay for what’s effectively a free game, there’s little that’s better. You can throw in further qualifiers, like Online Play, Length of Average Play, Learning Curve, and whatnot, as non-numeric values to further broaden the reader’s ability to decide if the game is for them. Ultimately though, throwing out the numbers isn’t as helpful as it could be. Providing readers the ability to sift out review scores in relation to what *they* find important is extremely useful however. But honestly, it’s also very important that you’re able to easily, quickly, and ‘objectively’ compare titles, especially ones that are in the same genre. (Before you complain, objectively is in quotes for a reason. :P)
But yeah, most reviews nowadays are extremely arbirtary… They’re not really developed on any sort of structured criteria. The single number you get at the end is some sort of bizarre average of various unmentioned criteria that’s weighted in the ways that the reviewer prefers, and thus the reader is left out of the equation entirely. As the industry begins to fragement more and more as the market grows, bringing in more varied tastes and styles, reviews and reviewers are going to become increasingly out of touch with the average reader unless this kind of thing changes. To a degree that can’t be helped. Some people are just not going to find the same things fun… But if we’re able to sort of have a better idea as to where the reviewer is placing emphasis, we can get a better idea of whether or not the game is for us as readers.
September 9th, 2007 at 1:52 am
Blake is right. A 4 or 5 star scale would be perfect. It will let you know what the reviewer thought of the quality of the title and is vague enough that you really have to read the review to find out what the game is like.
its worked for movies for decades, it would work for games too. There is no such thing as a “perfect” game, so using a 10 just doesn’t work. 4 or 5 stars would mean great, thats the best any game should get, it would stop a lot of fighting.
September 9th, 2007 at 7:10 am
There’s nothing wrong with using a numerical system for rating games. I think the problem comes in to play with the reality that most of these reviewers aren’t really able, for whatever reason, to:
1) Review “casual” games for the New Gamer audience. Games like Bust-A-Move and Marble Mania got rather low scores, but I know a few of these so-called “casual” gamers who think these simple games are fantastic. Hell, my mom called me last week at 2am because she beat Bust-A-Move (it was the first time she’s ever beaten a game. She was rather proud).
or
They can’t seem to properly rate a Wii game without drawing parallels to things like graphics on the 360 pr PS3.
This bias in particular stands out, I think, because in Wii games where the graphics ARE good, we usually have the qualifier, “For a Wii game” thrown in.
“Metroid’s graphics are great for a Wii game”. “Sonic has great graphics – for a Wii game – but…”
A game like Bust-A-Move or Tetris is a perfect example, though, of what’s wrong with the rating system. Bust-A-Move is a great game. EVERYBODY likes it. But new editions get scores in the 5-6 range, despite the fact that it’s still a damn fun game.
It shows that more often than not, reviewers aren’t rating games based on their merits.
September 9th, 2007 at 4:30 pm
MAn, i have another great example of a game who got screwed lately by th ereviewers, Mario Party 8… You go and play that with 3 other friends and then come back and tell me it`s not the funniest game you have`played in a long time. We LOL`D all the time.
September 9th, 2007 at 5:49 pm
I think you’re missing something. The reason that casual games, sports games, and etc. generally receive lower scores is because they deserve them. You can’t argue that casual games be judged only on how fun and addictive they are, any more than you can judge an RPG only on it’s graphics. You argue that games should be reviewed according to their genre, but in reality all video games are in the same category, because they are all video games. It’s okay for there to be low scores. You list Wii Sports as an example of bias in game review, but why? Yes, it’s fun, but it has almost no depth, and the graphics are very underwhelming. And also, don’t pretend like it was a free game: in Japan it’s a stand-alone game that costs $50, they just bundled it with the hardware in America and bumped the price of the Wii up to $250.
It’s perfectly fine to love a game that scored a 7 in reviews. What’s not okay is to say that it deserves a higher score because it wasn’t the reviewer’s favorite type of game. There are some exceptions, but in most cases the kind of bias that you reference doesn’t exist. A good game reviewer doesn’t give higher scores to a specific genre.
September 10th, 2007 at 1:59 am
Wii Sports, like in-person multiplayer games dating back to Pong, has legs that something like Bioshock (or Zelda, or Metroid Prime) simply doesn’t. After you beat Bioshock, you’re not gonna want to play it for a while. You can’t beat Wii Sports at all, just keep improving yourself. To anyone who started playing games before the NES, that’s awesome.
Sykil, there will always be genre-busting games just as there’ll be genre-busting music. I like a lot of that stuff myself. Of course, genre-busting music is usually also indie music and therefore only gets reviewed by indie magazines, who (a) have reviewers with open minds and (b) often don’t quantify their reviews and so they’re filled with wanking as I mentioned in my first post. We don’t have too many indie game magazines, if any, and of course Elebits wasn’t an indie game anyway.
September 10th, 2007 at 9:23 am
Vibe, on reflection I kinda agree with you that video games should be treated as a single category, but you failed miserably to really understand the rammifications of that statement. You’re arguing against the logical conclusion of discarding genre specifics when you say that judging ‘casual’ games solely on ‘fun’ is somehow broken. (I should also note that this is how Nintendo seems to be viewing things in regards to their rejection of the ‘hardcore’ vs ‘casual’ debate, prefering simply to view gamers as a single whole. Looking at where this goes and Nintendo’s focus lately, this does seem to make an even greater amount of sense than it did before.)
If you want to make comments about games as a whole, without looking into specifics and genre, then you must, in order to remain consistent, view them ALL solely on the merits of how ‘fun’ they are. Otherwise you inherently place a higher value on a specific aspect of gaming and apply a specific genre convention to the whole. Doing so immediately invalidates your review as catering to a specific type of player. And while this may seem to be a bit of a contradiction, since you’re effectively placing a higher value on the aspect of ‘fun’, it’s really not. All games are meant to be played. All games are meant to be enjoyed in some fashion or other. If you treat all video games as being the same beast, if you attempt to remove genre specifics, or try to make direct comparisons between ‘casual’ and ‘traditional’ games, you cannot use any other frame of reference. It’s the only thing they all share, it’s the lowest common denomenator of gaming so to speak.
That’s not to say that those other areas like graphics don’t have an impact on the ‘fun’… If the graphics help in making the game ‘fun’ then that’s fine, but just because they’re there doesn’t mean they’re actually doing that. A polished turd is still a turd, and all that. And if the graphics detract from the ability for the game to be fun, then that’s something to worry about too. But it doesn’t so much matter what they look like relative to each other, but rather that they *work* for the given game. Context, and not the graphics themselves, is what is important… And because context is so important, the concept of using a single category for video games, while still catering to anything other than ‘fun’ is inherently contradictory.
In otherwords, you can only view concepts like “Graphics”, or “Depth”, in how they impact the entertainment value of the game. This is because graphics and depth that make sense in a specific context do not necessarily make sense in a different one. Because of this they can’t be compared directly. To use a really simple example, you don’t want a verison of Sudoku that uses a lot of graphical embellishment, because it would get in the way and detract from what you actually want to be doing, which is the puzzle. Similarly you don’t want to have the depth and complexity of, say, Europa Universalis in everything either, because while that depth and complexity are well suited for turn based strategy (or at least a subset of it), it doesn’t work for a platformer or a first person shooter. Halo is painfuly simplistic compared to Europa Universalis, but that doesn’t mean that we can say, “Oh it’s too simple, so it gets a 7.” This is the sort of problem we end up with when we take one specific genre(or set of genres) and hold everything else up to it. When we say, implying a fault, that something is not deep, or rather, not deep enough, then we have to ask, “Relative to what?” If the answer is something other than “what the game needs to be to be as fun as possible” then we’ve applied an arbitrary external metric to the game. Such a metric is increasingly invalid the further away you get from the personal preferences of the reviewer too.
And for all of that, the vast bulk of reviews we see today fall exactly into that sort of trap, using standards that simply do not apply to the things that they are being compared to. For example, graphics and depth don’t really apply to something like Wii Sports, because it purposefully doesn’t target those areas, and ‘improving’ in them doesn’t necessarily increase the enjoyment that it’s target audience gets from the game. Making note of that fact is important so people know what to expect. But lowering the score because the reviewer wants them to be there only talks to me about the reviewer, and not about the game.
I do have more that I could say on this topic, but this is already way too long for a comment post, so maybe I’ll hit up the forum.
September 10th, 2007 at 11:18 am
I agree: we should all be driving Priuses!
(What? Did I miss the point again??)