Steam Reviews Changed- Overall Score No Longer Includes Reviews Via Steam Key Activation
42 replies, posted
[t]http://i.imgur.com/tXW13X8.jpg[/t]
[quote]We are also changing the default review score that we show at the top of each product page (and in search results) to not include reviews written by users that obtained the product through a Steam key. Here's why:
When we introduced the Steam Customer Review System in November 26, 2013, our primary goal was, and still is, to help customers make an informed decision when considering the purchase of a new game. To achieve that goal, we've put an emphasis on written reviews that encourage customers to share their experience in a game so that other's can decide for themselves whether the game sounds like something they would enjoy playing.
As the number of reviews on any particular product grew, it became difficult to get a sense of whether customers were generally happy or unhappy with how well the game met their expectations. To make it easier to tell whether customers overall would recommend purchasing the game, we created a review score. We've intentionally kept this score as transparent as possible, by simply calculating the percentage of positive reviews.
We know this review score has become a valuable shortcut for customers to gauge how well the game is matching customer expectations. But the review score has also become a point of fixation for many developers, to the point where some developers are willing to employ deceptive tactics to generate a more positive review score.
The majority of review score manipulation we're seeing by developers is through the process of giving out Steam keys to their game, which are then used to generate positive reviews. Some developers organize their own system using Steam keys on alternate accounts. Some organizations even offer paid services to write positive reviews.[/quote]
As noted however, Valve has included more filtering options.
[url=http://store.steampowered.com/news/24155/]Steam page[/url]
Seems to make sense, I hadn't thought of the fact that most review manipulation would use copies activated through keys.
That'll probably help with the fake reviews on garbage Greenlight games.
[QUOTE=Grandzeit;51043507]That'll probably help with the fake reviews on garbage Greenlight games.[/QUOTE]
the fuckers will just find another way tbh such as buying a massive amount of their own game and then sending it out through those greenlight groups
what I really want to know is how they'll stop review bombers who buy and immediately refund the game. 1 hour play time to review a game? (then again that wouldn't stop ANYTHING would it)
[QUOTE=Butthurter;51043506]cant believe it took them this long[/QUOTE]
Because the solution was obvious?
I don't think I've ever read anyone suggest removing the capacity for key activated games to review.
I never even thought of this, but now that they implemented it, it seems fair and worthwhile.
So if I get a physical copy or a key from some legit site like GMG cause they're priced better my review won't matter for shit? Not sure I understand this logic.
If some small shitty greenlight game from an unknown dev comes out and has nothing but positive reviews it's already suspicious enough
[QUOTE=J!NX;51043522]the fuckers will just find another way tbh such as buying a massive amount of their own game and then sending it out through those greenlight groups
what I really want to know is how they'll stop review bombers who buy and immediately refund the game. 1 hour play time to review a game? (then again that wouldn't stop ANYTHING would it)[/QUOTE]
Could display the play time someone has in a game as of them writing a review on the review, so if someone does a review within the refund period and they're not saying it's complete shit or has technical issues you've got a warning it might be bogus.
[quote]We know this review score has become a valuable shortcut for customers to gauge how well the game is matching customer expectations. But the review score has also become a point of fixation for many developers, to the point where some developers are willing to employ deceptive tactics to generate a more positive review score.
The majority of review score manipulation we're seeing by developers is through the process of giving out Steam keys to their game, which are then used to generate positive reviews. Some developers organize their own system using Steam keys on alternate accounts. Some organizations even offer paid services to write positive reviews.[/quote]
what is the point?
if your game gains traction because it has a positive score, people will play it, realize how shit it is, refund it and (likely) leave a negative review
[QUOTE=Thunderbolt;51043564]So if I get a physical copy or a key from some legit site like GMG cause they're priced better my review won't matter for shit? Not sure I understand this logic.
If some small shitty greenlight game from an unknown dev comes out and has nothing but positive reviews it's already suspicious enough[/QUOTE]
Your review will still be visible for all to see. Do you have a better solution for the gaming of the system by the devs themselves distributing keys for good reviews? Is it really that important to you that yours counts towards the aggregate score?
This is one of those odd cases where selecting a smaller sample size more accurately reflects the overall opinion of the game. As a consumer, I care more about that than just being yet another number in a statistic.
[QUOTE=MasterKade;51043621]what is the point?
if your game gains traction because it has a positive score, people will play it, realize how shit it is, refund it and (likely) leave a negative review[/QUOTE]
so people don't waste their time buying and refunding the game in the first place?
[QUOTE=Ziks;51043502]Seems to make sense, I hadn't thought of the fact that most review manipulation would use copies activated through keys.[/QUOTE]
This means review copies and copies bought from stores like Humble won't count as part of the review %.
[QUOTE=Rossy167;51043680]This means review copies and copies bought from stores like Humble won't count as part of the review %.[/QUOTE]
More simply said bought anywhere other than Steam.
The best thing about this for me is that now I can finally see reviews in both English and Spanish instead of all of the Spanish reviews automatically being hidden from me.
[QUOTE=Blade Rx69;51043673]Your review will still be visible for all to see. Do you have a better solution for the gaming of the system by the devs themselves distributing keys for good reviews? Is it really that important to you that yours counts towards the aggregate score?
This is one of those odd cases where selecting a smaller sample size more accurately reflects the overall opinion of the game. As a consumer, I care more about that than just being yet another number in a statistic.[/QUOTE]
I don't care about [I]my own[/I] reviews, but there's a considerable amount of people buying keys from outside steam, and I feel like their opinions should be just as important as those of people who buy the games directly from steam (sometimes just to leave a negative review and refund the game, also contributing to the overall score's inaccuracy).
I can see this being well used for games that went through kick starters. More often than not backers will be given keys once said games release on Steam. Bias would most likely be involved when they all go to make their reviews so it should provide easier reading between the fan base and first time players.
I also like how valve acknowledged they have other issues which they're looking into.
[QUOTE]There are some titles where the most helpful reviews don't seem to accurately match the general customer sentiment. For example, there are a couple of prominent titles that have review scores of 'positive' but all the reviews marked as helpful are negative. We need to look at this to figure out how to represent cases where the community has highly divergent opinions.
There are some titles where a small group of users are able to consistently mark specific reviews as helpful, and as a result can present a skewed perception of what customers are saying about the game. This is obviously not ideal, so we're looking at ways to ensure that a few users don't have outsized influence over the system.
Some off-topic reviews get marked as 'helpful' simply because they are funny. These don't appear to actually be helpful in determining whether you should buy the game, so we're working on some ways to better detect and filter out these.[/QUOTE]
[QUOTE=Thunderbolt;51043928]I don't care about [I]my own[/I] reviews, but there's a considerable amount of people buying keys from outside steam, and I feel like their opinions should be just as important as those of people who buy the games directly from steam (sometimes just to leave a negative review and refund the game, also contributing to the overall score's inaccuracy).[/QUOTE]
When looking at a game's review score, I want it to reasonably represent the community's opinion of the game, not be skewed by a developer's underhanded tactics to boost it. This is an issue that has come up several times. The refunding aspect doesn't seem to be an significant issue yet, but it might in the future.
I've actually been offered free games to leave good reviews on for being a "notable member" of the Steam community. Now, these were mostly your generic RPGmaker games, though these are the games that this underhanded tactic has the most impact on: the unknown games with far fewer players and reviews.
As nice as it would be to get everyone's honest opinion on the game, it can be incredibly difficult to determine the authenticity of a review. I would consider this move a good compromise as it selects a subset of players that can be proven to have purchased the game for themselves. People who didn't buy it from Steam can still leave reviews to be read by others.
When it comes to buying products, I'll take a more accurate representation over being completely inclusive any day.
How unfortunate. Probably over half of my games are steam keys. I buy most new games on GMG because it's almost always cheaper. I don't write reviews often, but I do every now and then.
It seems to me that this is just another case of Valve restricting their customers further because they don't want to have to regulate their own platform. We've seen it time and time again, with steam support, trading, and recently the very controversial game gifting restrictions. This simultaneously helps and hurts Steam users.
I'll admit, without hiring a store moderation team, I don't see an alternative. And I realize how annoying it would be to regulate this manually. It's just a shame how many innocent people will not have their reviews counted.
I buy keys from greenman gaming because of my location so this is kind of lame but then again I've only used reviews for a laugh
[QUOTE=The Pretender;51044601]It's just a shame how many innocent people will not have their reviews counted.[/QUOTE]
About 5% of games are activated through keys instead of steam purchases, if the picture up top is anything to ballpark by. Even if that isn't entirely accurate, it's still a minority. And the reviews [I]still count and show up[/I], they're just not put into the statistic to try to combat review-bloating by shitty devs.
Yeah, this won't go over well for those who get games from GMG or Humble Bundle.
And it won't combat the "review bombing" either. People will still post numerous shitty reviews, and may even take advantage of the refund system to do so.
There really is no way to combat this unless you moderate reviews, but then we go into the territory of user reviews being manipulated (removed) despite them being legit criticisms, which has also happened.
Basically, there's really no way to solve this that I can think of that gives everyone an equal footing.
So I guess if I decide to make a review of Left 4 Dead 2 or Portal 2, they won't count since I bought the physical disc
[QUOTE=Mio Akiyama;51044858]So I guess if I decide to make a review of Left 4 Dead 2 or Portal 2, they won't count since I bought the physical disc[/QUOTE]
Exactly.
[QUOTE=Combine 177;51043800]More simply said bought anywhere other than Steam.[/QUOTE]
What seems nuts is that reviewers can't have an influence on steam reviews, since they'll have got keys.
[QUOTE=Mio Akiyama;51044858]So I guess if I decide to make a review of Left 4 Dead 2 or Portal 2, they won't count since I bought the physical disc[/QUOTE]
Physical media doesn't usually rely on review scores anyways. Its not like valve still puts out disks (and even when they did they just re-directed you to steam to download it).
Imagine walking into a Game-Stop or equivalent game retailer and seeing a bunch of game review score stickers on the covers. Its an instant loss of profit for anything that isn't high in score simply by perception alone.
Steam on the other hand has no overhead on a game (aside from the traffic of downloading the product) and as such they don't really care whether or not a game is good or not, and are more than happy to tell you whether it is with metacritic and steam user reviews.
Consider how many devs would opt-out of displaying user reviews if they had the choice to. Steam needs the user reviews to be as accurate as possible since it will reduce the general amount of unrest, downloads, and refunds of disingenuous or bad games.
Good that they're trying, but I can't help that think there's a more direct way to deal with the problem.
Maybe create a report category for fake spam reviews? Stop counting reviews from shared IPs? Hire additional moderators and support? Actually fucking moderate Greenlight?
[QUOTE=C0linSSX;51045551]Good that they're trying, but I can't help that think there's a more direct way to deal with the problem.
Maybe create a report category for fake spam reviews? Stop counting reviews from shared IPs? Hire additional moderators and support? Actually fucking moderate Greenlight?[/QUOTE]
Can't get much more direct than removing the motive of profit itself. These things turn into an endless arms race where the side where the more superior and cost effective approach wins.
Fake reviews and comments plague any site that involves profit or public perception in some way. Amazon, Yelp, Metacritic, etc.: I can guarantee you people will be looking for ways to exploit them if they haven't already established ways to.
You crack down on one detectable aspect, then people will figure out a way around it. For example: a number of reviews coming from one IP, so they block that IP. The operator notices the accounts aren't able to post any more, figures out why, then upgrades to a system that utilizes a variety of IPs.
The smart ones will develop systems that are even hard for humans to detect. After all, it's just posting reviews, not interaction with a person, so a Turing test is out of the question. The naïve suspects will just have a bot spam the same review with [game name here] or something. The smart ones will make them vary, some even getting into machine learning with legitimate reviews.
By far the most efficient way to solve this is a cost barrier. To make a review that impacts the number on the store page, the account must have provably purchased the game. This removes the profit motive, which I would call the most direct solution.
I suppose now reviewers will receive gifts instead of keys for reviews.
[QUOTE=Rossy167;51045398]What seems nuts is that reviewers can't have an influence on steam reviews, since they'll have got keys.[/QUOTE]
[QUOTE=rndgenerator;51045785]I suppose now reviewers will receive gifts instead of keys for reviews.[/QUOTE]
Does the press/reviewers even interact with the user review system? I thought that all went into metacritic or the curator system
Sorry, you need to Log In to post a reply to this thread.