A few days ago, Valve changed how user reviews work on Steam. Now, user reviews on the search page and at the top of the product page will only reflect customers who purchased the game from Steam. Other user reviews will still be collected, but only from the product's reviews panel with a more broad filter applied, which must be done manually.
This change was made because Valve detected some titles where review scores varied greatly between Steam user and outside keys. If the vast majority of reviewers who purchased the content on Steam and the vast majority of reviewers who acquired the game outside of Steam are the same, then random error converges quickly. An average of 1000 reviews should be within 3% of the average opinion of 1,000,000 random customers, for 95% of titles. 99% of titles would be within 4% of the average opinion, given 1000 reviews for a million customers.
Of course, the differences are not always truly random. Keys which were given to crowd-funding backers could be abnormally good, if it well-served the niche audience that helped it get made, or abnormally bad, if it slighted that audience.
In the worst case, developers could be giving away keys to services that flood fraudulent reviews.
As such, Valve took the position that
it will (Update: Yeah, I kind-of messed up the grammar on this sentence when I restructured it in editing… read it without the strikethrough, and this update of course) only reviews from their direct customers would be promoted. This upset many developers, although some games received a bump in score, if you trust Steam Spy. Again, if their title was a hit on Kickstarter, Patreon, or other services, then it subtracts their most evangelical users.
On the other hand, from Valve's perspective, they want to promote the opinion that best applies to someone browsing on Steam. This makes sense, since a review should be intended to guide someone who doesn't already have an opinion of the title one way or the other. Again, reviews are designed to be the general consensus of a random group of people — the expected value of an average user — but constrainted to a certain set of properties.
Of course, it would be beneficial to Valve to run further experiments to make sure that an average Steam reviewer reflects an average Steam customer for each, specific title. Basically, it's a good hypothesis, but testing isn't done. It could change greatly as it evolves through the Scientific Method.
As a backer of more than 700
As a backer of more than 700 video game related crowd-funded projects, a user of legit key sites other than Steam (like GMG), go fuck yourself, Valve.
Yeah, that’s fair. Even
Yeah, that's fair. Even though their goal is to be representative of the average customer on Steam, that implicitly says they don't value the opinion of people who don't fit that archetype.
Hi Scott, could you verify
Hi Scott, could you verify what Penteract said and update the news? if true, looks a legit move to avoid bad reviews affected by other sore limitations (Window store)
Valve’s official statement,
Valve's official statement, from the source link:
We are also changing the default review score that we show at the top of each product page (and in search results) to not include reviews written by users that obtained the product through a Steam key. Here's why:
You can still review, it just will not appear in the default filter, which is used by the search page. I mean, there could be a bug that lets other reviews into that filter, but it would either be a just a bug, or Valve's statement would be wrong.
Well it’s a Steam review of
Well it’s a Steam review of Steam’s customers’ purchased through Steam gaming titles. I guess it’s somewhat fair. There is still the option of seeing:
“Other user reviews will still be collected, but only from the product’s reviews panel with a more broad filter applied, which must be done manually”
So users can still get at that data with a manual setting. It’s Steam’s actual customer base doing the reviews, with access still allowed for others’ reviews with some manual settings. It least there is a filter option that allows for the other data to be shown.
That’s disappointing. I’ve
That’s disappointing. I’ve gotten a number of really good games in Humble Bundles that I wouldn’t have played/bought/reviewed otherwise.
Scott your article is wrong
Scott your article is wrong because it says you are required to buy the game from Steam. That isn’t true – you just need to have activated the game with the Steam code you get regardless of who you purchased it from.
I tested it to make sure.
So, this really is an attempt to clean up Steam’s review system and prevent people from making comments about games they have never played. If you bought it, you can comment.
If you’ve linked a non-Steam game to Steam, well, I suggest that you post your reviews on the sites of the entity you bought it from (like, GMG has its own system) or any of the various other outlets for such discussion. If you haven’t actually played the game then you have no legitimate reason to comment on it. Trying to boost interest in a game you, for example, backed through a kickstarter, is just abusing a system to get free advertising.
I’m no fan of Valve or Steam. I have no idea why people worship at the altar of Gabe Newell. They do things I don’t like and the Better Business Bureau rating of “F” for their customer service is well deserved. But this isn’t trying to put more money in Valve’s pocket, it is just trying to make the reviews on Steam more sensible by limiting the people that down-vote because of some stupid idea they got from an ignorant friend or upvote just because they like the developer. That, in my opinion, is a good thing.
Lol they actually have an F
Lol they actually have an F from the BBB!?
Based on how slow they’ve been to combat their platform being used to launder money, distribute malware, and how shitty their forums are it makes sense.
Yet a shadey local contractor
Yet a shadey local contractor in your city has an A+. I dont trust the BBB for their ratings. Unless I am mistaken Its an extortion scheme where companies can pay money to remove bad marks on their score.
BBB can try all it wants to
BBB can try all it wants to be credible, but it’s pretty pathetic. IMO, nothing is more powerful than YouTube when it comes to reviews of products. As for services, word of mouth from people on NextDoor are also pretty good as there is pretty strong regulation of users (must be residents of a geographic region) and ads (basically not allowed).
No, as I said, you can post
No, as I said, you can post your review, it will just not be visible with the default filter, which is used for the score on the search page. You can still add a review, and users can manually choose to see it, but it won't be there by default, with default filters.
Ah sorry you’re right, I was
Ah sorry you’re right, I was wrong about the what happens when you post your review and didn’t buy your game from Steam. 🙂
How did you “test” it? It
How did you “test” it? It says you can still post reviews no matter where the key came from, but they no longer count non-steam purchased reviews towards the final Steam “percent positive” review score.
Did you post an individual review and see the overall percent positive score change appropriately in response to your one review?
For Gabe Newell, Linux users
For Gabe Newell, Linux users have thanks for STEAM OS and for gaming and graphics! So that alone is reason enough to have great respect for the Prophet Gaben, to keep gaming/graphics from under the complete control of the evil M$ and its DX12 minions!
Also as Valve continues to develop its Steam OS, more hardware support will become available for all the Debian based/other Linux distros! So there will at least be better driver and Vulkan graphics API support for all hardware and the Linux/Vulkan and the Linux software/Linux Kernel ecosystem.
I’m looking to build a Linux OS based rendering system for Blender 3D using AMD’s AM4 Bristol Ridge based CPUs/APUs and Polaris GPUs with the AM4 system’s ability to be updated to Zen/Polaris or Zen/Vega in 2017. So it’s not all about Steam’s gaming as it is about that gaming support leading to better Linux/Debian hardware support for GPUs, and CPUs/Other PC/Laptop hardware. I like that Valve’s work with the Linux OS(Debian based Steam OS is improving the hardware support for the Linux/Linux Kernel in general, as well as also supporting Vulkan and a true cross platform graphics API not tied into any single OS/GPU hardware maker or closed ecosystem.
This is a good change for
This is a good change for Steam. When I read reviews on Amazon or Newegg, I always ignore reviews not made by verified purchasers.
The only downside that I can see is how it will affect reviews of games that are “only worth getting when on sale”. Unlike professional reviews and professional reviewers (that get the games for free or buy them as part of their operation budget), consumers must weigh games in a “fun per dollar” mentality. A large number of reviews will say something like, “This is a good game for under $20.” or something to that effect. “I would be upset if I paid $60 for this game, but for $35 it is definitely worth it.”
Yeah, that’s a whole other
Yeah, that's a whole other topic altogether.
I mean, I can see people who purchase content strictly for entertainment value that judge it based upon its perceived entertainment value. The problem, of course, is that art can also have intrinsic value. I like to describe it as the difference between checking what's playing at the cinema and deciding what to see, and planning to see a specific movie when it comes out.
That's one of the problems with numerical scores in general: it tries to apply a rubric to content without even considering why someone would want to see it. We tried to get around it with our game review system, used for exactly one title (Heart of the Swarm), by applying the numerical score to how good a game is technically (stability, DRM, makes use of PC features, etc.) and using a vague recommendation badge for the artistic content. Both scores are completely decoupled. The idea being, if you want a high score, all you need to do is put effort into QA, technology, and UX. With the value of the platform (engine, gameplay code) that delivers the content out of the way, the art is then graded as loosely as possible.
Wasn't really worth the time, though, and honestly I kind-of felt like an awkward, pretentious jerk.
I would love to see reviews
I would love to see reviews that are based upon objective metrics. If anything, that’s exactly why I subscribe to and support Digital Foundry:
1. frame rate stability
2. frame times
3. overall glitches
5. texture/object pop in
6. controller/input responsiveness
These are factors that can make a game with good content unplayable or less enjoyable. Factors such as plot, characters, art, etc. are all subjective and should be evaluated accordingly.
I agree that would be a
I agree that would be a pretty cool review system for pc games and as console comes to reassemble pc it could help game makers fix there sh** after release. Sadly i fear it wouldnt generate enough clicks for the poly**** or the kota**** euro***** to expend the effort.
I am glad I didn’t get a
I am glad I didn’t get a Vive, I bet the vr games have better scores than they deserve. haha