Super Bunnyhop - The Ethics of Reviewing Early Access - General

[video=youtube;2TeH8fXBkcM]http://www.youtube.com/watch?v=2TeH8fXBkcM[/video] Also the new bunny logo is the old Foxhound logo from Metal Gear 2. [IMG]http://img1.wikia.nocookie.net/__cb20091109151143/metalgear/images/d/d9/SFG_FOXHOUND.jpg[/IMG]

did he really use minecraft as an example of early access done right?

[QUOTE=endorphinsam;44820492]did he really use minecraft as an example of early access done right?[/QUOTE] Well to be fair, it started out as this little block building thing and grew a shitload with weekly updates and stuff for a very long time during development before it's "release". And a massive amount of people are happy with what we got. Personally I don't think it's "finished" but the majority of people like it.

Funny enough I'm actually working on pretty much a series for early access "review". The difficult has been making it fair to the developers and the viewer, as obviously their games are subject to change from the point when I release the videos, I've spent this last week so far just polishing my criteria and making a few test videos to get the direction down better. If anyone wants me to post one, I'm working on hopefully releasing the first episode Monday. Not to hijack the thread or anything, but I'm really just glad that there is vocal demand for what I've been working so hard on over the last three weeks. Also, Minecraft isn't a really good example for early access, considering it never really explained what it was, even though it didn't really deliver a "finished" feeling product, it still was successful. Both Cortex Command and Minecraft have the same real issue in common though, they can't feel finished because they have no real explained end state. Minecraft was just suddenly said to be done, but no one knew what done really meant and they all had imagined something different, leaving many to be super bitter about it not being "done." It might also partially be that as the players spend so much time at the end game level of what they are able to play, they end up ruining the experience of the release because the release isn't a lot of additional content, they were already playing a 99% complete version with no ending, and when this ending comes, they don't like it because they feel they've been waiting so long for it to the point where nothing can actually feel like a proper ending. If Minecraft wasn't in early access but simply released, it would have had a completely different reception than the real release did.

[QUOTE=endorphinsam;44820492]did he really use minecraft as an example of early access done right?[/QUOTE] When Notch lost interest in developing Minecraft (he never outright said this but it's very obvious) he at least had the courtesy to push forward until development could continue without him rather than cancel the project or let it become vaporware. He was also smart enough to never have paid alpha/betas again once he was done. I [I]still[/I] wouldn't consider it to be Early Access done right though, given how many promises were broken and the fact that the official release was a glorified price hike. Really, I'd consider it a good cautionary tale about how even with the best intentions the model can easily become a pain in the ass for both the developer and the consumer.

[QUOTE=Aredbomb;44820827]When Notch lost interest in developing Minecraft (he never outright said this but it's very obvious) he at least had the courtesy to push forward until development could continue without him rather than cancel the project or let it become vaporware. He was also smart enough to never have paid alpha/betas again once he was done. I [I]still[/I] wouldn't consider it to be Early Access done right though, given how many promises were broken and the fact that the official release was a glorified price hike. Really, I'd consider it a good cautionary tale about how even with the best intentions the model can easily become a pain in the ass for both the developer and the consumer.[/QUOTE] I thought Scrolls had a paid "beta"

Well I wholeheartedly agree with him... My friend donated to the kick starter of Dead Linger and we both bought Zomboid :P

I do not agree with the idea of applying an objective measurement to a subjective experience, so I would rather that no games are given a score than all games are. I also do not agree with the notion that Early Access titles are being sold as complete products. Steam and Journalists may be presenting them as finished products, but the premise under which they are actually is sold is access to title which is still in need of refinement. What Early Access [I][B]should[/B][/I] be is a niche utility for dedicated fans to aid development of a title they care about and I cannot stand what steam has made it into by presenting it as just another product.

I've bought a few early access titles. Project Zomboid is progressing really slowly but other then that I'm fine with my early access purchases. But given the recent flood of shit titles. I'll probably stay away from early access unless the developer is very well known from now on.

[QUOTE=Rufia;44824400]I do not agree with the idea of applying an objective measurement to a subjective experience, so I would rather that no games are given a score than all games are. I also do not agree with the notion that Early Access titles are being sold as complete products. Steam and Journalists may be presenting them as finished products, but the premise under which they are actually is sold is access to title which is still in need of refinement. What Early Access [I][B]should[/B][/I] be is a niche utility for dedicated fans to aid development of a title they care about and I cannot stand what steam has made it into by presenting it as just another product.[/QUOTE] They are still being sold and thus need to have their actual value and long term chances to become a full product assessed.

[QUOTE=Rufia;44824400]I also do not agree with the notion that Early Access titles are being sold as complete products. Steam and Journalists may be presenting them as finished products, but the premise under which they are actually is sold is access to title which is still in need of refinement. What Early Access [I][B]should[/B][/I] be is a niche utility for dedicated fans to aid development of a title they care about and I cannot stand what steam has made it into by presenting it as just another product.[/QUOTE] That's the concept behind early access, but it's become perverted into a way to sell an unfinished game rather than to aid development. It lowers peoples' standards so they can get away with selling an incomplete game for at least half price or more. With Towns as an example, the developer can at any point say it's "finished" and walk away with the money. And thanks to early access, "release" has lost any meaning, because what differentiates "release" from "beta"? The game very likely won't change much from "beta" to "release" aside from a patch at best. Selling an unfinished game is becoming the norm rather than the exception, so they should be reviewed and critiqued as complete games if they're going to be sold like complete games.

[QUOTE=Rufia;44824400]I do not agree with the idea of applying an objective measurement to a subjective experience, so I would rather that no games are given a score than all games are. [/QUOTE] I don't think he agrees with them too, just most people take them seriously and choose whether or not to buy a game based on its score. He just wants the normal rules to apply to early access more than anything.

I really don't like this idea. If we're reviewing early access and slapping a score on an early build saying the game needs to be reviewed "With the understanding the game is a work in progress", you're already acknowledging your score might be wrong in a week after publication. You wouldn't put a score on a build that's most likely missing one or two of its design pillars. What Eurogamer and Gamespot are doing are fine. Review the playable build, don't damn the final product to a shitty Metacritic score. People buy early access to get the final product, and not just early access. Any risks involved are also willingly paid by the consumer.

[QUOTE=Super Muffin;44825044]I really don't like this idea. If we're reviewing early access and slapping a score on an early build saying the game needs to be reviewed, "With the understanding the game is a work in progress.", you're already acknowledging your score isn't representative of final quality. You wouldn't put a score on a build that's most likely missing one or two of its design pillars. What Eurogamer and Gamespot are doing are fine. Review the playable build, don't damn the final product to a shitty Metacritic score. People buy early access to get the final product, and not just early access. Any risks involved are also willingly paid by the consumer.[/QUOTE] Should DayZ or RUST be eligible for a review considering they sold a million units?

[QUOTE=KillerJaguar;44825078]Should DayZ or RUST be eligible for a review considering they sold a million units?[/QUOTE] I don't know. DayZ and Rust are games where reviews don't impact sales much. Games that spread through social media and let's plays are usually immune from criticism among regular players since they become "brand evangelists". I'm not thinking of the huge early access juggernauts that are selling so well. I'm thinking of games like Nuclear Throne or any F2P game on Steam. You can review a game that's releasing consistent updates through the business as a service model, but the problem with that is the key points of your review can be wrong immediately after publication. Polygon's review for Warframe comes to mind, where half of the issues touched on in the piece were fixed a week later. I just don't feel slapping a number on an early build is ethical or honest. [b]e:[/b] For example, try sticking a scored review on any of Minecraft's earlier revisions. Compared to now, there's a lot more content.

It seems that what we're really struggling with is how to review an evolving game experience. This problem isn't exclusive to early access games. Take, for example, TF2 which was released as a "finished" product 7 years ago but has slowly changed into a dramatically different experience. Are the original reviews even meaningful anymore? I see this a lot in MMO and F2P reviews as well: reviewers struggle to place value on the potential of a game, hoping that its developers will fill in gaps with free updates down the road. In that light, the "early access" label isn't really the issue. A lot of developers are shifting to this model of frequent, free updates after purchase (inspired by pioneers like Minecraft and TF2, I think). It presents a challenge to consumers and reviewers regardless of whether the original product is labelled as in-development or a full release.