Meanwhile, the 'blue team' is tasked with fixing the mess they (intentionally) make. 'The saboteur's job is basically to go through common failure points, or dustier areas of the platform that we haven't really tested as recently, and make sure that stuff is still up to scratch,' explains Dixey Hefty. That's why Midwinter Entertainment took a two-pronged approach to quality assurance, tasking a 'red team' to act as saboteur. When development is moving fast, it's all too easy to assume things are working smoothly, simply because you haven't been alerted otherwise. Red team and blue team QA testing kept runbooks up-to-date 'The only way to evaluate a complex, large-scale multiplayer game like Scavengers is to touch it, to feel it, to play it, - and to play it the way that players are going to play it.' 'Games are fuelled by your ability to understand, diagnose and respond to real player issues and real player updates,' explains Matteson. When they came across sticky issues, cloud-based infrastructure and global support teams in opposing time zones meant engineers could work in their sleep - or, rather, work got done while engineers slept. It also used 'sim players' to load-test the system from end to end, giving them the inside track on what real players would see. 'Games are fuelled by your ability to understand, diagnose and respond to real player issues and real player updates'Дaily playtesting allowed them to see through players' eyesД«ecause the internal infrastructure available via Improbable Multiplayer Services (IMS) matches the infrastructure deployed for live production, Midwinter Entertainment were able to constantly playtest internally from early on in the development process, and get realistic results. That day-to-day experience gives them a nice way to validate,' says Dixey Hefty. Developers can actively make changes because they're all running alongside each other in an isolated fashion, solving the problems they need to worry about, the things they need to iterate on quickly. 'In what we call the development environment, it's kind of the Wild West. Not only that, it was also more cost-efficient because the same machine was used for various tests. By maximising the number of iterations in development, they were able to test multiple scenarios simultaneously and fix things faster. Parallel testing meant Midwinter Entertainment could deploy builds in clusters without impacting other builds. Parallel testing got Scavengers to market quicker Improbable Multiplayer Service helped deliver a successful launch for Scavangers 'We were able to respond to customer demand almost instantaneously - bringing up regions where we had a higher population than we thought we were going to have, so that we could place game servers and lobby servers in the location where players could have the best latency experience.'Д®ager to know more, we caught up with Matteson and principal engineer for Scavengers Live Operations, Jared Dixey Hefty, to find out how their teams worked together to improve Scavengers' gameplay and pull off a successful launch. ' with Improbable Multiplayer Services, we brought up and down multiple regions on the day of launch and split regions in two,' he explains.

Thanks to a tactical collaboration, the Scavengers Early Access launch avoided 'a lot of the classic challenges,' according to Midwinter Entertainment's Vice President of Technology, Eric Matteson. How Midwinter Entertainment pulled off the Scavengers Early Access launch with minimal backend issues How do you get sufficient hardware to test your game? How do you effectively scale-test your systems? Where do you get the resource to manage and build your deployment process? For smaller studios that don't have access to the infrastructure of larger companies, rigorous testing ahead of launch is a constant challenge. It's a game developer's nightmare, all too often turned-reality. You turn to the runbook, but it's clear almost immediately it's out of date. Players all over the world are experiencing your game for the first time - tomorrow, the first reviews will be all over the web.

It's 2am on launch night and you're in the war room.

Scavengers live event