Xbox’s Matt Booty ‘dreams’ of having AI as QA testers

The head of Xbox Game Studios says it’s his dream to one day have an army of artificial intelligence QA testers.

Speaking at a Q&A at PAX West 2022 over the weekend, Booty explained that as games get more complex they’re more prone to bugs, and claimed that current QA testing methods struggle to keep up with this.

As such, Booty says he’s asked AI researchers at Microsoft to come up with an AI that is able to replace a human when it comes to testing games.

“Some of the processes we have, have not really kept up with how quickly we can make content,” Booty said. “One of those is testing.

“You think about a game, one of the biggest differences between a game and something like a movie, is if we’re working on a movie and you come in and say ‘hey, this ending, let’s tighten this up, let’s edit this, let’s cut that scene’, it usually doesn’t break anything at the beginning of the movie.

“But in a game you can be ready to ship, and a designer’s like, ‘I’ve got this one little feature, I’m just going to change the colour on this one thing’’ and then it somehow blows up something and now the first 10 minutes of the game doesn’t play.

“So that testing aspect, every single time anything new goes into a big game the whole game has to be tested, front-to-back, side-to-side.

“My dream – there’s a lot going on with AI and machine learning right now, and people using AI to generate all these images.

“What I always say when I bump into the AI folks, is: ‘Help me figure out how to use an AI bot to go test a game.’

“Because I would love to be able to start up 10,000 instances of a game in the cloud, so there’s 10,000 copies of the game running, deploy an AI bot to spend all night testing that game, then in the morning we get a report. Because that would be transformational.

“I always kind of laugh a little bit, people always say ‘the game shipped on Tuesday but I hear they were fixing bugs on Saturday night’ – there’s months of testing and things that have to happen before a game goes out.”

Further reading

Activision Blizzard says all US-based QA testers will become full-time employees

The move follows months of unrest at Warzone developer Raven

The topic of QA testing being run by AI is one that is likely to become more widely discussed in the coming years, as publishers look for more cost-effective and efficient ways to test large-scale games.

Recent campaigns to unionise QA testers and recognise them as full-time employees may also make some publishers consider alternative testing methods in the future.

Activision Blizzard announced in April that all of its US-based QA testers would become full-time employees on July 1, receiving an hourly wage increase to a minimum of $20 per hour.