This Google AI will play video games for you. SIMA is already an impressive technology
One of the joys of playing a video game is actually… playing. If you just want to watch a game, you can simply watch a video of someone playing, a stream, or something else. But Google is working on an artificial intelligence model that has the ability to play video games for you, just by following what you tell it to do. This AI is called SIMA, which stands for Scalable Instructable Multiworld Agent, and if it works as advertised, the practice of playing video games could change.
This Google AI Will Play Video Games for You
Google DeepMind, the AI subsidiary of the American giant, announced this new model in a blog post and on X. According to Mountain View’s firm, SIMA is the first general AI agent that can react to instructions in natural language in 3D environments. In other words, it is capable of playing based on your own instructions. Tell it to “turn left” and SIMA will turn the character to the left.
Google DeepMind worked with eight video game studios to train SIMA, including Hello Games, the creators of No Man’s Sky, and Tuxedo Labs, the makers of Teardown. The development team wanted a variety of different types of games to do this, as each new variable added skills to the model. Google DeepMind even designed a sandbox-like environment, where SIMA had to build structures to assess its understanding of physics and object manipulation.
What makes SIMA so impressive, in theory at least, is that it does not need technical information about the video game itself. It simply acts using game images and your natural language commands. Google DeepMind states that SIMA can perform over 600 “basic actions,” such as turning in a specific direction, interacting with objects, and using in-game menus. However, Google DeepMind is working on more complex actions, as well as commands that contain multiple sub-actions. It’s one thing to tell an AI to climb a ladder in front of the character, but it’s another to train it to respond correctly to an order like “mine resources to build a shelter.” The company notes a limitation with large language models, as bots easily respond to simple commands but still struggle to perform intuitive actions independently.
SIMA, An Impressive Technology Already
At the same time, Google DeepMind highlights its success with its multi-game training model, stating that SIMA outperforms models trained on a single game. According to the tech giant, SIMA can even perform better in a game it has never seen before than a model trained only on that specific game.
While SIMA is not yet available to the general public, there are potential use cases for this technology. It could be a great accessibility option: for players who have difficulty using a traditional controller, verbally indicating to the AI what you want to do should change everything. Google’s ultimate goal here goes beyond that, as the AI should be able to play games on its own. This could be very helpful for performing repetitive tasks, like passing levels or earning money, but then the question arises: why play a video game if a robot does all the work?
This also marks the second major AI project for video games. Last month, the tech giant announced working on a model to generate 2D platform games through natural language commands. Who knows, maybe in the near future we’ll see the arrival of “Google Gaming”: you just have to tell the AI the type of game you’d like to see, and the technology would generate the game and play it for you in real time.