Asher Fishman 🎧💥 on LinkedIn: Google Sees AI Agents Getting Into the Video Games We Play (2024)

Google DeepMind's Latest AI Agent Learned To Play Goat Simulator 3: Will Knight, writing for Wired: Goat Simulator 3 is a surreal video game in which players take domesticated ungulates on a series of implausible adventures, sometimes involving jetpacks. That might seem an unlikely venue for the next big leap in artificial intelligence, but Google DeepMind today revealed an AI program capable of learning how to complete tasks in a number of games, including Goat Simulator 3. Most impressively, when the program encounters a game for the first time, it can reliably perform tasks by adapting what it learned from playing other games. The program is called SIMA, for Scalable Instructable Multiworld Agent, and it builds upon recent AI advances that have seen large language models produce remarkably capable chabots like ChatGPT.[...] DeepMind's latest video game project hints at how AI systems like OpenAI's ChatGPT and Google's Gemini could soon do more than just chat and generate images or video, by taking control of computers and performing complex commands. "The paper is an interesting advance for embodied agents across multiple simulations," says Linxi "Jim" Fan, a senior research scientist at Nvidia who works on AI gameplay and was involved with an early effort to train AI to play by controlling a keyboard and mouse with a 2017 OpenAI project called World of Bits. Fan says the Google DeepMind work reminds him of this project as well as a 2022 effort called VPT that involved agents learning tool use in Minecraft."SIMA takes one step further and shows stronger generalization to new games," he says. "The number of environments is still very small, but I think SIMA is on the right track." [...] For the SIMA project, the Google DeepMind team collaborated with several game studios to collect keyboard and mouse data from humans playing 10 different games with 3D environments, including No Man's Sky, Teardown, Hydroneer, and Satisfactory. DeepMind later added descriptive labels to that data to associate the clicks and taps with the actions users took, for example whether they were a goat looking for its jetpack or a human character digging for gold. The data trove from the human players was then fed into a language model of the kind that powers modern chatbots, which had picked up an ability to process language by digesting a huge database of text. SIMA could then carry out actions in response to typed commands. And finally, humans evaluated SIMA's efforts inside different games, generating data that was used to fine-tune its performance. Further reading: DeepMind's blog post.Read more of this story at Slashdot.

1

Google DeepMind's Latest AI Agent Learned To Play Goat Simulator 3: Will Knight, writing for Wired: Goat Simulator 3 is a surreal video game in which players take domesticated ungulates on a series of implausible adventures, sometimes involving jetpacks. That might seem an unlikely venue for the next big leap in artificial intelligence, but Google DeepMind today revealed an AI program capable of learning how to complete tasks in a number of games, including Goat Simulator 3. Most impressively, when the program encounters a game for the first time, it can reliably perform tasks by adapting what it learned from playing other games. The program is called SIMA, for Scalable Instructable Multiworld Agent, and it builds upon recent AI advances that have seen large language models produce remarkably capable chabots like ChatGPT.[...] DeepMind's latest video game project hints at how AI systems like OpenAI's ChatGPT and Google's Gemini could soon do more than just chat and generate images or video, by taking control of computers and performing complex commands. "The paper is an interesting advance for embodied agents across multiple simulations," says Linxi "Jim" Fan, a senior research scientist at Nvidia who works on AI gameplay and was involved with an early effort to train AI to play by controlling a keyboard and mouse with a 2017 OpenAI project called World of Bits. Fan says the Google DeepMind work reminds him of this project as well as a 2022 effort called VPT that involved agents learning tool use in Minecraft."SIMA takes one step further and shows stronger generalization to new games," he says. "The number of environments is still very small, but I think SIMA is on the right track." [...] For the SIMA project, the Google DeepMind team collaborated with several game studios to collect keyboard and mouse data from humans playing 10 different games with 3D environments, including No Man's Sky, Teardown, Hydroneer, and Satisfactory. DeepMind later added descriptive labels to that data to associate the clicks and taps with the actions users took, for example whether they were a goat looking for its jetpack or a human character digging for gold. The data trove from the human players was then fed into a language model of the kind that powers modern chatbots, which had picked up an ability to process language by digesting a huge database of text. SIMA could then carry out actions in response to typed commands. And finally, humans evaluated SIMA's efforts inside different games, generating data that was used to fine-tune its performance. Further reading: DeepMind's blog post.Read more of this story at Slashdot.

1

Google DeepMind's Latest AI Agent Learned To Play Goat Simulator 3: Will Knight, writing for Wired: Goat Simulator 3 is a surreal video game in which players take domesticated ungulates on a series of implausible adventures, sometimes involving jetpacks. That might seem an unlikely venue for the next big leap in artificial intelligence, but Google DeepMind today revealed an AI program capable of learning how to complete tasks in a number of games, including Goat Simulator 3. Most impressively, when the program encounters a game for the first time, it can reliably perform tasks by adapting what it learned from playing other games. The program is called SIMA, for Scalable Instructable Multiworld Agent, and it builds upon recent AI advances that have seen large language models produce remarkably capable chabots like ChatGPT.[...] DeepMind's latest video game project hints at how AI systems like OpenAI's ChatGPT and Google's Gemini could soon do more than just chat and generate images or video, by taking control of computers and performing complex commands. "The paper is an interesting advance for embodied agents across multiple simulations," says Linxi "Jim" Fan, a senior research scientist at Nvidia who works on AI gameplay and was involved with an early effort to train AI to play by controlling a keyboard and mouse with a 2017 OpenAI project called World of Bits. Fan says the Google DeepMind work reminds him of this project as well as a 2022 effort called VPT that involved agents learning tool use in Minecraft."SIMA takes one step further and shows stronger generalization to new games," he says. "The number of environments is still very small, but I think SIMA is on the right track." [...] For the SIMA project, the Google DeepMind team collaborated with several game studios to collect keyboard and mouse data from humans playing 10 different games with 3D environments, including No Man's Sky, Teardown, Hydroneer, and Satisfactory. DeepMind later added descriptive labels to that data to associate the clicks and taps with the actions users took, for example whether they were a goat looking for its jetpack or a human character digging for gold. The data trove from the human players was then fed into a language model of the kind that powers modern chatbots, which had picked up an ability to process language by digesting a huge database of text. SIMA could then carry out actions in response to typed commands. And finally, humans evaluated SIMA's efforts inside different games, generating data that was used to fine-tune its performance. Further reading: DeepMind's blog post.Read more of this story at Slashdot.

Introducing Genie: The Game-Changing AI from Google DeepMindIn a world where artificial intelligence is increasingly becoming a part of our daily lives, Google's DeepMind has just upped the ante with Genie, a revolutionary AI that's pushing the boundaries of what's possible. Picture this: an AI that watched 30,000 hours of video games and then learned how to create its own. That's Genie for you – a leap into the future of generative AI in gaming.Where Imagination Meets PlayWhat sets Genie apart isn't just its ability to create; it's how it creates. By digesting a vast library of simple platformer games (think of those nostalgic early Nintendo classics), Genie can now take a doodle or photo and turn it into a fully interactive video game. Imagine sketching a character and its world, and then watching it come to life, ready for you to explore. This is no small feat. It's a magic portal where your creativity directly influences the game you play.Learning Without LimitsHere's where Genie truly dazzles: it learns without needing everything spelled out. In traditional game development, every element is meticulously coded and designed. Genie, however, crafts game environments on the fly, dynamically responding to player inputs, creating a game world frame by frame. It's akin to learning a language by immersion rather than from a textbook, a method that hints at an exciting new direction for AI development. This ability to learn from vast, unlabeled video content on the internet could revolutionize how AIs are trained.Just the BeginningIt's early days for Genie, and like any pioneer, it's not without its quirks. Today's versions might run a bit slowly, at one frame per second, and sometimes the worlds it creates can get a little...weird. But these are just stepping stones. As Genie evolves and digests more data, these early hiccups are expected to smooth out, making way for even more seamless and complex creations.Beyond the GameThe potential of Genie stretches far beyond making games. DeepMind envisions it as a tool that could simplify game development, yes, but also as a technology with applications we're only beginning to dream of. From controlling robots to teaching other AI agents, Genie's underlying technology could one day change how we interact with the digital and physical worlds alike.A Look AheadWith Genie, DeepMind isn't just showing off what they've made; they're opening a door to the future. In a field that's as competitive as it is collaborative, projects like Genie signal a bold step forward in the exploration of AI's potential. It's a testament to the vision that drives the AI community: a belief in a future where our digital creations can inspire, entertain, and maybe even help us see the world in new ways.

Unleashing AI Potential: GPT-4 Revolutionizes Minecraft Gameplay#GPT-4,#minecraft ,#ai ,#languagemodel,#automation Full article:https://lnkd.in/eBPpUU8F-Discover how the integration of GPT-4, a powerful language model, into Minecraft is unlocking new possibilities for AI. A Minecraft bot named Voyager taps into GPT-4's text generation capabilities to acquire new skills and solve in-game challenges, showcasing the potential of language models to automate tasks and enhance computer interactions.The groundbreaking potential of GPT-4, the advanced language model powering ChatGPT and numerous other applications, extends beyond mere conversation. Linxi "Jim" Fan, an AI researcher at Nvidia, alongside colleagues, devised a pioneering approach to harness the power of GPT-4 within the popular blocky video game, Minecraft.Their project involved the development of a Minecraft bot named Voyager, which leverages GPT-4's abilities to tackle in-game problems. By generating objectives and relevant code, the language model assists the bot in exploring the game and improving its skills over time.Unlike human players, Voyager interacts with the game using an API to directly perceive its state. For instance, it can identify the presence of a fishing rod in its inventory and a nearby river, prompting GPT-4 to suggest the goal of gaining experience through fishing. Subsequently, Voyager utilizes this objective to generate the necessary code, guiding the character's actions accordingly.The project's most innovative aspect lies in the code generated by GPT-4 to enhance Voyager's behaviors. If the initial code fails to execute flawlessly, Voyager refines it through error messages, game feedback, and descriptions provided by GPT-4. As time progresses, Voyager builds an extensive code library, enabling it to undertake increasingly complex tasks and explore new facets of the game. A comparative chart reveals Voyager's impressive capabilities, surpassing other AI agents by acquiring over three times as many items, venturing twice as far, and constructing tools fifteen times faster.Video games have long served as testing environments for AI algorithms. Notably, AlphaGo, the machine learning program that achieved mastery in the complex board game Go in 2016, honed its skills by playing simple Atari video games. AlphaGo employed reinforcement learning, which utilizes positive and negative feedback within a game to train algorithms.

Generative AI will transform gaming business modelsThe gaming sector has traditionally been an early adopter of machine learning and artificial intelligence, however the speed of innovation in AI has become faster while development timelines for major gaming titles has lengthened. When Bethesda's most recent Starfield title was previewed at E3, a gaming conference, in 2018, OpenAI was refining GPT-1 into what would become GPT-2 in 2019. When Starfield was finally launched in September 2023, GPT-4 was six months old. Any AI current at the beginning of the 5+ year development cycle would have been far out of date long before launch.Mods to the rescue!For the non-gamers on LinkedIn, a well-known sub-discipline of gaming is the fan-driven modder community that builds extensions and modifications to existing titles. A quick Google search will show the sheer scale of this technical fanbase and the breadth of mods available for a broad range of game titles and genres. This built-in extensibility will provide game developers a mechanism to incorporate current AI technologies and publishers a way to keep these expensive product efforts (Starfield was rumored to cost over $200m to develop) relevant and generating revenue for years to come.One recent example released is Mantella, a fan mod for another one of Bethesda's marquee titles, Skyrim (released in 2011). Mantella incorporates OpenAI's ChatGPT as well as its Whisper speech-to-text stack. This enables a natural language interface to interact with non-player characters in the game via voice conversation. For Starfield, that could include a detailed conversation with a planetary Governor on the history of mining minerals on Mars, as one example. When you come back to the NPC later in the game, they not only remember your previous conversation but also incorporate context from events that have transpired since your last interaction. This will deepen immersion in the game and also create more opportunities to cross over with education (edu-gaming) applications. The current weak spot are limitations in text-to-speech, but this should be addressed by technologies such as voice cloning offered by ElevenLabs (elevenlabs.io). Based on limited samples of a voice, the cloning can replicate a speaker saying provided text in a very convincing manner. This will open up additional revenue streams for companies that are able to license these clone models. Previous voice actors for Bethesda's series of games (Elder Scrolls) in the Skyrim genre include Sir Patrick Stewart, Lynda Carter, and (now deceased) Max Von Sydow and Christopher Plummer. One can imagine having a voice conversation with King Arthur, voiced dynamically by a clone model of Sean Connery, regarding the nuanced politics of Camelot.These technologies will shift the current main title/DLC revenue model to one of ongoing improvements provided via mods, and will create recurring revenue for game publishers for years to come.

34

8 Comments

#ChatGPT made paid stuff free.OpenAI announced their new flagship AI model GPT-4, which brings the faster and more accurate GPT-4 capabilities to free users previously reserved for paid customers.The voice demo showed GPT-4 could generate a bedtime story, solve a math equation shown through a phone camera, and attempt to read facial expressions/emotions.Though safeguards were mentioned, few details were provided on preventing misuse of the voice and facial recognition capabilities.OpenAI is reportedly close to a deal to integrate their AI into Apple's iPhone operating system.🐾 Our commentWe find OpenAI's GPT-4 very promising for general purposes due to its free price and fast speed.If integrated well, features like voice interaction and emotion recognition could enable more immersive and engaging mobile games. We're excited about the potential AI brings to enhance mobile games and user experiences.🔔 Afraid of missing out on our news?Worry not!Place the emoji 😀 in a comment below, and we'll make sure to tag you once we publish anything new.👑 Here are our VIP alert subscribers:Martyn Redstone, Adi Beker, Junwoo Yun, Maryanne Collins, Kirill Shamanskiy,Vakhtang Matskeplishvili,Marianna Inozemtseva,Elad Cohen, Chen Offek, Kamil Sawko, Michal Lissak, Eze Vidra,Yash Baid, Cosmin Tanase, Shashikiran R, Erez Rachmil, Meir Peleg, Raz Rosh, Almog Ben Simon#tech#innovation #mobile #game #development

10

Exploring the Potential of Generative AI in Game Development - how to use it eff: In a recent survey of 243 game makers, 87% use AI as part of development, while 99% plan to adopt it in the future, with ChatGPT, Midjourney, Stable...

🤖🎮 Harnessing the Power of AI: How Game Development Studios Can Level Up! 🚀✨Greetings, gamers! Today, we're diving into the exciting world of artificial intelligence (AI) and how game development studios can leverage this cutting-edge technology to revolutionize the gaming experience. Join us as we explore the myriad ways AI can enhance game development and take interactive entertainment to new heights! 🌟🎮1️⃣ Intelligent NPCs: AI allows game developers to create more realistic and immersive non-player characters (NPCs). By utilizing AI algorithms, NPCs can exhibit complex behaviors, adapt to the player's actions, and provide dynamic interactions. This brings game worlds to life, making each encounter feel unique and believable. From crafting intelligent enemy AI to developing realistic ally behaviors, AI empowers developers to create engaging and challenging gameplay experiences.2️⃣ Procedural Generation: AI algorithms can be employed to generate vast and procedurally generated game worlds, reducing the need for manual design and saving development time. By leveraging AI, developers can create diverse landscapes, unique levels, and dynamic quests, offering players endless possibilities and replayability. Procedural generation powered by AI can breathe life into open-world environments, making every playthrough a fresh and exciting experience.3️⃣ Adaptive Difficulty: AI-driven adaptive difficulty systems enable games to dynamically adjust the level of challenge based on player performance. By analyzing player behavior and skill, AI algorithms can modify enemy strength, puzzle complexity, or game mechanics to ensure players are consistently engaged. This personalized experience keeps players in the "flow" state, offering a balance between challenge and enjoyment for players of different skill levels.4️⃣ Natural Language Processing: AI-powered natural language processing (NLP) can enhance player interactions within games. Voice recognition and text-based AI systems enable players to engage in immersive conversations with NPCs, opening up new possibilities for storytelling and quest design. NLP also allows for more advanced chat systems, enhancing multiplayer experiences and fostering better communication between players.5️⃣ Quality Assurance and Bug Detection: AI can assist in the quality assurance process by automatically detecting bugs, glitches, and performance issues. Machine learning algorithms can analyze game data, identify patterns, and pinpoint potential problem areas, reducing the time and effort required for manual testing. AI-powered testing tools can improve game stability, optimize performance, and enhance the overall player experience.By harnessing the power of AI, game development studios can unlock a realm of possibilities and elevate their games to new heights. From intelligent NPCs to procedural generation and adaptive difficulty, AI empowers developers to create immersive, dynamic, and engaging experiences.

2

9 Trending AI Tools To Create Captivating NFTs: 1. DALL-E: Most Popular Tool From OpenAI · 2. MidJourney: Not Just Any Design Tool But Powerful NFT Generator Tool · 3. NightCafe: Easy To Use AI Tool...