Odyssey’s AI Model Streams Interactive 3D Worlds in Real-Time

Odyssey, a startup founded by self-driving technology veterans Oliver Cameron and Jeff Hawke, has unveiled a groundbreaking AI model capable of streaming interactive 3D worlds in real-time. This innovative technology allows users to explore and interact with video content in a manner akin to navigating a 3D-rendered video game.

The AI model, accessible through an “early demo” on Odyssey’s website, generates and streams video frames at an impressive rate of 40 milliseconds per frame. Viewers can utilize basic controls to navigate and explore different areas within the video stream, creating a unique and immersive experience.

In a blog post, Odyssey explains that the model predicts the next state of the world based on current conditions, incoming actions, and a history of states and actions. This is powered by a novel world model that demonstrates capabilities such as generating realistic pixels, maintaining spatial consistency, learning actions from video, and outputting coherent video streams for durations of five minutes or more.

Odyssey joins a growing number of companies, including DeepMind, World Labs, Microsoft, and Decart, in the pursuit of advanced AI world models. These models hold the potential to revolutionize interactive media, enabling the creation of immersive games, movies, and realistic simulations for training robots and other applications.

While the potential of AI in creative fields is vast, there are concerns about its impact on creative professionals. A Wired investigation revealed that some game studios are using AI to reduce costs, while a study by the Animation Guild suggests significant job displacement in the film, television, and animation industries due to AI.

Odyssey, however, emphasizes its commitment to collaborating with creative professionals rather than replacing them. The company envisions a future where interactive video, powered by Odyssey’s technology, unlocks new forms of entertainment and transforms various sectors, including advertising, education, and travel.

Despite being in its early stages, Odyssey’s demo shows promise. While the generated environments are currently somewhat blurry and unstable, the company plans to rapidly improve the model’s capabilities. Currently, the model can stream video at up to 30 frames per second using Nvidia H100 GPUs, with an estimated cost of $1 to $2 per user-hour.

Odyssey is also exploring richer world representations to improve temporal stability and persistent state. The company is uniquely positioned in the world modeling space due to its use of a 360-degree, backpack-mounted camera system for capturing real-world landscapes. This approach, Odyssey believes, will lead to higher-quality models compared to those trained solely on publicly available data.

With $27 million in funding from investors like EQT Ventures, GV, and Air Street Capital, and the guidance of Pixar co-founder Ed Catmull, Odyssey is well-positioned to push the boundaries of AI-powered interactive video. Furthermore, the company is developing software to integrate its generated scenes with industry-standard tools like Unreal Engine and Blender.