Most AIS videos depict the appearance of the world, but PAN, developed by the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), aims to comprehend its inner workings.
When MBZUAI launched PAN last month, the focus was on its ability to generate videos. The true innovation, however, lies in its design to teach robots cognitive skills, not just to produce films.
Stay informed with the latest updates. Follow KT on Whatsapp Channels
“PAN is distinct from standard video generators that convert text prompts into realistic visuals; it’s engineered to understand and replicate the world’s essence,” Jon Carvill, MBZUAI’s Vice President of Marketing and Communications, exclusively told Khaleej Times. “While video models mimic appearances, PAN captures the dynamics that make those visuals authentic.”
While platforms like OpenAI’s Sora and Google’s Veo create cinematic imagery, PAN focuses on reasoning — a crucial capability for robots and autonomous systems.
This distinction is vital due to a major challenge in robotics: training robots in real-world settings is costly, time-consuming, and hazardous. Companies like Tesla’s Optimus and Figure AI’s Helix face significant hurdles, as a single prototype can cost hundreds of thousands of dollars. Training through real-world trial and error risks substantial financial loss with every mishap.
Current robotics firms require hundreds of human operators to perform thousands of repetitive demonstrations to teach just a few skills, resulting in enormous expenses and extended timelines.
PAN revolutionizes this process by developing a “world model”—an AI that not only generates visuals but also understands cause and effect, physics, and the long-term consequences of actions.
In PAN’s simulated environment, a robotic hand can experiment with various ways to grasp a cup before physically interacting with one.
Enhancing Behavior
“Within PAN, a robotic agent can practice thousands of interactions—from navigating traffic as an autonomous vehicle to folding laundry or loading a dishwasher as a household robot—perfecting its behavior before engaging with the real world,” Carvill elaborated.
The technical framework enabling these capabilities sets PAN apart from its competitors. Unlike other systems that generate entire videos in one sweep, PAN retains an internal memory of scene elements and object movements, updating its understanding incrementally as each new frame is created.
“PAN’s architecture is a hybrid: diffusion ensures visual quality, while the LLM sustains the world’s semantics over extended periods,” Carvill noted.
The implications are profound. Advanced physics simulations can train robots 430,000 times faster than real-world learning, compressing decades of practice into mere hours of computational time. This reduces costs and broadens access to advanced robotics.
The system signifies a shift towards “embodied AI”—intelligent systems that grasp physical consequences, not merely data patterns. Current large language models excel in text but lack understanding of physical world dynamics.
Carvill emphasized: “Intelligent agents must advance beyond text-only reasoning. By modeling both language and the world’s behavior and responses, PAN lays the foundation for genuinely embodied systems.”
MBZUAI’s distributed development approach—spanning teams across Abu Dhabi and Silicon Valley—expedited PAN’s development with well-defined roles and research pipelines, tapping into global talent across time zones.
Setting the Standard for Intelligent Agents
PAN aligns with MBZUAI’s broader Institute of Foundation Models strategy, which recently introduced K2 Think, an AI reasoning system. This work propels the university’s mission to develop AI capabilities that benefit the global research community.
Looking ahead to 2030, Carvill envisions a future where PAN-like world models become the standard framework for intelligent agents, driving safe autonomous systems, realistic virtual environments, and AI that understands consequences, not just correlations.
For Abu Dhabi, PAN places MBZUAI at the crossroads of two converging technologies: advanced AI and physical robotics. The city’s unique blend of government support, international talent, and regional industry partnerships offers “perspectives not found elsewhere” in global AI development, according to Carvill.
