Physical AI: The Next Frontier in AI and Robotics to Build Truly AutonomousMachines -- (I)

Ankit Parag Shah *, Minghao Liu,Jinrui Huang,Bhavya Pranav Tandra, Yu Wang, Sakshi Agarwal, Paul Wu, Aishik Konwer, Rhett Rozga, Selli P. Kondamuri, Alper Halbutogullari, Qin Zhang and  Wei Wei

Center for Advanced AI, Accenture

* Correspondence: ankit.parag.shah@accenture.com

Abstract

For decades, artificial intelligence transformed digital domains while the physical world, with its unforgiving dynamics and infinite variability, remained largely beyond reach.  That boundary is now dissolving: humanoid robots work full shifts on assembly lines, autonomous vehicles make millions of safety-critical decisions daily, and robotic surgical platforms have performed over twenty million procedures worldwide.  The convergence of foundation models, high-fidelity simulation, and embodied control is producing Physical AI, machines that perceive, reason, and act reliably in open environments. However, existing surveys treat individual components of this stack in isolation, covering vision-language-action architectures, world models, or sim-to-real transfer separately, and none traces the full pipeline from sensor to deployment or grounds analysis in commercial outcomes. Here we synthesise the end-to-end Physical AI technology stack: multimodal sensing and fusion; edge hardware and accelerators; world modeling and simulation; vision-language-action models; learning paradigms from reinforcement to imitation; and deployment infrastructure spanning safety assurance, fleet learning, and governance. Our analysis reveals three cross-cutting findings. First, foundation- model generalisation, not task-specific engineering, is the economic lever that separates scalable  deployments from those that stall at pilot stage. Second, digital twins have become prerequisites rather than accelerants: no deployment at fleet scale proceeds without high-fidelity virtual rehearsal. Third, regulatory and assurance barriers, not algorithmic limitations, now gate the transition from pilot to production across every domain. We document these patterns through commercial case studies in manufacturing, logistics, autonomous mobility, agriculture, and healthcare, and propose a four-phase maturity taxonomy characterising adoption from research prototype to fleet-scale operation.  We conclude with six coupled research challenges and a roadmap anchored to regulatory milestones through 2030, offering researchers, industry leaders, and policymakers a practical map for scaling embodied intelligence from controlled environments to the complexity of the real world.

Keywords: physical AI; embodied intelligence; foundation models; vision-language-action models; sim-to-real transfer; digital twins; world models; robot learning

Intelligence that merely thinks is no longer enough. For decades, artificial intelligence achieved remarkable feats in digital domains: mastering board games, generating fluent prose, and recognising objects in photographs. Yet the physical world, with its unforgiving dynamics, infinite variability, and millisecond deadlines, remained largely beyond reach.  Moravec’s paradox [1] captured the irony: the sensorimotor skills that a toddler performs effortlessly demand far greater computational sophistication than chess or theorem-proving. Today, that paradox is beginning to yield. Humanoid robots walk onto factory floors and work full shifts alongside people, performing dexterous assembly tasks that only human hands could accomplish a few years ago [2]. Autonomous vehicles navigate dense city traffic on multiple continents, making millions of safety-critical decisions daily [3]. Robotic surgical platforms have carried out over twenty million procedures worldwide [4], while autonomous tractorstill entire fields without a human aboard [5]. These advances share not a single algorithm but a convergence: foundation models trained on internet-scale data are merging with high-fidelity simulation, multimodal perception, and embodied control to produce machines that can perceive, reason, and act in the open world. We call this convergence Physical AI [6], and it marks the moment when artificial intelligence steps out of the screen and into reality.

1.1. What Is Physical AI?

Physical AI enables autonomous systems to perceive, understand, reason, and act in the physical world, adapting to situations never encountered during training [6]. The concept builds upon decades of research in embodied intelligence [7] and situated cognition, which emphasised that intelligence emerges from the interaction between agents, their bodies, and their environments. Four core functions define Physical AI systems:

Perceive: gathering and interpreting real-time data from cameras, LiDAR, radar, force-torque sensors, and other modalities to build a rich picture of the surrounding environment.

Understand:comprehending spatial relationships, object affordances, and the physical behaviour of the 3D world—going beyond pattern recognition to situational awareness.

Reason: making informed, goal-driven decisions under uncertainty, including planning multi- step actions that respect dynamics, contact, and safety constraints.

Act: executing or orchestrating complex physical movements through actuators—motors, grip- pers, wheels, and whole-body controllers—to interact with and reshape the environment.

Physical AI extends generative AI—which produces text and images in digital domains—to real-world tasks through embodied interaction.  Unlike traditional robotics, which relies on fixed programs in controlled settings, Physical AI replaces hand-crafted routines with adaptive policies that generalise, learn from experience, and handle novelty in realtime. This transition from passive computation to physical agency raises the bar for reliability, safety, and real-time performance beyond what digital-only systems require.

Fundamental challenges.

Several deep technical problems distinguish Physical AI from other AI domains. Moravec’s paradox  [1] observes that sensorimotor skills trivial for humans—grasping objects, navigating uneven terrain— require enormous computational resources for machines, reflecting millions of years of evolutionary optimisation that current AI lacks.  The reality gap causes policies trained in simulation to fail on physical hardware due to unmodelled dynamics, sensor noise, and environmental variation [8]. Data scarcity compounds these difficulties: even the largest robot demonstration datasets contain orders of magnitude fewer examples than the trillion-token corpora available for language models, limiting the learning approaches that have transformed NLP and vision. Real-time constraints demand that perception-to-action loops complete within milliseconds, precluding the iterative refinement possible in offline settings. Finally, contact-rich manipulation—handling deformable objects, assembling parts with tight tolerances, manipulating cables—remains challenging because accurate contact modelling is computationally expensive and small errors compound into task failure.

1.2. Evolution Toward Physical AI

The notion of intelligent physical systems has evolved through distinct paradigm shifts. Early robots such as Shakey (1966–1972) embodied the sense-plan-act paradigm, with symbolic planners like STRIPS searching for action sequences achieving specified goals [9,10]. Brooks’ subsumption architecture challenged this approach, demonstrating that robust behaviour could emerge from layered reactive controllers without explicit symbolic reasoning [11]. Both paradigms struggled with generalisation— classical planners required hand-crafted domain models; behaviour-based systems required hand- designed behaviours—and neither could leverage the statistical regularities in large datasets. The 2010s brought deep reinforcement learning into robotics: algorithms such as PPO [12] enabled robots to learn locomotion and manipulation through trial-and-error in continuous action spaces, while simulation tools like MuJoCo [13] and domain randomisation [8] began to bridge the gap between virtual and physical environments.

The current transformation is driven by the convergence of large language models (LLMs), vision-language-action (VLA) policies, and simulation-driven training pipelines.  Vision-language  models such as CLIP [14] bridged perception and language, and systems like SayCan [15] and Code-as- Policies [16] demonstrated that pre-trained LLMs could decompose high-level instructions into feasible robot actions. VLA models represent the latest evolution, directly integrating perception, language understanding, and motor control: RT-2 [17] co-trains on web-scale data and robotic demonstrations to generalise across tasks, PaLM-E [18] fuses multiple sensor modalities with language reasoning, and π0  [19] uses flow matching to produce continuous motor commands at 50 Hz across seven platforms. Beyond task-specific models, generalist agents such as Gato [20] treat text, images, and motor commands as tokens in a single sequence, providing early evidence that cross-modal transfer can benefit physical control. The 2023–2025 period has seen unprecedented industrial adoption, with low-code deployment platforms [21], foundation models for industrial grasping [22], and commercial- scale fleets in logistics, manufacturing, and urban mobility [23,24]. Physical AI companies attracted  over $7 billion in venture funding in 2024 alone [25], with mega-rounds signalling that investors view embodied foundation models as a generational opportunity. These deployments share a common enabler: high-fidelity simulation for safe exploration and rapid iteration at a fraction of the cost of physical prototyping, making digital twins and GPU-accelerated simulators essential infrastructure for the entire development cycle.

Figure 1 illustrates this end-to-end workflow.  While the pipeline superficially resembles the classical sense-plan-act loop critiqued by Brooks [7], modern Physical AI systems differ in three fundamental ways: (1) learned policies replace symbolic planning, enabling reactive behaviour without hand-crafted world models; (2) continuous feedback loops link simulation, deployment, and model retraining so that fleet experience refines the next generation of policies; and (3) foundation models provide implicit representations that blur the boundary between perception and action, allowing a single architecture to serve multiple embodiments and tasks.

Figure 1. Physical AI pipeline overview. Solid dark arrows (left to right) denote sequential data flow through seven stages: SENSE → EMBODY → UNDERSTAND → ADAPT → LEARN → DEPLOY → APPLY. Dashed coloured arrows (curving backward) denote three continuous feedback loops: (a) Continuous Learning—fleet telemetry feeds policy updates; (b) Digital Twin—deployment logs validate policies in simulation; (c) Foundation Model—edge cases trigger VLA retraining. Each stage maps to a numbered survey section.

1.3. Scope, Contributions, and Organisation

Existing surveys address individual components of the Physical AI stack in isolation—VLA architectures [26], world models for embodied AI, robot learning paradigms, or sim-to-real transfer— but none traces the full pipeline from sensor to deployment or grounds its analysis in commercial outcomes. Table 1positions this survey relative to recent related work.

Table 1. Scope comparison with recent related surveys. √ indicates substantial coverage; 。indicates partial or incidental treatment; — indicates not covered.

1. Full-stack integration perspective. We trace the complete Physical AI pipeline from multimodal sensing through edge hardware, world models, foundation policies, and fleet deployment (Fig- ure 1), exposing cross-cutting dependencies—between simulation fidelity and policy robustness, between hardware cost curves and deployment breadth, between governance frameworks and scalability—that single-topic surveys cannot capture.

2. Deployment-grounded analysis. We systematically document commercial deployments with measured outcomes across logistics, manufacturing, autonomous vehicles, and emerging do- mains, and propose a four-phase maturity taxonomy (Section 7) that characterises adoption trajectories from research prototypes to fleet-scale operations.

3. Critical gap identification and roadmap. We identify six coupled challenges—data ecosystems, sim-to-real resilience, lifelong adaptation, safety assurance, workforce integration, and sustainable hardware—and anchor a concrete research roadmap to regulatory milestones through 2030 (Section 8).

Survey methodology.

We surveyed literature from Google Scholar, Semantic Scholar, arXiv, and IEEE Xplore, focusing on 2020–2025 with selective inclusion of foundational work from earlier decades.  Search queries  combined terms from each pipeline stage (e.g., “vision-language-action models,” “sim-to-real transfer  robotics,” “robot fleet learning”) with deployment-oriented terms (“warehouse,” “manufacturing,” “autonomous driving”). We supplemented peer-reviewed publications with industry white papers, press releases, and verified deployment reports where academic sources were unavailable for recent commercial systems. All citations were verified against primary sources; arXiv preprints that have since appeared at peer-reviewed venues are cited with their published metadata.

The paper follows the pipeline of Figure 1. Appendix Asurveys multimodal sensing and fusion. Section 2 examines hardware platforms, edge accelerators, and the co-design of morphology with control. Section 3covers world modelling—SLAM, semantic mapping, intuitive physics, simulation infrastructure, and the emerging role of world foundation models. Section 4analyses VLA models and generalist policies from RT-2 through π0 to Gemini Robotics. Section 5reviews learning paradigms: reinforcement learning, imitation, self-supervised methods, and sim-to-real transfer.  Section 6 ad- dresses safety assurance, governance, fleet deployment, and societal considerations. Section 7presents  industry case studies with measured outcomes. Section 8synthesises open challenges into a research  roadmap through 2030.

2. Hardware Platforms and System Architectures

Physical AI systems depend on tightly coupled hardware stacks that span electromechanical design, sensing, compute and software.  Progress in any layer matters only when it improves the combined system: lighter end-effectors enable faster motion planning, more accurate sensors reduce the burden on perception models, and efficient accelerators make it feasible to run large policies on-device. This section reviews how platform costs, sensing, compute and simulation infrastructure are evolving together to enable practical Physical AI deployments at scale.

2.1. Platform Landscape and Cost Curves

Today’s robot platforms can reliably execute repetitive, well-defined tasks such as welding, depalletising, and single-SKU picking, but cannot yet generalise across environments, adapt to novel objects, or recover from unexpected failures without human intervention. This capability boundary, not component cost, explains why the market remains dominated by specialised form factors rather than general-purpose platforms [29]. Global shipments are nevertheless accelerating: IFR estimates 575 000 industrial units in 2025 and 700 000 by 2028 [30], with growth led by electronics and automotive. Traditional industrial arms from ABB, KUKA, and Fanuc dominate high-payload applications, while collaborative robots such as the UR series (Universal Robots) and GoFa (ABB) enable safer human- robot interaction in assembly and inspection tasks. Humanoid platforms are nevertheless accelerating: Figure announced a long-term agreement with BMW in 2024 to pilot its Figure 02 humanoids on assembly tasks, while Chinese manufacturer Unitree set an aggressive starting price from $4 900 for its R1 general-purpose humanoid aimed at researchers and service work [31,32].  Tesla’s Optimus  program targets retail pricing in the $20 000–$30 000 range for eventual manufacturing deployment  [33]. These examples illustrate a bifurcated hardware roadmap where niche automation continues reducing cost today while generalist platforms mature through targeted pilots.

2.2. Integrated Sensing and Actuation Stacks

Integrated sensor suites can now fuse vision, depth, and force-torque data well enough for a single robot to switch from gross handling to fine assembly without retooling, but only when the sensing rig,calibration pipeline, and task domain are co-designed; migrating a proven configuration to a new robot or environment still requires months of re-integration [29]. Warehouse automation similarly benefits from integrated sensor suites: Zebra Technologies’ vision systems and RFID readers combine with mobile robot bases from companies like Fetch Robotics to enable real-time inventory tracking and dynamic path planning in crowded fulfillment centers. Digital twin infrastructure now ingests heterogeneous sensor data—RGB cameras, LiDAR, radar and force-torque signals—allowing engineers to validate sensing layouts before hardware is fabricated.  NVIDIA’s Omniverse Cloud Sensor RTX stack renders physically accurate camera, radar and LiDAR returns in realtime, enabling teams to design sensing rigs and validate fusion algorithms against synthetic edge cases that would be difficult to stage on the shop floor [34]. The same simulation pipelines drive continuous calibration: when the real robot drifts, the twin highlights which sensor or actuator needs maintenance, tightening the perception–control loop and minimizing downtime.

Three architectural patterns recur across these integrated stacks. First, the sensing modality mix is converging: most production deployments now fuse at least a camera stream, a depth or range sensor, and a force-torque channel, yet the calibration and fusion pipelines remain bespoke to each integrator, creating a fragile N×M coupling between N sensor types and M task domains. Second, simulation-in-the-loop calibration is replacing periodic offline routines, but its effectiveness depends on digital-twin fidelity that few operators can verify independently. The critical open challenge is sensor-stack portability: because perception performance is validated against a specific mechanical mounting, cable routing, and timing budget, migrating a proven sensing configuration to a new robot form factor still requires months of re-integration, an obstacle that no amount of individual component improvement can overcome without standardised electromechanical and temporal interfaces.

2.3. Edge Compute and Neuromorphic Accelerators

A stark capability gap defines edge compute for Physical AI: current hardware runs classical detection and SLAM at real-time rates but cannot execute the vision-language models needed for open-ended reasoning at control frequencies physical tasks demand. On the widely deployed Jetson  Orin (275 TOPS, 60 W), YOLO-class detectors reach ~100 FPS, yet VLMs achieve only 0.1–0.4 FPS, far below the ≥10 FPS threshold for closed-loop perception [35]. Architects therefore split workloads: lightweight detectors handle the fast control loop while VLMs run asynchronously for scene under- standing, or heavy inference is offloaded to the cloud at the cost of latency and reliability. Matching model architectures to heterogeneous accelerators is thus a first-order systems design problem, not a  standalone algorithmic choice.

The next generation of edge silicon aims to close this gap. NVIDIA’s Jetson Thor (~2 000 TOPS, 130 W), shipping since August 2025, integrates a transformer engine sized to run multiple VLA policies concurrently on a mobile base [35,36]; OEMs including Boston Dynamics, Amazon Robotics, Figure, and Agility Robotics are early adopters [35]. Qualcomm’s RB5 (15 TOPS, 15 W) targets drones and service robots where 5G offload compensates for limited on-device capacity [37]. At the opposite end of the power spectrum, neuromorphic accelerators such as Intel’s Loihi 2 achieve orders-of-magnitude energy savings on event-driven workloads (<1 W), but cannot run dense transformers, restricting their role to spiking tactile controllers and reflexive loops [38,39]. Table 2summarises what each platform tier can and cannot run today. Figure 2illustrateshow these platforms integrate with the complete Physical AI hardware stack.

Table 2. Edge AI capability gap: what today’s accelerators can and cannot run at real-time control frequencies. Specs are compressed into the platform column; the two capability columns capture the deployment reality that raw TOPS numbers obscure.

Figure 2.  Physical AI hardware and compute stack.  The six layers are read bottom-to-top:  Sensing → Edge Compute → AI Models → Middleware → Actuation → Digital Twin/Cloud. Left arrows denote the fast onboard control loop (1–200 Hz); right arrows denote the slow cloud-based policy update loop (daily/nightly). Colour coding distinguishes layer function; labels within each layer list representative platforms.

2.4. Simulation Infrastructure and Digital Twins

Digital twins are now central to both hardware design and fleet operations; Section 3.5 provides the full treatment. The critical hardware constraint is a fidelity–scalability trade-off: high-physics- fidelity twins that model contact dynamics and sensor noise demand substantial GPU compute, limiting most organisations to twinning individual work cells rather than entire facilities [40].  A second under-addressed problem is twin drift: as hardware wears and layouts shift, the virtual replica diverges from reality, yet no standardised metric exists to quantify when a twin’ssafety guarantees  have degraded below an acceptable threshold [34].

2.5. Middleware: ROS 2 and Alternatives

ROS 2 is now viable as fleet-scale middleware: its ros2_control framework supports over 50 platforms [41], and containerised deployment enables cloud-native fleet management [42]. However, it cannot yet replace proprietary hard-real-time stacks for sub-millisecond control loops such as force-controlled insertion.  Most production robots therefore run a dual-stack architecture:  ROS 2 for perception, planning, and fleet orchestration atop a vendor-locked real-time bus for actuation, with an impedance mismatch at the boundary that complicates debugging, certification, and latency accounting [43]. Closing this gap, either by pushing deterministic guarantees into the open stack or by standardising narrow bridge interfaces between the real-time and best-effort domains, remains a first-order middleware challenge.

2.6. Hardware-Accelerated Perception Pipelines

GPU-accelerated perception libraries such as NVIDIA’s Isaac ROS [44,45] provide CUDA- accelerated SLAM and pose estimation integrated with ROS 2, achieving up to 10 × throughput via zero-copy transfer [46].  Vendor-agnostic alternatives—Intel OpenVINO, Google Intrinsic, and community packages such as MoveIt—remain critical because NVIDIA dominates multiple layers of the Physical AI stack (simulation, edge compute, perception, world models), creating supply-chain fragility when a single vendor’s roadmap changes; Section 8 discusses these concentration risks in detail.

3. World Modeling and Reasoning

The sensors and compute platforms described in previous sections generate vast streams of multimodal data—point clouds from LiDAR, RGB-D images, force-torque readings, and proprioceptive measurements—at rates exceeding gigabytes per second. Yet raw data alone does not enable intelligent action. A robot must synthesize these observations into structured internal representations: maps that encode navigable space, object models that capture affordances and semantics, and physics models that predict the consequences of actions. This transformation from perception to understanding is the domain of world modeling.

World modeling encompasses mapping, localization, intuitive physics, and semantic under- standing, combining classical probabilistic methods with foundation models to build actionable representations for planning and decision-making in dynamic environments.

3.1. Mapping and Localization

Simultaneous Localization and Mapping (SLAM) algorithms construct a map of the environment while estimating the robot’s pose within it. Classic SLAM approaches combine LiDAR, cameras and  inertial measurements using probabilistic filters such as extended Kalman filters and particle filters [47]. These methods excel at incrementally building consistent maps for navigation tasks. Modern visual SLAM systems leverage deep neural networks to jointly estimate depth and camera pose, achieving real-time performance even in challenging lighting conditions [48,49].  ORB-SLAM3 [48] supports  monocular, stereo, RGB-D, and visual-inertial configurations, making it versatile for warehouse mobile robots and delivery drones. DROID-SLAM [49] uses learned optical flow and bundle adjustment to surpass classical methods on difficult sequences. Radar-based SLAM is gaining traction in industrial settings because radio waves penetrate dust, fog, and smoke [50], enabling reliable navigation in factories and warehouses where visibility varies.

3.2. 3D Scene Understanding and Semantic Mapping

Beyond geometry, robots must recognize objects, affordances and semantics to interact intelligently with their environment. Vision-language models (VLMs) such as CLIP [14] allow robots to ground natural-language instructions in perception by learning visual concepts from text supervision. GPT-4V [51] extends this capability to complex scene understanding, enabling queries like “find the redbin on the top shelf” without task-specific training. Recent work such as ConceptFusion [52] fuses CLIP features into 3D semantic maps, supporting open-vocabulary queries in realtime. Foundation VLA models such as π0 [53] take this further by integrating VLM perception directly with motor control, enabling cross-embodiment manipulation from text prompts (Section 4). In warehouse environments, Fetch Robotics uses VLM-based semantic mapping to identify and pick novel SKUs without retraining, adapting to inventory changes on the fly. Similarly, BMW employs semantic bin picking systems that combine RGB-D depth with VLM scene understanding to achieve human-level versatility in assembly tasks.

3.3. Intuitive Physics and Causal Reasoning

Physical intelligence requires an understanding of how forces, friction and dynamics affect ob- jects. Traditional robotics relies on explicit physics engines such as MuJoCo [13] and PyBullet [54] for simulation and control. MuJoCo provides efficient contact simulation using a convex approximation that enables fast, stable computation, though its soft contact model simplifies certain physical phe- nomena such as material deformation history and complex friction dynamics. PyBullet’s open-source accessibility has made it a staple for research prototyping. Deep learning methods can learn implicit representations of physics from data without hand-crafted models [55]. Graph neural networks, for instance, predict object interactions and trajectory outcomes purely from visual observations. Liquid Time-constant Networks (LTCs) [56], developed by Hasani, Lechner, Amini, Rus, and Grosu across  MIT, IST Austria, and TU Wien, are time-continuous neural models that adapt their dynamics based on  input, enabling generalization to new environments and conditions. In drone navigation experiments, liquid network-powered systems demonstrated robust generalization to previously unseen seasonal and geographic conditions [57], outperforming standard recurrent neural networks.  Manufactur- ing applications leverage predictive physics models to plan manipulation sequences for deformable objects—such as cable routing or fabric handling—where classical rigid-body assumptions fail.

3.4. Planning and Decision-Making

World models feed into planning algorithms that determine how to achieve goals.  Classical planning uses graph search over discrete states; for example, A* [58] finds optimal paths in known  environments by combining cost-to-come and heuristic estimates. Sampling-based methods such as  Rapidly-exploring Random Trees (RRT) [59] excel in high-dimensional continuous spaces, making them  ideal for robot arm motion planning. Trajectory optimization approaches like CHOMP [60] refine paths by minimizing a cost functional that balances smoothness, obstacle avoidance and dynamics constraints. Model-based reinforcement learning [61] leverages learned dynamics to plan in continuous spaces, simulating candidate actions internally before execution. Hierarchical approaches [62] decompose long-horizon tasks into subgoals; for example, a high-level policy might decide that a box should be moved from a shelf to a table, while a low-level controller handles the grasping and transport maneuvers.

Each category trades off interpretability, sample efficiency, and generalization. Combining the strengths of different planners—for instance, using a high-level vision-language planner to propose candidate behaviors and an optimization-based planner to refine trajectories—remains an active research direc- tion. In warehouse settings, Amazon uses hierarchical planning to coordinate fleet-wide robot traffic while individual robots run local optimization for obstacle avoidance. Figure 3illustrateshow these planning components integrate with perception, world modeling, and action execution in a complete Physical AI pipeline.

Table 3. Comparison of planning approaches for Physical AI systems. Each method offers distinct trade-offs for interpretability, computational cost, and generalization.

Figure 3. World modeling pipeline for Physical AI systems. Arrows denote data flow from sensory input (left) through four complementary representation types—geometric (maps, point clouds), semantic (scene graphs, VLM features), physics-based (dynamics models, causal graphs), and digital twin (virtual replicas, fleet-level simulation)—to planning and action execution (right). The gear icon denotes physics-based reasoning (dynamics engines); the cloud/loop icon denotes the digital twin layer (simulation and fleet optimisation). Feedback arrows show how execution outcomes refine each representation.

3.5. Simulation and Digital Twins

Simulation remains essential for validating robot behaviors at scale and producing synthetic training data without physical risk or cost.  Modern digital twin pipelines combine photorealistic rendering, accurate physics, and tight integration with robotics middleware. NVIDIA Isaac Sim 5.0 [64], released as open source in 2025 and built atop the Omniverse platform, exemplifies this approach: USD-based scene composition [65] allows CAD designers, controls engineers, and ML practitioners to collaborate on a single source of truth, while PhysX [66] supplies contact-rich dynamics and native ROS 2 Jazzy interfaces ease deployment. Amazon Robotics maintains operational twins of fulfilment centres for rehearsing workflows before fleet-wide rollout [67]; deployment outcomes are discussed in Section 7.

Siemens’ Xcelerator platform provides comprehensive digital twins that integrate mechanical design, electrical systems, and multi-physics simulation with robotics [68]. The platform incorporates physics-based human models for safety simulation, enabling virtual commissioning of factory systems before physical deployment.  In 2024, Siemens expanded its partnership with NVIDIA, certifying industrial PCs for GPU-accelerated automation tasks from robotics to quality inspection [69].

Academic simulators complement these commercial stacks: Habitat 3.0 [70] targets indoor embod- ied tasks with dynamic human avatars, MuJoCo [13] and CARLA [71] remain standard benchmarks for contact-rich manipulation and autonomous driving respectively, and open tools such as Gazebo [72] and PyBullet [54] lower the entry barrier. Across all platforms, key trends include differentiable physics for gradient-based learning, libraries of “SimReady” assets encoding physical materials, and tight coupling with dataset curation pipelines so that real-world logs continuously refine the twin.

Three architectural fault lines emerge across this design space. The first is a fidelity–throughput trade-off : platforms that invest in photorealistic rendering and multi-physics coupling achieve high visual and dynamic realism but struggle to generate the millions of rollouts per hour that reinforce- ment learning demands, whereas lightweight contact engines sacrifice material accuracy for speed. The second is a scope–openness tension: vertically integrated commercial stacks provide end-to-end workflows from CAD import through synthetic data export, yet their proprietary scene graphs and asset formats raise lock-in concerns; conversely, open-source engines offer extensibility and repro- ducibility but fragment the toolchain, requiring researchers to stitch together rendering, physics, and  middleware bridges themselves. The third is a single-agent–fleet gap: most simulators model one robot  interacting with a static or scripted environment, but real-world deployments require coordinating dozens of heterogeneous agents sharing dynamic spaces with humans. Notably absent is a principled sim-to-real feedback loop: no existing simulator automatically quantifies its own reality gap, ingests field-deployment logs, and re-calibrates its physics, rendering, or traffic models without manual effort.

Differentiable rendering enables a complementary class of inverse problems: optimising scene  parameters (geometry, materials, lighting) through gradient descent to match target observations. Sys- temssuch as Mitsuba 3 [73] support physically accurate light transport with automatic differentiation, enabling three key Physical AI use cases: synthetic data generation with accurate reflectance, scene reconstruction that recovers material properties classical SLAM cannot, and sim-to-real transfer by gradient-based minimisation of the visual gap between rendered and real sensor observations [74,75]. As differentiable physics and rendering mature, their integration promises to accelerate sim-to-real gap closure across the pipeline.

3.6. World Foundation Models

World foundation models (WFMs) extend the capabilities of Physical AI systems by providing predictive models of how the world evolves under different actions. Rather than relying solely on hand-engineered physics or low-level policies, WFMs offer a learned “imagination” layer that allows robots to forecast the consequences of candidate plans, evaluate safety, and adapt to new environments with limited fine-tuning. These models complement the perception and control stacks by supplying long-horizon reasoning signals grounded in multimodal data.

NVIDIA’s Cosmos platform [76] exemplifies deployableWFMs, combining action-conditioned video generation, sim-to-real domain adaptation (boosting navigation success from 54% to 91% in hybrid pipelines [77]), and physics-grounded commonsense reasoning [78]. Google’s Genie 2 pursues similar goals with interactive environment generation from single images.

However, a critical “Gen2Real gap” persists across all current WFMs: generative models produce statistically plausible but physically inconsistent outputs—objects vanish under occlusion, gravity is violated, and scaling model size improves visual fidelity without reliably improving physical correctness [76,77].  Hybrid pipelines that combine generative augmentation with physics-based simulation are emerging as the practical response.

The Genesis simulator [79] represents a complementary approach: an open-source generative physics engine combining learned generative models with physics-based simulation.  This hybrid neuro-symbolic approach aims to reduce hallucinations in world models while maintaining genera- tive diversity, enabling safer deployment of predictive models in safetycritical applications such as warehouse automation and collaborative robotics. As WFMs mature, the integration of predictive world modeling with system-level digital twins—such as discrete-event simulators used in manufacturing—will enable end-to-end optimization of robot fleet deployments, from layout design to failure recovery strategies.

3.7 Systems-Level Simulation and Operations Optimization

At the plant level, discrete-event simulation platforms such as Simio [80] complement robotics- specific simulators by modelling whole-plant workflows. Procter & Gamble’s “Control Tower” virtual twin reduced deadhead truck movements by approximately 15% [81], illustrating how coupling fleet- level digital twins with robot-level simulation enables end-to-end optimisation from layout design to failure recovery. AsWFMs mature, the next frontier is real-time predictive control: running inference at 10+ Hz during deployment so that model-predictive loops can forecast the consequences of candidate  actions milliseconds before execution.  The critical barrier remains verification—generative world models may hallucinate physically impossible outcomes, requiring hybrid architectures that pair fast physics-based simulation for short-horizon dynamics with generative models for longer-horizon semantic predictions.

4. Perception,VLMs/VLAs,and Generalist Policies

World models provide robots with structured representations of spatial layout, object properties, and physical dynamics—the geometric and mechanical understanding necessary for navigation and manipulation. Yet this traditional approach relies on explicit state representations and hand-crafted perception pipelines that struggle with open-ended reasoning: How should a robot respond to the instruction "tidy the kitchen"? What does "fragile" imply about grasping strategy? Which object is "the red one on the left" when multiple candidates exist?  Answering such questions requires not just geometric understanding but semantic reasoning grounded in language and visual context. This capability emerges from vision-language-action models that unify perception, language understanding, and control within a single neural architecture.

Large language and vision-language models are increasingly embedded throughout robotics stacks: they translate natural-language goals into symbolic procedures, ground perception into action, and even generate code that stitches together traditional controllers. By tapping internet-scale priors, these models move robots beyond scripted behaviours toward systems that communicate, interpret intent, and adapt in situ. The same flexibility, however, introduces new risks around hallucinated plans, unsafe tool use, or brittle performance when the model faces embodiments it has never seen.

Addressing these challenges requires tight integration with the components discussed in earlier sections—world models,digital twins, and safety monitors. Rich simulation environments supply the counterfactual experience necessary to fine-tune large models, while runtime guardrails constrain their suggestions to verifiable behaviours. Within this section we review three complementary trends: language models for planning, multimodal models that join perception and action, and generalist policies trained across diverse robot fleets. For a comprehensive architectural survey of VLA models covering vision encoders, language models, action decoders, and training methodologies, we refer readers to Kawaharazuka et al.  [26], which provides systematic analysis of modality integration techniques and cross-embodiment learning paradigms.

4.1. Perception Foundations

Before language can ground action, robots must parse raw sensor streams into structured scene  representations. Three developments have reshaped robotic perception. First, open-vocabulary detection  and segmentation: foundation models such as Segment Anything (SAM) [82] and Grounding DINO [83] enable robots to localise and segment arbitrary objects from natural-language queries without task- specific training, replacing hand-crafted detection pipelines. Second, 3D scene representations: neural radiance fields and 3D Gaussian splatting now serve as compact, differentiable world representations for manipulation planning; Distilled Feature Fields (F3RM) [84] embed CLIP features into NeRF volumes for few-shot 6-DOF grasping, while Gaussian splatting variants enable real-time navigation  with safety guarantees [85].  Third, point cloud backbones:  PointNet++ [86] remains a widely used architecture for 3D object recognition in manipulation, though transformer-based alternatives are rapidly closing the accuracy gap. These perception modules provide the visual grounding that downstream VLA models rely on for open-ended task execution.

4.2. LLMs for Robot Planning

One of the most profound shifts in robotics is the integration of large language models (LLMs) into the decision-making loop. Classical controllers were hard-coded or trained on narrow datasets; in contrast, LLM-based planners exploit broad commonsense priors to reason about new tasks and environments. Early experiments such as PIGLeT combined text-driven reasoning with simulated household environments, showing that language-conditioned agents can acquire intuitive physics and object semantics without exhaustive manual programming.

PaLM-SayCan [15] popularised the idea of pairing a pre-trained language model with a grounded affordance model. The LLM decomposes a natural-language goal into candidate high-level skills, while the affordance model evaluates whether the robot can execute each skill given its sensors, actuators, and current state. Filtering proposed plans through feasibility scores helps maintain safety—commands that would violate kinematic limits or required preconditions are discarded before reaching low-level controllers. Variants of this architecture now support long-horizon task planning, hierarchical control, and interactive replanning as the environment changes.

Interactive dialogue and code as action.

Another branch of work treats the LLM as a just-in-time programmer. Systems such as Code-as- Policies [16] prompt the model with API documentation and a textual task description, allowing it  to emit Python or ROS code that stitches together existing primitives. Chain-of-thought prompting improves reliability by forcing the LLM to articulate intermediate reasoning and safety checks. Human overseers can inspect, edit, or veto generated code before execution, creating a collaborative loop in which the model proposes novel behaviours while operators retain ultimate authority. Although these pipelines remain semi-autonomous, they point toward robots that can extend their own repertoires on demand by composing libraries of perception, planning, and control routines.

4.3. Vision-Language-Action Models

Vision-language models (VLMs) address the grounding problem in robot planning by integrating visual perception with language understanding, enabling more accurate and context-aware control. OpenAI’s GPT-4V [51] demonstrates vision-language capabilities in embodied settings: researchers have shown that GPT-4V can interpret visual scenes and generate natural language instructions translatable into robot actions through code-as-policies frameworks. The Robotic Vision-Language  Planning (ViLa) approach [87] leverages GPT-4V to generate sequences of actionable steps based on visual observations and high-level language instructions, integrating perceptual information directly into reasoning and planning.  PaLM-E [18], an embodied multimodal language model, integrates the PaLM LLM with multiple sensor encoders—including Vision Transformers (ViT),Object Scene  Representation Transformers (OSRT), and state estimation vectors—enabling rich scene interpretation  and natural language instruction generation for downstream controllers.

Vision-Language-Action (VLA) models represent a further evolution, unifying perception, in- struction, and control within a single architecture. Google’s Robotics Transformer series exemplifies this progression: RT-1 [88] demonstrated single-task learning from demonstrations, while RT-2 [17] extended the approach by co-training on web-scale visual and textual data alongside robotic trajectory data. This integration enables RT-2 to generalize to unseen tasks and objects with minimal fine-tuning, leveraging analogies from internet-scale priors to execute novel instructions without explicit training. Open-source efforts have accelerated VLA research: OpenVLA-7B [89] provides a 7-billion parameter model achieving strong cross-embodiment performance; Octo [90] explores generalist policies for manipulation; RDT-1B [91] introduces diffusion-based action prediction for smooth trajectory synthe- sis; ManipLLM [92] and ReasonManip [93] investigate object-centric manipulation and multimodal reasoning. These systems demonstrate that VLA architectures can follow natural instructions and adapt to new environments through efficient fine-tuning on limited robot-specific data.

4.4. Generalist Agents and Cross-Embodiment Policies

The transition from task-specific controllers to generalist agents marks a fundamental shift in robotics. DeepMind’s Gato [20] provided early evidence by training a single 1.2B-parameter trans- former on 604 tasks spanning text, images, and robot control. Despite modest per-task performance, Gato demonstrated that cross-modal transfer—visual understanding improving motor control—is achievable when training diversity is sufficient.

Building on this foundation, Physical Intelligence introduced π0 (pi-zero) as a robot foundation model specifically designed for continuous control across multiple embodiments [19]. The architecture employs a flow matching approach built upon PaliGemma (3 billion parameters) augmented with 300 million additional parameters for continuous action generation at 50 Hz. Unlike discrete action models that output categorical decisions, π0 ’s flow matching enables smooth, real-time trajectory generation essential for dexterous manipulation. Training on data from seven robotic platforms across 68 unique tasks—including autonomous laundry folding and cardboard box assembly—the model demonstrates cross-embodiment transfer: skills learned on one robot morphology transfer to different hardware configurations without architecture modification. The February 2025 open-source release of π0 and its successor π0.5 democratized access to state-of-the-art robotic intelligence, with π0.5 extending generalization capabilities to entirely novel environments not represented in the training distribution [94].

NVIDIA’s GR00T N1 [95] targets humanoid embodiments specifically: a 3B-parameter model combining a frozen Eagle vision-language backbone with a flow-matching action transformer, trained on a mixture of real humanoid demonstrations, synthetic trajectories from the GR00T-Dreams simu- lation blueprint, and internet-scale video. The successor GR00T N1.5 triples task success rates over N1 on the DreamGen benchmark (38.3% vs. 13.1% across 12 manipulation tasks),demonstrating that  synthetic data generated in Isaac Sim and augmented with Cosmos Transfer can substantially reduce the months-long data-collection campaigns that have historically bottlenecked humanoid learning.

Figure AI’s Helix [96] addresses the latency-capability trade-off through a dual-system archi- tecture: System 2 (a 7B VLM at 7–9 Hz) handles scene understanding and planning, while System 1 (avisuomotor policy at 200 Hz) generates precise whole-body motor commands. This decoupling enables zero-shot manipulation of novel objects from natural language prompts while running entirely on embedded GPUs—a prerequisite for economically viable humanoid deployment. Helix-powered robots are piloting material handling at BMW Spartanburg, demonstrating industrial readiness.

Google DeepMind’s Gemini Robotics [97], built on Gemini 2.0, is the broadest VLA platform reported to date, doubling prior VLA performance on generalisation benchmarks.  Its successor, Gemini Robotics 1.5 [98], introduces transparent reasoning that surfaces intermediate decision steps, cross-embodiment motion transfer that enables skills learned on one platform to transfer to different hardware, and an Embodied Reasoning (ER) variant providing state-of-the-art spatial understanding. Partnerships with Boston Dynamics, Apptronik, and Agility Robotics signal rapid transition from research to commercial deployment across leading hardware platforms.

These developments indicate convergence toward unified architectures that integrate perception, reasoning, and action.  The success of dual-system architectures—combining fast reactive control with slower deliberative reasoning—suggests this paradigm may become dominant for general- purpose robotics, analogous to how transformer architectures unified natural language processing. Figure 4illustrates the evolution of VLA model architectures from early single-task systems through current generalist agents, showing the progression in model scale, training data diversity, and control capabilities that enable cross-embodiment deployment. Table 4provides a systematic architectural comparison across these models, revealing the design choices that differentiate each system.

Figure 4. Evolution of Vision-Language-Action model architectures from 2022 to 2025, illustrating the progression from single-task systems to generalist agents capable of cross-embodiment control. The timeline shows increasing model scale, training data diversity, and control sophistication enabling industrial deployment.

Table 4. Architectural comparison of representative VLA models (2022–2025), ordered chronologically within three development phases. The table highlights the progression from discrete-action, single-embodiment systems to continuous-control, cross-embodiment generalist agents. Freq. denotes effective control frequency; Emb. is the number of distinct robot morphologies used during training.

Design tradeoff analysis.

Three architectural trends emerge from this comparison. First, the action decoder has evolved from discrete tokenisation (RT-1/2, OpenVLA) through diffusion models (Octo, RDT-1B) to flow matching (π0, GR00TN1): discrete tokens leverage pretrained language-model weights but cap control bandwidth at 3–6 Hz; diffusion enables multi-modal action distributions at the cost of inference latency; flow matching provides continuous, low-latency generation at 50+ Hz. Second, dual-system architectures have emerged independently in Helix and GR00T N1, decoupling slow deliberative reasoning (VLM at 7–10 Hz) from fast reactive control (visuomotor loop at 120–200 Hz)—an echo of the System 1/System 2 distinction from cognitive science that appears necessary for contact-rich manipulation where millisecond-level responses are critical. Third, parameter efficiency consistently outweighs raw scale: OpenVLA (7B) surpasses RT-2-X (55B) with 7× fewer parameters, while Octo (93M) provides competitive generalisation at a fraction of the compute, suggesting that training data diversity across embodiments matters more than model size alone. Notably, six of the ten models are fully open-source, enabling rapid community iteration and lowering the barrier to entry for new research groups.

4.5. Benchmarking and Evaluation

Systematic benchmarking across 20 Open-X-Embodiment datasets reveals that parameter ef- ficiency outweighs raw scale:  OpenVLA (7B) outperforms RT-2-X (55B) by 16.5% with 7× fewer parameters and exceeds Diffusion Policy by 20.4% [89,99]. GPT-4o achieves the most consistent cross- task performance through prompt engineering, but all models struggle with tasks requiring multi-step planning or adaptation to significant environmental changes [99]. The field has grown from a handful  of VLA systems to over 45 specialised implementations between 2022–2025 [28], yet standardised  metrics for safety, interpretability, and failure recovery remain underdeveloped—aspects critical for real-world deployment but underrepresented in current evaluation frameworks.