Physical AI
Assembly Line
When we kicked off @MachinaLabs_ to build a robotic craftsman, we knew it wasn’t just about building a robot; it was about building a mind of the craftsman. Two major challenges stood in our way:
— Edward Mehr (@EdwardMehr) December 19, 2024
1- Dexterity: A robot needs to handle diverse manufacturing operations like a human… https://t.co/WxTal2qVco pic.twitter.com/wIMemTm2VH
Connecting the Dots: How AI Can Make Sense of the Real World
Working with Infineon, a global semiconductor manufacturer and leader in sensing and IoT, we are exploring how such powerful human-like functions can be developed and deployed in real-world applications using generative physical AI models like Newton. These models seamlessly integrate real-time events captured by simple, ubiquitous sensors — such as radars, microphones, proximity sensors, and environmental sensors — with high-level contextual information to generate rich and detailed interpretations of real-world behaviors. Importantly, this is achieved without requiring developers to explicitly define such interpretations or relying on complex, expensive, and privacy-invasive sensors like cameras.
Generative physical AI models, such as Newton, are able to overcome these challenges for the first time, unlocking a boundless range of applications. We explored Newton’s ability to interpret real-world context and human activities by combining radar and microphone data. In our demo scenarios, Newton powers a home assistant in a kitchen setting, helping a user through their morning routine in one situation and in another helping to keep residents safe when the smoke alarm goes off.
When fused with additional contextual data — such as location, time, day of the week, weather, news, or user preferences — Newton can provide personalized and relevant recommendations or services. This capability makes it possible to go beyond basic sensor interpretations, offering meaningful insights tailored to the needs of individual users or organizations.
Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators
Neural operators, serving as physics surrogate models, have recently gained increased interest. With ever increasing problem complexity, the natural question arises: what is an efficient way to scale neural operators to larger and more complex simulations - most importantly by taking into account different types of simulation datasets. This is of special interest since, akin to their numerical counterparts, different techniques are used across applications, even if the underlying dynamics of the systems are similar. Whereas the flexibility of transformers has enabled unified architectures across domains, neural operators mostly follow a problem specific design, where GNNs are commonly used for Lagrangian simulations and grid-based models predominate Eulerian simulations. We introduce Universal Physics Transformers (UPTs), an efficient and unified learning paradigm for a wide range of spatio-temporal problems. UPTs operate without grid- or particle-based latent structures, enabling flexibility and scalability across meshes and particles. UPTs efficiently propagate dynamics in the latent space, emphasized by inverse encoding and decoding techniques. Finally, UPTs allow for queries of the latent space representation at any point in space-time. We demonstrate diverse applicability and efficacy of UPTs in mesh-based fluid simulations, and steady-state Reynolds averaged Navier-Stokes simulations, and Lagrangian-based dynamics.
US researchers develop speedy AI solver for engineering problems
Researchers from John Hopkins University have developed an AI that enables powerful modeling and physics simulations to be solved in seconds using desktop computers.
The DIMON (Diffeomorphic Mapping Operator Learning) AI has so far been used to model human hearts and predict the often-fatal condition of cardiac arrhythmia, reducing the time it takes to recommend treatments from a week to seconds. DIMON uses a generic approach to quickly predict solutions to partial differential equations, a process which up until now has been a time-consuming and computationally intense process.
The researchers have shown the same AI can be used to solve complex engineering problems, such as modeling how cars deform in a crash, how spacecraft respond to extreme environments, or how bridges resist stress. DIMON uses AI to understand how physical systems behave across different shapes, without needing to recalculate everything from scratch for each new shape. Instead of dividing shapes into grids and solving equations over and over, the AI predicts how factors such as heat, stress, or motion will behave based on patterns it has learned, making it much faster and more efficient in tasks like optimizing designs or modeling shape-specific scenarios.
Building beyond human imagination with foundation models for geometry and physics
As much as physics simulation is a bottleneck to better engineering, so too is human imagination. To build beyond, our researchers have created a foundation model for geometry that understands how any 3D shape relates to any other, and knows what lies between an aircraft wing and a feathered bird’s wing.
Our latest Large Geometry Model, LGM-Aero, has 100M parameters (approximately as many as OpenAI’s GPT1) and has seen 25M diverse 3D shapes. Our foundation model has already been deployed in real-world customer challenges, outperforming current state-of-the art engineering workflows.
The typical process for engineering physical parts starts with engineers using computer aided design (CAD) to create a precise digital schematic of the part, often with tuneable parameters such as the length of a wing or the size of an inlet. The CAD software then outputs an explicit representation (called a mesh) of the geometry, comprising vertices joined by faces. The mesh is then fed into numerical simulations – such as computational fluid dynamics (CFD) or finite element analysis (FEA) – that are used to estimate the performance characteristics of the design, allowing engineers to understand how their components would perform in the real world and identify potential issues without needing to build physical prototypes.
Engineers then iterate on the design by tuning the parameters based on the insights they gain from simulations. This process usually involves multiple rounds of testing and refinement to reach the necessary performance, requiring hundreds of hours of waiting for simulation results and months of iteration time.
With a foundation model of geometry we are setting out to endogenize the full description of the shape of an object and so make it subject to an optimization process - rather than a small number of ex ante defined parameters pertaining to some starting shape.
NVIDIA Advances Physical AI With Accelerated Robotics Simulation on AWS
NVIDIA announced at AWS re:Invent that Isaac Sim now runs on Amazon Elastic Cloud Computing (EC2) G6e instances accelerated by NVIDIA L40S GPUs. And with NVIDIA OSMO, a cloud-native orchestration platform, developers can easily manage their complex robotics workflows across their AWS computing infrastructure.
Physical AI describes AI models that can understand and interact with the physical world. It embodies the next wave of autonomous machines and robots, such as self-driving cars, industrial manipulators, mobile robots, humanoids and even robot-run infrastructure like factories and warehouses. With physical AI, developers are embracing a three computer solution for training, simulation and inference to make breakthroughs.
Newton - Zero-shot forecasting of a spring-mass system
A Framework For Efficiently Scaling Neural Operators
Universal Physics Transformers (UPTs) are a novel learning paradigm to efficiently train large-scale neural operators for a wide range of spatio-temporal problems - both for Lagrangian and Eulerian discretization schemes.
The architecture of UPT consists of an encoder, an approximator and a decoder. The encoder is responsible to encode the physics domain into a latent representation, the approximator propagates the latent representation forward in time and the decoder transforms the latent representation back to the physics domain.
On machine learning methods for physics
Simulation methods are employed to resolve the behaviour of matter (solids, fluids, gases, etc.), fields (electromagnetic, pressure, velocity, density), and any number of other physical phenomena that are driven by known local rules, particularly partial differential equations (PDEs). Therefore, traditional simulation methods typically involve some kind of discretisation of the physical domain of interest, such that the rules of the governing PDE can be locally well-approximated by a tractable computation. Local computations are stacked together and iterated upon until we converge to a solution. Beyond a narrow class of problems where a closed-form solution can be provided, this is generally how complicated problems are addressed. Many PDEs can exhibit chaotic behaviour in their full form, which often causes us to resort to simpler approximations at the PDE level, even before discretisation, to make them computationally feasible and ensure convergence.
ML methods in general provide a new approach to accomplishing engineering tasks. Any of these methods greatly accelerate iterations in the design optimisation workflow, as they allow us to search the space faster and guide our search towards promising areas. This results in better exploration, overall lower computational costs for simulating physics, and ultimately, higher quality designs in a shorter time-frame and with lower manual effort.
The models discussed so far do not leverage the fact that we often know the PDE that generates data and governs solutions; the focus has been to approximate the physical laws from observations, rather than impose them in the model structure explicitly. This is primarily because of the difficulty of incorporating such prior knowledge into the models, but also because simulation data may disobey the exact PDE due to the approximations required to facilitate numerical simulations. However, a new approach to simulation has recently been proposed that takes advantage of this prior knowledge in an effort to reduce data requirements and promote physically consistent solutions. Physics-Informed Neural Networks (PINNs), as presented by Raissi, Perdikaris, and Karniadakis (2017a, 2017b; Zhu et al. 2019; Karniadakis et al. 2021) introduces an artificial neural network (ANN) that takes as input the coordinates of any point in the domain of the PDE, and outputs the value for the solution field at that point. The ANN is tasked with representing the solution field, and is trained by sampling points randomly in the domain and penalising deviations from the PDE at those points. As long as the activation function of the ANN is sufficiently differentiable, residuals in the terms of the PDE can be easily evaluated, which can be combined into the loss function to be minimised with respect to the ANN parameters. The ANN is an ansatz about a parametrised form of the solution (albeit a particularly flexible one) and we attempt to fit the parameters such that it best matches the PDE. The idea harks back to older variational numerical simulation methods, like the generalised Galerkin approximation and others.
Infineon to pilot new AI developer model by Archetype AI to enhance AI sensor solution innovation
Infineon Technologies AG and Archetype AI, Inc. announced a strategic partnership to accelerate the development of sensor-based chips with AI functionalities. Archetype AI’s Large Behavior Model (LBM) will be piloted by Infineon to uncover hidden patterns in unstructured sensor data and create a living view of the world. The partnership will enable Infineon to generate AI agents that automatically create code for customer-specific sensor use cases, making devices like TVs, smart speakers, and smart home appliances more aware of their surroundings. This collaboration aims to advance decarbonization and digitization, and is expected to have a significant impact on various industries, including automotive, consumer electronics, and healthcare.
Under the multi-year partnership, Infineon will be the first company to utilize the LBM AI developer platform to generate AI agents that automatically create code for customer-specific sensor use cases to run as edge models on customer devices. Sensors built by Infineon with help of the LBM platform make devices like TVs, smart speakers, and smart home appliances aware of people and the world around them. Devices can automatically wake and surface information at the right time without interrupting, users can control devices with gestures, and devices can turn-on and off based on when people are around which reduces the use of energy and contributes to decarbonization.