Archetype AI
Assembly Line
Connecting the Dots: How AI Can Make Sense of the Real World
Working with Infineon, a global semiconductor manufacturer and leader in sensing and IoT, we are exploring how such powerful human-like functions can be developed and deployed in real-world applications using generative physical AI models like Newton. These models seamlessly integrate real-time events captured by simple, ubiquitous sensors — such as radars, microphones, proximity sensors, and environmental sensors — with high-level contextual information to generate rich and detailed interpretations of real-world behaviors. Importantly, this is achieved without requiring developers to explicitly define such interpretations or relying on complex, expensive, and privacy-invasive sensors like cameras.
Generative physical AI models, such as Newton, are able to overcome these challenges for the first time, unlocking a boundless range of applications. We explored Newton’s ability to interpret real-world context and human activities by combining radar and microphone data. In our demo scenarios, Newton powers a home assistant in a kitchen setting, helping a user through their morning routine in one situation and in another helping to keep residents safe when the smoke alarm goes off.
When fused with additional contextual data — such as location, time, day of the week, weather, news, or user preferences — Newton can provide personalized and relevant recommendations or services. This capability makes it possible to go beyond basic sensor interpretations, offering meaningful insights tailored to the needs of individual users or organizations.
Can AI Learn Physics from Sensor Data?
The challenge of understanding the physical world, whether for humans or AI, is that we cannot experience or learn about it directly — we can only observe it indirectly through sensors. These sensors might be our natural biological ones, like our eyes and ears, or the countless artificial sensors humans have invented to measure everything from acceleration to gas concentration. However, sensors always affect the outcome, distorting or obscuring the “true” physical behavior, making it harder to uncover the underlying laws that govern the physical world.
In both cases, the Newton model received real-time data from sensors and was able to accurately predict the behavior of the physical systems simply by observing the concurrent sensor data. What’s remarkable is that Newton had not been specifically trained to understand these experiments – it was encountering them for the first time and was still able to predict outcomes even for chaotic and complex behaviors. This ability of an AI model to accurately predict data it has not encountered previously is often referred to as zero-shot forecasting.
While these classic experiments were exciting, real world systems are far more complex and harder to describe. We then tested Newton on predicting the behavior of such systems as city electrical demand, daily temperature, and temperature in electrical transformers, to name a few. Figure 2 demonstrates that Newton was able to accurately zero-shot forecast the behavior of these complex systems with no additional training data for systems that are challenging even for humans to model.
Newton - Zero-shot forecasting of a spring-mass system
Archetype AI Introduces Foundation Model to Pioneer Physical AI
Archetype AI, a physical AI company helping humanity make sense of the world, announced its emergence from stealth and the introduction of Newton™, a first-of-its-kind foundation model that understands the physical world. With Newton, Archetype AI is on a mission to use the power of artificial intelligence to solve real-world problems – empowering people and organizations with an understanding of the physical environment that wasn’t previously possible.
In support of this mission, Archetype AI has raised a $13 million seed funding round led by Venrock, with participation from Amazon Industrial Innovation Fund, Hitachi Ventures, Buckley Ventures, Plug and Play Ventures and several angel investors. In conjunction with the financing, Ganesh Srinivasan, Partner at Venrock, will join the board.
With Newton, Archetype AI is introducing a first-of-its-kind physical AI foundational model that is capable of perceiving, understanding and reasoning about the world. Newton fuses multimodal temporal data – including signals from accelerometers, gyroscopes, radars, cameras, microphones, thermometers and other environmental sensors – with natural language to unlock insights about the physical world in real-time.
Infineon to pilot new AI developer model by Archetype AI to enhance AI sensor solution innovation
Infineon Technologies AG and Archetype AI, Inc. announced a strategic partnership to accelerate the development of sensor-based chips with AI functionalities. Archetype AI’s Large Behavior Model (LBM) will be piloted by Infineon to uncover hidden patterns in unstructured sensor data and create a living view of the world. The partnership will enable Infineon to generate AI agents that automatically create code for customer-specific sensor use cases, making devices like TVs, smart speakers, and smart home appliances more aware of their surroundings. This collaboration aims to advance decarbonization and digitization, and is expected to have a significant impact on various industries, including automotive, consumer electronics, and healthcare.
Under the multi-year partnership, Infineon will be the first company to utilize the LBM AI developer platform to generate AI agents that automatically create code for customer-specific sensor use cases to run as edge models on customer devices. Sensors built by Infineon with help of the LBM platform make devices like TVs, smart speakers, and smart home appliances aware of people and the world around them. Devices can automatically wake and surface information at the right time without interrupting, users can control devices with gestures, and devices can turn-on and off based on when people are around which reduces the use of energy and contributes to decarbonization.