Large behavior model
Assembly Line
Connecting the Dots: How AI Can Make Sense of the Real World
Working with Infineon, a global semiconductor manufacturer and leader in sensing and IoT, we are exploring how such powerful human-like functions can be developed and deployed in real-world applications using generative physical AI models like Newton. These models seamlessly integrate real-time events captured by simple, ubiquitous sensors — such as radars, microphones, proximity sensors, and environmental sensors — with high-level contextual information to generate rich and detailed interpretations of real-world behaviors. Importantly, this is achieved without requiring developers to explicitly define such interpretations or relying on complex, expensive, and privacy-invasive sensors like cameras.
Generative physical AI models, such as Newton, are able to overcome these challenges for the first time, unlocking a boundless range of applications. We explored Newton’s ability to interpret real-world context and human activities by combining radar and microphone data. In our demo scenarios, Newton powers a home assistant in a kitchen setting, helping a user through their morning routine in one situation and in another helping to keep residents safe when the smoke alarm goes off.
When fused with additional contextual data — such as location, time, day of the week, weather, news, or user preferences — Newton can provide personalized and relevant recommendations or services. This capability makes it possible to go beyond basic sensor interpretations, offering meaningful insights tailored to the needs of individual users or organizations.
Infineon to pilot new AI developer model by Archetype AI to enhance AI sensor solution innovation
Infineon Technologies AG and Archetype AI, Inc. announced a strategic partnership to accelerate the development of sensor-based chips with AI functionalities. Archetype AI’s Large Behavior Model (LBM) will be piloted by Infineon to uncover hidden patterns in unstructured sensor data and create a living view of the world. The partnership will enable Infineon to generate AI agents that automatically create code for customer-specific sensor use cases, making devices like TVs, smart speakers, and smart home appliances more aware of their surroundings. This collaboration aims to advance decarbonization and digitization, and is expected to have a significant impact on various industries, including automotive, consumer electronics, and healthcare.
Under the multi-year partnership, Infineon will be the first company to utilize the LBM AI developer platform to generate AI agents that automatically create code for customer-specific sensor use cases to run as edge models on customer devices. Sensors built by Infineon with help of the LBM platform make devices like TVs, smart speakers, and smart home appliances aware of people and the world around them. Devices can automatically wake and surface information at the right time without interrupting, users can control devices with gestures, and devices can turn-on and off based on when people are around which reduces the use of energy and contributes to decarbonization.
Toyota Research Institute Unveils Breakthrough in Teaching Robots New Behaviors
The Toyota Research Institute (TRI) announced a breakthrough generative AI approach based on Diffusion Policy to quickly and confidently teach robots new, dexterous skills. This advancement significantly improves robot utility and is a step towards building “Large Behavior Models (LBMs)” for robots, analogous to the Large Language Models (LLMs) that have recently revolutionized conversational AI.
TRI has already taught robots more than 60 difficult, dexterous skills using the new approach, including pouring liquids, using tools, and manipulating deformable objects. These achievements were realized without writing a single line of new code; the only change was supplying the robot with new data. Building on this success, TRI has set an ambitious target of teaching hundreds of new skills by the end of the year and 1,000 by the end of 2024.