Carnegie Mellon University (Carnegie Mellon)
Canvas Category Consultancy : Research : Academic
A private, global research university, Carnegie Mellon stands among the world’s most renowned educational institutions, and sets its own course. Over the past 10 years, more than 400 startups linked to CMU have raised more than $7 billion in follow-on funding. Those investment numbers are especially high because of the sheer size of Pittsburgh’s growing autonomous vehicles cluster – including Uber, Aurora, Waymo and Motional – all of which are here because of their strong ties to CMU.With cutting-edge brain science, path-breaking performances, innovative startups, driverless cars, big data, big ambitions, Nobel and Turing prizes, hands-on learning, and a whole lot of robots, CMU doesn’t imagine the future, we create it.
Assembly Line
HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For example, navigation relies on root velocity tracking, while tabletop manipulation prioritizes upper-body joint angle tracking. Existing approaches typically train individual policies tailored to a specific command space, limiting their transferability across modes. We present the key insight that full-body kinematic motion imitation can serve as a common abstraction for all these tasks and provide general-purpose motor skills for learning multiple modes of whole-body control. Building on this, we propose HOVER (Humanoid Versatile Controller), a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER enables seamless transitions between control modes while preserving the distinct advantages of each, offering a robust and scalable solution for humanoid control across a wide range of modes. By eliminating the need for policy retraining for each control mode, our approach improves efficiency and flexibility for future humanoid applications.
Data Fuels Robotics Research
At Mill 19, the site of the Advanced Robotics for Manufacturing Institute and Carnegie Mellon University’s Manufacturing Futures Institute, robotics data provides the foundation for AI and digital twins.
The AI Data Foundry was started in March 2024 and is intended to be “a national center that collects and aggregates data for robots working in manufacturing settings in order to accelerate and develop AI and machine- learning algorithms that are relevant to various manufacturing tasks,” Fedder explains. This data will come from robots at ARM’s facility, as well as ARM’s members, performing all sorts of tasks: machine tending, welding, grinding, polishing, painting and more.
This data will provide the foundation for algorithms that will enable robots to work with challenging workpieces (such as flexible parts) or complete complex tasks (plugging cables into sockets or screwing nuts and bolts together). These tasks are more challenging to automate because they aren’t perfectly repetitive and therefore require so much data to program that gathering enough real-world data could be prohibitive. According to Fedder, synthetic, or computer-generated data could bridge this gap. Researchers can use synthetic data for tests and validate the results with real-world data.
Skild AI Raises $300M Series A To Build A Scalable AI Foundation Model For Robotics
Skild AI, an AI robotics company building a scalable foundation model for robotics, announced it has closed a $300M Series A funding round. The round was led by Lightspeed Venture Partners, Coatue, SoftBank Group, and Jeff Bezos (through Bezos Expeditions), with participation from Felicis Ventures, Sequoia, Menlo Ventures, General Catalyst, CRV, Amazon, SV Angel, and Carnegie Mellon University. The funding brings the company to a valuation of $1.5B. The capital will be used to continue scaling the company’s model and training datasets for future commercial deployment of its technology, in addition to hiring for roles across AI, robotics, engineering, operations, and security.
Skild AI is building intelligence that is grounded in the physical world. The company is breaking the data barrier in robotics, training its model on at least 1,000X more data points than competing models. As opposed to vertically designed robots that are built for specific applications, Skild’s model serves as a shared, general-purpose brain for a diverse embodiment of robots, scenarios and tasks, including manipulation, locomotion and navigation. From resilient quadrupeds mastering adverse physical conditions, to vision-based humanoids performing dexterous manipulation of objects for complex household and industrial tasks, the company’s model will enable the use of low-cost robots across a broad range of industries and applications.
AMGPT: a Large Language Model for Contextual Querying in Additive Manufacturing
Generalized large language models (LLMs) such as GPT-4 may not provide specific answers to queries formulated by materials science researchers. These models may produce a high-level outline but lack the capacity to return detailed instructions on manufacturing and material properties of novel alloys. Enhancing a smaller model with specialized domain knowledge may provide an advantage over large language models which cannot be retrained quickly enough to keep up with the rapid pace of research in metal additive manufacturing (AM). We introduce “AMGPT,” a specialized LLM text generator designed for metal AM queries. The goal of AMGPT is to assist researchers and users in navigating the extensive corpus of literature in AM. Instead of training from scratch, we employ a pre-trained Llama2-7B model from Hugging Face in a Retrieval-Augmented Generation (RAG) setup, utilizing it to dynamically incorporate information from ∼50 AM papers and textbooks in PDF format. Mathpix is used to convert these PDF documents into TeX format, facilitating their integration into the RAG pipeline managed by LlamaIndex. Expert evaluations of this project highlight that specific embeddings from the RAG setup accelerate response times and maintain coherence in the generated text.
Electric vehicle battery chemistry affects supply chain disruption vulnerabilities
We examine the relationship between electric vehicle battery chemistry and supply chain disruption vulnerability for four critical minerals: lithium, cobalt, nickel, and manganese. We compare the nickel manganese cobalt (NMC) and lithium iron phosphate (LFP) cathode chemistries by (1) mapping the supply chains for these four materials, (2) calculating a vulnerability index for each cathode chemistry for various focal countries and (3) using network flow optimization to bound uncertainties. World supply is currently vulnerable to disruptions in China for both chemistries: 80% [71% to 100%] of NMC cathodes and 92% [90% to 93%] of LFP cathodes include minerals that pass through China. NMC has additional risks due to concentrations of nickel, cobalt, and manganese in other countries. The combined vulnerability of multiple supply chain stages is substantially larger than at individual steps alone. Our results suggest that reducing risk requires addressing vulnerabilities across the entire battery supply chain.
Chip startup Efficient Computer raises $16 million led by Eclipse
Chip startup Efficient Computer said on Thursday it had raised $16 million in a seed funding round led by Silicon Valley venture capital firm Eclipse to help fund work on its low-power chip designs. Pittsburgh-based Efficient developed a new design, or architecture, for its chips that focuses on producing processors that use the least possible amount of energy. Called Fabric, the architecture was developed by Efficient’s founding team over seven years at Carnegie Mellon University.
Efficient has built a test chip called Monza and plans to use the funding help with research and development, and go-to-market to begin to sell chips. The company will market the chips to customers in industries such as health devices, civil infrastructure monitoring, satellites, defense and security. Devices running on chips that use a tiny amount of power will last longer in the field without the need for replacement power.
CMU Robotics Institute develops system to detect and fix problems in gas pipelines
Researchers in Carnegie Mellon University’s Robotics Institute are developing a modular robot that can creep inside natural gas pipelines to map where pipes are, detect decrepit or leaking pipes, and, when necessary, repair the pipe by applying a resin coating along its inner wall.
Natural gas in the US arrives at 75 million homes and more than five million commercial customers through a network of 1.2 million miles of distribution main lines and 900,000 miles of service lines, according to the DOE. It costs up to $10 million per mile to excavate and repair these existing lines. The REPAIR program aims to use robots and smart coatings to build new pipes within leaky ones. This process – leaving the pipes in place and repairing them from the inside out – could drastically cut costs by DOE estimates.
The researchers have evaluated their system using a testbed built by Peoples Gas. The robotic system now has a 200-foot range, Li said, but the eventual goal is two kilometers (around 6,500 feet). Lu said the current version of the robot is designed for 12-inch diameter pipes and a version for 6-inch pipes is in development.
Creative Robot Tool Use with Large Language Models
We introduce RoboTool, enabling robots to use tools creatively with large language models, which solves long-horizon hybrid discrete-continuous planning problems with the environment- and embodiment-related constraints.
In this work, we are interested in solving language-instructed long-horizon robotics tasks with implicitly activated physical constraints. By providing LLMs with adequate numerical semantic information in natural language, we observe that LLMs can identify the activated constraints induced by the spatial layout of objects in the scene and the robot’s embodiment limits, suggesting that LLMs may maintain knowledge and reasoning capability about the 3D physical world. Furthermore, our comprehensive tests reveal that LLMs are not only adept at employing tools to transform otherwise unfeasible tasks into feasible ones but also display creativity in using tools beyond their conventional functions, based on their material, shape, and geometric features.
🖨️🎛️ One-Camera Method Reveals Added Insights in Additive Manufacturing
We introduce an experimental method to image melt pool temperature with a single commercial color camera and compare the results with multi-physics computational fluid dynamic (CFD) models. This approach leverages the principle of two-color (i.e., ratiometric) thermal imaging, which is advantageous because it negates the need for a priori knowledge of melt pool emissivity, plume transmissivity, and the camera’s view factor. The color camera’s ability to accurately measure temperature was validated with a National Institute of Standards and Technology (NIST) blackbody source and tungsten filament lamp between temperatures of 1600 K and 2800 K. To demonstrate the technique, an off-axis high-speed color camera operating at 22 500 frames per second capturing a 2.8 mm by 2.8 mm area on the build plate was used to image both no-powder and powder single beads on a commercial laser powder bed fusion machine. Melt pool temperature fields for 316L stainless steel at varying processing conditions show peaks between 3300 K and 3700 K depending on the laser power and increased variability in the presence of powder. Measurements of nickel superalloy 718 and Ti-6Al-4V show comparable temperatures, with increased plume obstruction, especially in Ti-6Al-4V due to vaporization of aluminum. Multi-physics CFD models are used to simulate metal melt pools but some parameters such as the accommodation and Fresnel coefficients are not well characterized. Fitting a FLOW-3D® CFD model to ex-situ measurements of the melt pool cross-sectional geometry for 316L stainless steel identifies multiple combinations of Fresnel coefficient and accommodation coefficient that lead to geometric agreement. Only two of these combinations show agreement with the thermal images, motivating the need for thermal imaging as a means to advance validation of complex physics models. Our methodology can be applied to any color camera to better monitor and understand melt pools that yield high-quality parts.
Closed-loop fully-automated frameworks for accelerating materials discovery
Our work shows that a fully-automated closed-loop framework driven by sequential learning can accelerate the discovery of materials by up to 10-25x (or a reduction in design time by 90-95%) when compared to traditional approaches. We show that such closed-loop frameworks can lead to enormous improvement in researcher productivity in addition to reducing overall project costs. Overall, these findings present a clear value proposition for investing in closed-loop frameworks and sequential learning in materials discovery and design enterprises.
UVA Research Team Detects Additive Manufacturing Defects in Real-Time
Introduced in the 1990s, laser powder bed fusion, or LPBF uses metal powder and lasers to 3-D print metal parts. But porosity defects remain a challenge for fatigue-sensitive applications like aircraft wings. Some porosity is associated with deep and narrow vapor depressions which are the keyholes.
“By integrating operando synchrotron x-ray imaging, near-infrared imaging, and machine learning, our approach can capture the unique thermal signature associated with keyhole pore generation with sub-millisecond temporal resolution and 100% prediction rate,” Sun said. In developing their real-time keyhole detection method, the researchers also advanced the way a state-of-the-art tool — operando synchrotron x-ray imaging — can be used. Utilizing machine learning, they additionally discovered two modes of keyhole oscillation.
auton-survival: An Open-Source Package for Regression, Counterfactual Estimation, Evaluation
Real-world decision-making often requires reasoning about when an event will occur. The overarching goal of such reasoning is to help aid decision-making for optimal triage and subsequent intervention. Such problems involving estimation of Times-to-an-Event frequently arise across multiple application areas, including, predictive maintenance. Reliability engineering and systems safety research involves the use of remaining useful life prediction models to help extend the longevity of machinery and equipment by proactive part and component replacement.
Discretizing time-to-event outcomes to predict if an event will occur is a common approach in standard machine learning. However, this neglects temporal context, which could result in models that misestimate and lead to poorer generalization.