University of Colorado
Assembly Line
ARMADA: Augmented Reality for Robot Manipulation and Robot-Free Data Acquisition
Teleoperation for robot imitation learning is bottlenecked by hardware availability. Can high-quality robot data be collected without a physical robot? We present a system for augmenting Apple Vision Pro with real-time virtual robot feedback. By providing users with an intuitive understanding of how their actions translate to robot motions, we enable the collection of natural barehanded human data that is compatible with the limitations of physical robot hardware. We conducted a user study with 15 participants demonstrating 3 different tasks each under 3 different feedback conditions and directly replayed the collected trajectories on physical robot hardware. Results suggest live robot feedback dramatically improves the quality of the collected data, suggesting a new avenue for scalable human data collection without access to robot hardware.
Textbooks come alive with new, interactive AI tool
With just an iPad, students in any classroom across the world could soon reimagine the ordinary diagrams in any physics textbook—transforming these static images into 3D simulations that run, leap or spin across the page. These new, living textbooks are the brainchild of a team of computer scientists led by Ryo Suzuki at CU Boulder.
“Usually, those diagrams are fixed. We have to imagine what happens,” said Suzuki, assistant professor in the ATLAS Institute and Department of Computer Science. “But what if we could take any static diagram from any textbook and make it interactive?”
He and his colleagues recently took home a “best paper” award for their work at the 37th Annual ACM Symposium on User Interface Software and Technology this October in Pittsburgh.
The tool relies on a model called Segment Anything from the tech company Meta. It’s a computer visualization tool that allows users to click on a photo to isolate particular objects—a dog, or maybe a face. Similarly, through Augmented Physics, students and teachers select various objects inside a diagram, such as the skier and the ski jump, and assign those objects roles. The AI then applies some basic physics, such as the force of gravity, to make those objects move.