Robot Programming
Assembly Line
Teaching a robot its limits, to complete open-ended tasks safely
To guide robots in executing these open-ended tasks, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) used vision models to see what’s near the machine and model its constraints. The team’s strategy involves an LLM sketching up a plan that’s checked in a simulator to ensure it’s safe and realistic. If that sequence of actions is infeasible, the language model will generate a new plan, until it arrives at one that the robot can execute.
This trial-and-error method, which the researchers call “Planning for Robots via Code for Continuous Constraint Satisfaction” (PRoC3S), tests long-horizon plans to ensure they satisfy all constraints, and enables a robot to perform such diverse tasks as writing individual letters, drawing a star, and sorting and placing blocks in different positions. In the future, PRoC3S could help robots complete more intricate chores in dynamic environments like houses, where they may be prompted to do a general chore composed of many steps (like “make me breakfast”).
The researchers’ method uses an LLM pre-trained on text from across the internet. Before asking PRoC3S to do a task, the team provided their language model with a sample task (like drawing a square) that’s related to the target one (drawing a star). The sample task includes a description of the activity, a long-horizon plan, and relevant details about the robot’s environment.
Making Robot Programming User Friendly with the Workbench for Offline Robotics Development (SWORD)
Over the years, the ROS-I Consortium has held frequent roadmapping sessions with a wide variety of end users and ROS developers, to address ease-of-use and continuing education. The identified need is a lower barrier of entry for non-programmers (or entry-level developers) to harness the power of tools in the ROS ecosystem, but in a way that aligns with industry adoption of digital thread and industry 4.0 strategies. The traditional ROS workflow is software programming intense, requiring developers deeply familiar with available ROS libraries and tools. Even experienced developers within the ROS-I ecosystem, and beyond, may spend significant time (days to weeks) on the initial setup and configuration of a ROS application. Listening to the voice of our own developers, our diverse stakeholders, and consortium members, we heard the need for easier access to the ROS motion planning tools, while maintaining a tie back to the CAD ecosystem where the products to be worked on are conceived and maintained.
SwRI is launching the SwRI Workbench for Offline Robotics Development (SWORD)™ featuring a graphical toolkit for developing and testing advanced robotic motion-planning applications. SWORD is implemented as a plugin to the open-source FreeCAD application, allowing users to integrate robotics capabilities into a cross-platform CAD environment. It provides a graphical interface to many powerful motion-planning libraries. The goal is to bring ROS to a manufacturing/industrial audience in a way that is more approachable and resides in an environment that is familiar. Most manufacturing engineers are competent with CAD and understand their processes, often doing various forms of programs on process-oriented systems. SWORD seeks to bring advanced motion-planning capability to this audience enabling to set up their systems and take advantage of these more advanced tools in their operational environments.