Robots learn to perform tasks faster with less human input with smart framework

UC Berkeley researchers have developed an AI-driven approach that could significantly accelerate how robots are programmed and trained for real-world tasks. Called agentic robotics, the approach uses teams of AI agents to generate robot-control programs, test them in simulated environments, and refine their performance. The method could reduce the need for extensive training data and manual engineering, two major challenges that have slowed the development of capable robots. Early trials have demonstrated improvements in robotic task performance, pointing to potential applications in industrial automation and warehouse operations. Robots learn through AI The new approach to robotic programming combines agentic AI, physics-based simulations, and modular control systems to help robots improve their performance without relying on enormous training datasets. Called agentic robotics, the approach allows teams of AI agents to generate robot-control programs, test them in simulated environments, identify failures, and revise their code before deploying it on physical machines. The technology could address two longstanding challenges in robotics: the extensive engineering required to program robot behaviors and the large volumes of real-world data needed to train robots through imitation learning. The team released an industry-standard benchmark for evaluating the performance of robots across industrial tasks, examples of which of shown here. Traditionally, roboticists have relied on two broad approaches. Model-based robotics involves engineers manually programming how machines perceive objects, plan movements, and control their actions. Model-free robotics uses demonstrations and imitation learning to teach machines how to perform tasks. While the former requires considerable engineering effort, the latter demands extensive datasets and lengthy training periods. Agentic robotics seeks to combine the advantages of these methods while reducing their limitations. The approach uses a framework called Graph-as-Policy, which divides complex robotic tasks into smaller, reusable skills. These include identifying objects, planning grasps, and moving robotic grippers without colliding with surrounding objects. AI agents connect these individual skills to construct complete robot-control programs. Using NVIDIA’s Isaac simulation framework, they test the programs under different conditions, identify problems, and repeatedly modify the code to improve performance. This allows robots to undergo multiple rounds of virtual testing before engineers deploy the resulting programs on physical machines. Accelerates robot programming One demonstration involved a robot using a sponge to wipe ink from a metal bar. Researchers supplied an AI system with a short video of the task and access to their simulation interface. Within an hour, the system generated and fine-tuned a physics simulation that reproduced the task, including realistic sponge deformations. The AI then used the simulation and Graph-as-Policy framework to generate a robot-control program. It repeatedly tested and refined the code before researchers transferred it to a physical robot. During one trial, the robot pushed over the metal bar because it had not accurately accounted for the object’s weight. The system used video of the failed attempt to update its simulation, revise the control program, and test a new approach. This feedback process allows AI agents to use failures during physical trials to improve simulations and refine subsequent attempts, potentially reducing manual engineering effort. The technology has also been tested in industrial logistics. A warehouse robotics company applied the methods to parcel sorting, analyzing production data and testing updates to robot behavior. The trials reportedly increased sorting capacity by more than 15,000 packages annually per robot. The company has since deployed the improvements across 30 percent of its U.S. fleet. The researchers are also releasing open-source code built around NVIDIA’s Isaac simulation framework, alongside an industrial benchmark for evaluating robot performance across different tasks. Initial applications are focused on industrial environments, where repetitive operations offer opportunities to test and refine robot behavior at scale. The approach could eventually make robotic systems easier to adapt to changing requirements without manually programming every movement or collecting extensive demonstrations for each new task. However, it does not mean robots can independently master arbitrary real-world activities. Its immediate potential lies in making robot programming, testing, and improvement faster and more adaptable, particularly in structured industrial settings. Get the latest in engineering, tech, space & science - delivered daily to your inbox.Jijo is an automotive and business journalist based in India. Armed with a BA in History (Honors) from St. Stephen's College, Delhi University, and a PG diploma in Journalism from the Indian Institute of Mass Communication, Delhi, he has worked for news agencies, national newspapers, and automotive magazines. In his spare time, he likes to go off-roading, engage in political discourse, travel, and teach languages.