Humanoid robot walks across sand, gravel and slopes using faster training framework

A team of Georgia Tech researchers has developed a new machine-learning framework that enables a humanoid robot to walk across sand, gravel, soggy grass, slopes, stairs, and slippery surfaces while cutting the time and computing power needed to train its controller. The researchers say their approach, called “Learn to Teach,” improves on a popular teacher-student reinforcement learning method by training both agents simultaneously instead of one after the other. The result is a controller that can handle terrain it was never trained on while requiring fewer computational resources. The controller was tested on a two-legged humanoid robot, which successfully crossed a range of challenging outdoor and indoor surfaces. The team also pushed and pulled the robot during experiments, and it adjusted its gait to remain stable. The work was presented at the IEEE International Conference on Robotics and Automation (ICRA), where the researchers described a training framework that could be adapted to other robots and tasks beyond walking. Teaching while learning Traditional teacher-student reinforcement learning relies on first creating a “teacher” model with access to detailed simulation data. Once fully trained, the teacher passes its knowledge to a “student” model that controls the real robot. According to lead researcher Feiyang Wu, that process has two major drawbacks. “There are two problems with this approach. It takes too much time to train them sequentially. Then, you’re wasting a lot of information that’s been gathered by the teacher.” Training robotic controllers through simulation can require hours of computation on expensive GPU hardware, making the process both time-consuming and costly. Instead of waiting for the teacher to master the task, the Georgia Tech team trained the teacher and student together. As the teacher gradually learned, it immediately began transferring knowledge to the student, significantly shortening the training process. “You don’t have to wait for the teacher to be an expert for it to begin teaching the student,” Wu said. “The teacher can gradually teach the student what they’ve learned along the way.” The researchers also allowed the teacher to learn from the student’s experiences. This reduced what roboticists call the teacher-student imitation gap, where the student encounters situations that differ from the teacher’s idealized simulation. Real terrain success The new controller was deployed on a full-sized humanoid robot in the lab of Associate Professor Ye Zhao. It navigated rough outdoor terrain and slippery indoor surfaces without relying on separate controllers for different environments. Wu said the team did not expect the same controller to perform so well across many conditions. “For this bulky, very tall humanoid robot, it really hasn’t been proven that you can do agile locomotion on such austere terrain. Somehow our very efficient training recipe here can actually work for all kinds of terrain and environments.” Zhao said the controller even outperformed the software supplied by the robot’s manufacturer, demonstrating the value of combining machine-learning research with real-world robotics. Beyond humanoid locomotion, the researchers believe the “Learn to Teach” framework could be applied to other robot designs and tasks that require reliable movement in unpredictable environments. The research was presented at the IEEE International Conference on Robotics and Automation. Recommended ArticlesGet the latest in engineering, tech, space & science - delivered daily to your inbox.With over a decade-long career in journalism, Neetika Walter has worked with The Economic Times, ANI, and Hindustan Times, covering politics, business, technology, and the clean energy sector. Passionate about contemporary culture, books, poetry, and storytelling, she brings depth and insight to her writing. When she isn’t chasing stories, she’s likely lost in a book or enjoying the company of her dogs.