General-purpose robots are difficult to train because they require large amounts of high-quality real-world data. EgoZero, created by NYU’s General-purpose Robotics and AI Lab, offers a scalable solution using egocentric data from smart glasses. Instead of relying on carefully arranged static cameras, EgoZero uses Meta’s Project Aria glasses to record tasks from a human’s point of view. This perspective automatically captures the most relevant visual information because the camera moves with the user’s head, similar to how we naturally look at the objects we interact with.
The key innovation is that EgoZero trains robots without any robot-generated data, which is typically expensive and time-consuming to collect. Instead of using full images which differ greatly between human hands and robot arms the system converts movements into 3D points in space. These points are mapped onto robot appendages, making the learning transferable across different robot designs.
Using only 20 minutes of human demonstration per task, the robot successfully performed manipulation tasks like picking up bread with a 70% success rate. The researchers are also developing smartphone-based grippers and touch sensors to make data collection even easier. Ultimately, their goal is to scale robot learning by capturing everyday human interactions, similar to how language models learn from the internet.
Read more-https://spectrum.ieee.org/smart-glasses-robot-training