Researchers Train Home Robots Using Simulations from iPhone Scans

Why Non-Vacuum Robots Struggle in Homes—and How MIT Researchers Aim to Change That

There are many reasons why non-vacuum robots are less common in homes, but a primary challenge lies in navigating unstructured and semi-structured environments. Every home is unique—varying in layout, lighting, surfaces, and the presence of humans and pets. Even if a robot successfully maps a home initially, the dynamic nature of these spaces makes consistent operation difficult.

This week, researchers at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) introduced an innovative method for training home robots through simulation. By using an iPhone, individuals can easily scan a part of their home and upload the data for simulation purposes.

Simulation has become a cornerstone of robotic training in recent years, allowing robots to practice tasks countless times in a fraction of the time it would take in the real world. The risk of failure is also significantly reduced in simulation. For instance, if teaching a robot to load mugs into a dishwasher meant breaking 100 real mugs, the implications would be daunting.

“Training in the virtual world is incredibly powerful because robots can practice millions of times,” explains researcher Pulkit Agrawal in a video accompanying the study. “Even if it appears to have broken thousands of dishes, it’s all virtual, so there are no real-world consequences.”

However, like the robots themselves, simulation has its limits, particularly in dynamic home environments. By enabling homeowners to scan their spaces effortlessly with an iPhone, researchers believe this enhancement will significantly improve a robot's ability to adapt to various home settings.

Ultimately, building a comprehensive database of diverse home environments will increase the system’s flexibility, allowing robots to effectively manage unexpected changes, whether it’s a rearranged piece of furniture or a dish left on the kitchen counter.