Hugging Face Unveils LeRobot: An Open Source Robotics Code Library for Developers

Earlier this year, in March 2024, Hugging Face, a leading startup specializing in open-source AI models and code, made headlines by hiring Remi Cadene, a former staff scientist at Tesla, to spearhead a new open-source robotics project.

This news was significant not just because Tesla is a renowned tech giant known for its humanoid robot project, Optimus, but also due to the recent breakthroughs in robotics fueled by transformer-based large language models (LLMs) and generative AI—areas where Hugging Face has extensive expertise.

Today, we have a glimpse into Hugging Face’s vision for open-source robotics with the launch of LeRobot—an open-source toolkit aimed at democratizing AI robotics and inspiring a new wave of roboticists, now available on GitHub.

Cadene shared on X (formerly Twitter): “The next step in AI development is its application to the physical world. We are building a community-driven effort around AI for robotics, and it’s open to everyone!”

He also provided a link to a free Discord server for those interested in collaborating on the LeRobot initiative and utilizing the open-source code.

What is LeRobot?

LeRobot acts as a robust framework, comparable to "Transformers for robotics." This toolkit goes beyond a mere software package; it serves as a comprehensive platform featuring a versatile library for data sharing, visualization, and training advanced models.

Users can access numerous pretrained models to kickstart their projects. Additionally, LeRobot seamlessly integrates with physics simulators, enabling enthusiasts and developers without physical robotics hardware to simulate and test their AI models in a virtual setting.

Capabilities and Features

LeRobot's toolkit is designed to accommodate a variety of robotic hardware—from educational robotic arms to sophisticated humanoids found in advanced research labs. The goal is to create an AI system that can adapt to and control diverse robotic forms, offering versatility and scalability in various applications.

Cadene highlighted some robotic capabilities now available in the LeRobot library on GitHub, including training robots to navigate unfamiliar spaces and grasp objects from video feeds, as well as enabling the transfer of items between robotic grippers.

Open-Source Philosophy

The decision to offer LeRobot as open-source is strategic, aiming to prevent the concentration of power and innovation among a select few corporations. By making it freely accessible, Hugging Face encourages a global community of developers, researchers, and hobbyists to contribute to and benefit from the collective advancement of AI robotics.

Building a Community and a Data Repository

Central to LeRobot’s development is the ambition to create the largest crowdsourced robotics dataset ever attempted. In collaboration with universities, startups, major tech firms, and hobbyists, Hugging Face is working to compile a vast repository of data.

This includes terabytes of onboard video recordings, formatted with the lightweight LeRobotDataset for swift uploads and downloads via the Hugging Face hub.

Cadene emphasized on X: “We are building a diverse community from various backgrounds in software and hardware to develop the next generation of smart robots in the real world.” This reflects the inclusive vision of the project.

By lowering barriers to entry and promoting an environment of shared knowledge and resources, Hugging Face aims to cultivate a community that could redefine the future of AI robotics.