The reliance on public web data for training AI models is limiting advancements in the field, according to Daniel Beutel, a tech entrepreneur and researcher at the University of Cambridge. As a co-founder of the startup Flower, Beutel aims to address this critical issue in AI research.
“Public, centralized data accounts for only a small portion of all available data,” Beutel explained in an email interview. “In contrast, distributed data—such as that stored on phones, wearables, and IoT devices, or within organizational units—represents a far richer and more extensive resource that is currently inaccessible for AI training.”
Flower, established in 2020 with Cambridge colleagues Taner Topal and Nicholas Lane, the former head of Samsung’s AI Center in Cambridge, seeks to "decentralize" the AI training process. The platform empowers developers to train models on data distributed across thousands of devices, leveraging a technique called federated learning. This approach ensures data privacy and compliance, as it provides no direct access to the underlying data.
“Flower believes that once the benefits of distributed data are made easily accessible, this method of AI training will not just become common but will redefine the standard,” Beutel stated.
Federated learning is not a new concept; it has been discussed in academia for years. This technique involves training AI algorithms across decentralized devices that hold data samples without the need to exchange those samples directly. While a central server may manage the algorithm’s training, it can also occur through peer-to-peer connections. Local algorithms are trained on their respective datasets, exchanging only the learned components—referred to as "weights"—to create a unified global model.
Flower’s platform utilizes federated learning to present a decentralized alternative for AI model training.
Numerous startups, including DynamoFL, DataFleets, and Sherpa, along with major tech companies like Google, are also implementing federated learning to develop AI models.
“With Flower, data remains on the source device, whether it's a company facility or another location, during the training process,” Beutel clarified. “This means that computation occurs where the data resides, and only the results of that training are transmitted and integrated across all sites, ensuring data integrity and privacy.”
Recently, Flower unveiled FedGPT, a federated approach to training large language models (LLMs) similar to OpenAI’s ChatGPT and GPT-4. Currently in preview, FedGPT allows organizations to develop LLMs using scattered, sensitive data across various devices, including data centers and workstations.
“FedGPT is significant as it enables businesses to build LLMs while maintaining ownership of their sensitive data without requiring interaction with an LLM provider,” Beutel noted. “Many companies have data that cannot be moved due to geographic restrictions or internal policies. FedGPT allows for the effective use of this data during LLM training, addressing privacy concerns and legal constraints on data movement.”
Additionally, Flower is collaborating with Brave, the open-source web browser, to launch a project named Dandelion. This initiative aims to create a federated learning system utilizing over 50 million active Brave browser clients, according to Beutel.
“AI is entering an era of heightened regulation and increased scrutiny over data usage,” Beutel emphasized. “Organizations can develop AI solutions using Flower that prioritize user privacy while tapping into more data than ever before. With federated learning principles, AI systems can be deployed and trained under various constraints.”
In recent months, Flower has experienced remarkable growth, with its developer community expanding to over 2,300 users. Beutel claims that numerous Fortune 500 companies and esteemed academic institutions, including Porsche, Bosch, Samsung, Banking Circle, Nokia, Stanford, Oxford, MIT, and Harvard, are among its users.
Encouraged by this progress, Flower, part of Y Combinator's 2023 cohort, has attracted investment from major players such as First Spark Ventures, Hugging Face CEO Clem Delangue, Factorial Capital, Betaworks, and Pioneer Fund. In its pre-seed funding round, the startup successfully raised $3.6 million.
According to Beutel, this funding will primarily support the expansion of Flower’s core team, including researchers and developers, while accelerating the development of the open-source software that underpins Flower's framework and ecosystem.
“AI is grappling with a reproducibility crisis, particularly acute in federated learning,” Beutel stated. “The limited training on distributed data has resulted in a shortage of widely adopted open-source software implementations. By collaborating as a community, we aim to create the largest repository of open-source federated learning techniques available on Flower.”
Ranking
