Meta's Open-Source Speech AI: Recognizing More Than 4,000 Spoken Languages for Enhanced Communication

Meta has unveiled a groundbreaking AI language model through its Massively Multilingual Speech (MMS) project, distinguishing itself from existing AI chatbots like ChatGPT. MMS is capable of recognizing over 4,000 spoken languages and producing text-to-speech in more than 1,100 languages. As part of its commitment to fostering language diversity, Meta has chosen to open-source MMS, inviting researchers to build upon its foundation. The company emphasized the importance of this initiative, stating, “We hope to make a small contribution to preserve the incredible language diversity of the world.”

Training speech recognition and text-to-speech models typically requires extensive audio datasets with accompanying transcription labels. These labels are vital for machine learning as they help algorithms categorize and understand data. However, adequate data for lesser-known languages—many of which are at risk of extinction—often doesn’t exist. Recognizing this gap, Meta adopted an innovative method for gathering audio data by utilizing recordings of translated religious texts, such as the Bible. “These translations have publicly available audio recordings in various languages, making them suitable for our research,” the company explained.

While this methodology might initially raise concerns about potential bias towards Christian perspectives, Meta assures that the model remains balanced. "Our analysis shows that this does not bias the model to produce more religious language," the company stated. This balance is attributed to their connectionist temporal classification (CTC) approach, which is less prone to bias than traditional large language models (LLMs) or sequence-to-sequence models. Moreover, despite the predominance of male voices in the recordings, the model effectively performs across both male and female voices.

After aligning the data for usability, Meta employed wav2vec 2.0, its self-supervised learning model, allowing it to train efficiently on unlabeled audio. This blend of unconventional data sources and advanced modeling yielded remarkable results, with MMS outperforming existing models. The company reported that models trained on MMS data achieved a word error rate half that of competitors while covering 11 times more languages than OpenAI’s Whisper.

Despite these successes, Meta acknowledges that the new models are not without flaws. They caution that the speech-to-text model may occasionally mistranscribe words or phrases, which could lead to offensive or inaccurate outputs. The company emphasizes the necessity of collaboration within the AI community for the responsible development of AI technologies.

By releasing MMS for open-source research, Meta aims to counteract the tendency of technology to favor only a handful of widely spoken languages. The vision is clear: to create an environment where assistive technology, text-to-speech, and even virtual and augmented reality tools empower individuals to communicate and learn in their native languages. Meta concluded, “We envision a world where technology encourages the preservation of languages, allowing people to access information in their preferred tongue.”