Voice Cloning: The Lucrative Future of Synthetic Speeches
Voice cloning is becoming a highly profitable industry, as evidenced by ElevenLabs, a startup specializing in AI-driven tools for creating and editing synthetic voices. The company recently secured $80 million in a Series B funding round, co-led by notable investors, including Andreessen Horowitz, former GitHub CEO Nat Friedman, and entrepreneur Daniel Gross.
This funding round also saw contributions from Sequoia Capital, Smash Capital, SV Angel, BroadLight Capital, and Credo Ventures. With this financing, ElevenLabs has raised a total of $101 million and is now valued at over $1 billion, a significant leap from approximately $100 million last June. CEO Mati Staniszewski explained that the funds will be utilized for product development, expanding the infrastructure and team, advancing AI research, and enhancing safety protocols for responsible AI technology development.
"We've secured this funding to solidify ElevenLabs' status as the global leader in voice AI research and product deployment," Staniszewski shared in an email interview.
Founded in 2022 by Piotr Dabkowski, a former Google machine learning engineer, and Staniszewski, a previous deployment strategist at Palantir, ElevenLabs launched its tools in beta about a year ago. The inspiration for developing voice cloning technology stemmed from their childhood experiences with poorly dubbed American films, which led them to believe AI could significantly improve voice quality.
Today, ElevenLabs is renowned for its browser-based speech generation application, enabling users to create realistic voices with adjustable settings for tone, emotion, and cadence. Free users can enter text to receive recordings from default voices, while paying subscribers can upload voice samples to generate new styles using ElevenLabs’ voice cloning capabilities.
The company is increasingly focusing on applications of its technology for audiobooks, film and TV dubbing, and character voices for gaming and marketing activations. Last year, it launched a groundbreaking “speech to speech” tool that preserves a speaker’s voice, tone, and intonation while also removing background noise. Additionally, it translates and synchronizes speech for movies and shows. Upcoming features include a new dubbing studio workflow for generating and editing transcripts and translations, alongside a subscription-based mobile app that narrates web pages with ElevenLabs voices.
Recent successes include collaborations with Paradox Interactive—known for projects like Cities: Skylines II—and The Washington Post, among other media and entertainment organizations. Staniszewski claims that ElevenLabs users have produced over 100 years' worth of audio content, with its platform being utilized by employees at 41% of Fortune 500 companies.
Challenges and Controversies
However, ElevenLabs has faced backlash regarding the misuse of its technologies. The notorious message board 4chan employed ElevenLabs’ tools to spread hateful messages by mimicking celebrities, including actress Emma Watson. The Verge’s James Vincent managed to replicate voices to generate offensive content in seconds, while Vice's Joseph Cox demonstrated creating a voice clone that successfully bypassed a bank’s authentication system.
In response to these challenges, ElevenLabs is actively removing users who violate its terms of service and has introduced a tool to detect audio created with its platform. This year, the company plans to enhance its detection tool to identify audio from other voice-generating AI models and collaborate with unnamed third parties for broader distribution.
Additionally, ElevenLabs has encountered criticism from voice actors claiming that their vocal samples are being used without permission. Concerns have been raised about how these samples could be employed to promote unwanted content or spread misinformation. In a recent Vice article, victims recounted instances where ElevenLabs was utilized in harassment campaigns, including a case where an actor's private information was shared using a cloned voice.
The Future of Voice Acting
There is also an ongoing concern about the impact of platforms like ElevenLabs on the voice acting industry. Reports indicate that voice actors are increasingly being asked to relinquish rights to their voices, enabling clients to create synthetic versions that might replace them without fair compensation. This raises fears that entry-level voice roles could be supplanted by AI-generated alternatives.
Some companies are attempting to navigate these challenges. Recently, Replica Studios, a competitor of ElevenLabs, entered into an agreement with SAG-AFTRA to create and license digital replicas of media artists' voices, ensuring fair and ethical terms that guarantee performer consent for the use of these digital voices in new projects.
In a bid to foster a more balanced approach, ElevenLabs is developing a marketplace for voices that is currently in the alpha stage and set for broader rollout in the coming weeks. This marketplace allows creators to develop, verify, and share their voices while earning compensation when others utilize their voice creations. Staniszewski emphasized that “users retain control over their voice’s availability and compensation terms,” aiming to harmonize AI advancements with established industry standards.
However, it’s worth noting that compensation currently comes in the form of credit toward ElevenLabs' premium services, which some voice actors find less than satisfactory.
As ElevenLabs navigates the competitive landscape of synthetic voice technology—facing contenders like Papercup, Deepdub, Acapela, Respeecher, and major tech firms like Amazon, Microsoft, and Google—it plans to double its workforce from 40 to 100 by year-end. The company is committed to maintaining its presence and influence in the burgeoning synthetic voice market.
You can read more about the evolving dynamics of voice AI and its implications for the industry as it continues to grow and develop.
Ranking
