Potential Risks of Zoom’s Custom AI Avatar Tool: What You Need to Know

Zoom Aims to Create AI-Animated, Photorealistic Avatars by Next Year

Zoom has announced an exciting new feature: AI-animated, photorealistic avatars that will be introduced sometime next year. During Zoom’s annual developer conference, the company revealed that these avatars will transform a short video clip recorded by users into a digital duplicate, showcasing a head, shoulders, and upper arms. Users can type a script for their digital clone, while Zoom’s technology generates synchronized audio that matches the avatar's lip movements.

According to Smita Hashim, Zoom’s chief product officer, these custom avatars are intended to enhance “asynchronous” communication among colleagues, promoting a “faster, more productive” way to engage. “Avatars save users valuable time and effort when recording clips, allowing them to scale video creation,” explained Hashim.

Despite their potential benefits, custom avatars bring concerns about deepfake technology. Several companies have developed AI tools that digitally replicate an individual's face while pairing it with convincingly synthetic speech. For instance, Tavus enables brands to create virtual personas for personalized video advertising, and Microsoft introduced a service last year that generates realistic digital substitutes.

While many of these tools incorporate strict safeguards against misuse, Zoom's approach remains somewhat unclear. Hashim referenced Zoom’s existing usage policies against malicious use, stating that the company is implementing “numerous safeguards” within its custom avatar feature, which includes “advanced authentication” and watermarking techniques. “We will continue to evaluate and introduce safeguards as necessary,” Hashim added. “Our technology aims to make it evident when a clip is generated with an avatar, helping to ensure the integrity of avatar-generated content.”

The introduction of Zoom’s digital avatars aligns with CEO Eric Yuan’s aspiration of creating AIs capable of participating in Zoom meetings, responding to emails, and making phone calls on behalf of users.

However, the timing is particularly sensitive, as deepfakes have proliferated on social media, complicating the distinction between fact and fiction. In recent months, deepfakes featuring notable figures such as President Joe Biden, Taylor Swift, and Vice President Kamala Harris have garnered millions of views. Additionally, following Hurricane Helene, fake generative AI images depicting destruction emerged online, further blurring reality.

Deepfakes have also been weaponized to impersonate individuals, resulting in significant financial losses. In 2022, impersonation scams reportedly caused more than $1 billion in losses, according to the FTC.

Details on how Zoom plans to combat the potential misuse of its tool remain scarce. Though a mock-up of a custom avatar video displays a visible watermark, such markers can easily be removed using screen-recording software.

Further details are expected closer to mid-2025 when Zoom plans to launch custom avatars for its Zoom Clips feature. This offering will be available as part of a premium add-on priced at $12 per user, per month.

As Zoom navigates this uncharted territory, regulatory efforts continue to tackle the rising tide of deepfakes. Lacking a federal law criminalizing deepfakes in the U.S., over ten states have enacted measures against AI-aided impersonation. California is currently stalled on legislation that would enable judges to mandate the removal of deepfakes, imposing potential financial penalties for offenders.