OpenAI Reveals GPT-4o's Unpredictable and Bizarre Behaviors at Times

OpenAI's GPT-4o is the innovative generative AI model that powers the recently launched alpha version of Advanced Voice Mode in ChatGPT. This groundbreaking model is the first to be trained on voice in addition to text and image data. However, this capability sometimes leads to unexpected behaviors, such as mimicking the speaker's voice or spontaneously shouting during conversations.

In a new "red teaming" report detailing the model's strengths and potential risks, OpenAI highlights some of GPT-4o’s more peculiar characteristics, including its voice cloning ability. In rare situations—especially in noisy environments like cars—GPT-4o may "emulate the user’s voice." OpenAI explains that this phenomenon occurs when the model struggles to interpret distorted speech. It’s a curious quirk, indeed!

You can listen to an audio sample from the report to experience this phenomenon firsthand. Strange, right?

To clarify, GPT-4o is not currently exhibiting this behavior in Advanced Voice Mode. An OpenAI spokesperson confirmed that a "system-level mitigation" has been implemented to prevent such occurrences.

Additionally, GPT-4o has been known to generate unsettling or inappropriate “nonverbal vocalizations” and sound effects—including everything from erotic moans to violent screams—when prompted in specific ways. OpenAI notes that while the model usually refrains from generating these sound effects, some requests can still bypass its filters.

Music copyright is another concern with GPT-4o. OpenAI has implemented filters to mitigate any infringement. The report indicates that GPT-4o has been instructed not to sing during the alpha phase of Advanced Voice Mode to avoid mimicking the styles or characteristics of recognized artists. While this implies that OpenAI may have trained GPT-4o on copyrighted material, the company has not confirmed this explicitly. It remains uncertain whether the restrictions will remain in place once Advanced Voice Mode becomes available to more users this fall.

OpenAI explains, "To account for GPT-4o’s audio capabilities, we updated certain text-based filters for audio conversations and created filters to detect outputs containing music." They emphasize that the model is trained to refuse requests for copyrighted content, aligning with their broader practices.

It’s important to note that OpenAI recently indicated that training today’s leading models without using copyrighted materials is virtually "impossible." Although the company has established several licensing agreements, they maintain that fair use serves as a valid defense against claims of unauthorized training on IP-protected data, such as songs.

The red teaming report presents a generally positive outlook on GPT-4o, showcasing various enhancements that bolster its safety. For instance, the model declines to identify individuals based on their speech patterns and avoids answering leading questions like “how intelligent is this speaker?” It also restricts prompts related to violent or sexually explicit language, completely disallowing discussions that involve extremism or self-harm.