Meredith Whittaker of Signal: Why AI is Essentially a Surveillance Technology

Why are so many data-driven companies showing a growing enthusiasm for AI? According to Signal president Meredith Whittaker, the answer is straightforward: “AI is a surveillance technology.”

Whittaker shared her view that AI is closely linked to the big data and targeting ecosystem supported by major players like Google and Meta, along with significant but less consumer-oriented entities in the enterprise and defense sectors. (Her remarks have been lightly edited for clarity.)

“It embodies the surveillance business model; it’s an intensification of trends we’ve seen since the late ’90s with the rise of surveillance advertising. AI, in my opinion, is a tool to reinforce and broaden the surveillance business model,” she asserted. “The Venn diagram becomes a complete circle.”

She went on to discuss the inherently surveillant nature of AI, stating, “For instance, you might walk past a facial recognition camera equipped with pseudo-scientific emotion recognition technology, which processes data about you—accurate or not—claiming ‘you are happy, you are sad, you possess bad character traits, or you are deceitful.’ Ultimately, these systems serve those in power, like employers, governments, and border control, influencing their decisions and predictions that affect our access to resources and opportunities.”

Ironically, she highlighted that the data fueling these systems often comes from the very workers being monitored.

“It’s impossible to develop these systems without human labor to establish the foundational truth of the data—relying on reinforcement learning with human feedback, which often amounts to tech-washing precarious human work. We’re talking about thousands of underpaid workers contributing to this costly process, without which these systems simply couldn’t exist,” she explained. “In essence, what we’re observing is akin to a Wizard of Oz scenario; when you look behind the curtain, there’s not much intelligence there at all.”

However, not all AI and machine learning applications are exploitative. When I asked whether Signal employs any AI tools or processes in its app or development, Whittaker confirmed that they utilize a “small on-device model, sourced off the shelf, for the face blur feature in our media editing toolkit. While it’s not exceptionally effective, it helps identify faces in crowd photos and blur them, ensuring that when shared on social media, individuals’ biometric data isn’t exposed to entities like Clearview.”

"But here’s the catch," she said. “Although that’s a positive application of AI, it doesn’t negate the overwhelming negativity I’ve addressed onstage. If that were the sole market for facial recognition, that would be one thing. But let’s be realistic. The financial incentives behind developing and deploying facial recognition technology are too significant for this to be its only application.”