Generative AI is rapidly infiltrating nearly every industry, including cybersecurity, whether we embrace it or not. The prospect of AI-driven malware creation and autonomous attacks should raise eyebrows among sysadmins at this early stage. Enter Wraithwatch, a new security firm determined to counteract malicious AI with advanced, beneficial AI technology.
While the idea of AI combating other AI may sound futuristic, let’s clarify: it’s not about Hollywood-style showdowns. This is about using software automation to empower both malicious and protective actors.
Wraithwatch's co-founders, Nik Seetharaman, Grace Clemente, and Carlos Más, previously worked at SpaceX and Anduril and witnessed a barrage of threats faced by organizations that safeguard sensitive information—be it in aerospace, defense, or finance. "This issue has persisted for over 30 years, and large language models (LLMs) are only amplifying the risks," Seetharaman noted. He emphasized the lack of discussion regarding the offensive implications of generative AI.
A basic illustration of the threat model mirrors standard software development. A developer might code one component and then use an AI copilot to replicate that function across different programming languages. If the output isn’t satisfactory, the AI iterates until it achieves the desired results, or generates several versions to assess performance. This process is practical, but not miraculous; someone remains accountable for the code.
Now, consider a malware developer employing the same approach. They can produce several versions of malicious software in mere minutes, evading superficial detection methods that focus on indicators like package sizes or common libraries. "It’s alarmingly easy for a foreign entity to direct an LLM to rapidly generate thousands of malware variations simultaneously. In our trials, some uncensored open-source models readily transform your malware into any desired form," Seetharaman explained. "The malicious actors are already active, indifferent to guidelines; you have to direct the LLMs to explore these harmful pathways and develop defenses accordingly."
A Reactive Industry
Wraithwatch is crafting a platform that will resemble tactical war games more than traditional cybersecurity measures, which are typically "fundamentally reactive" to identified threats. The onslaught of diverse attacks could soon overwhelm the largely manual cybersecurity response approach that many organizations currently rely on.
As articulated in the company’s blog, "New vulnerabilities and attack methods—occurring weekly—are challenging to comprehend and mitigate. They require extensive analysis to grasp the underlying attack mechanics and then translate that understanding into effective defensive strategies."
Clemente added, "Cyber teams face the constant challenge of zero-day vulnerabilities—flaws for which vendors lack prior notice to patch. By the time we learn of them, discussions about their mutated variations are already circulating. Organizations like SpaceX, Anduril, and U.S. government entities often receive customized attacks that are not publicly known. We cannot afford to wait until we see someone else impacted."
Although current custom attacks are predominantly human-generated, early instances of generative cyber threats, such as WormGPT, signal a change. While rudimentary, the emergence of advanced models in this arena seems inevitable.
Current Limitations and Detection
Más pointed out that present LLMs have limitations; however, security researchers have shown how popular code-generation APIs, like those from OpenAI, can be manipulated by malicious users. Open-source alternatives without alignment restrictions pose additional challenges. “By creatively leveraging these APIs, you can obtain unexpected results,” Más noted. Furthermore, “Agencies often detect attackers by identifying signatures of the methods and binaries used. Imagine if an LLM could generate such signatures at will, or create an entirely new advanced persistent threat (APT) on command.”
Seetharaman suggested that new AI agents—trained to engage with various software platforms and APIs like human users—could evolve into semi-autonomous threats capable of coordinated, persistent attacks. If cybersecurity teams aren’t equipped for this level of sustained assault, a significant breach is only a matter of time.
The Strategic Approach
So, what’s the solution? Essentially, a cybersecurity platform that utilizes AI to personalize detection and defense mechanisms against the offensive capabilities of malicious AI is essential.
“We intentionally positioned ourselves as a security company that harnesses AI, rather than merely an AI company applying security measures. Our experiences on the other side of the keyboard have shown us the types of attacks we faced until our last days at our former companies,” said Clemente.
While leading organizations like Meta and SpaceX may have top-tier security teams, smaller companies may struggle to establish equivalent capabilities. Every cybersecurity tool available has its limitations, and Wraithwatch aims to function as a command and control layer over the existing tools, weaving them into a cohesive system.
By employing attacker-like methods within a controlled environment, Wraithwatch hopes to predict the types of variations and assaults that LLM-infused malware may deploy. The capability of AI models to discern meaningful patterns from noise can create a protective framework that detects, and potentially reacts to threats autonomously. For instance, the system could prepare to combat multiple variants of a new attack as quickly as its admins can deploy patches.
“Our vision is a scenario where, instead of waking up wondering if we’ve been compromised, Wraithwatch is proactively simulating thousands of attack scenarios, detailing necessary adjustments, and automating those changes whenever possible,” Clemente explained.
Though the small team has put several thousand lines of code into the project, it is still in its infancy. Nonetheless, there is a compelling argument that malicious actors are already leveraging this technology. A nimble startup with a team of cybersecurity veterans and sufficient venture capital could innovate rapidly, free from the constraints of larger corporate structures.
Wraithwatch recently completed an $8 million seed funding round led by Founders Fund, joined by XYZ Capital and Human Capital. The goal is to expedite development, treating this as a race against time. "Given our backgrounds in high-pressure environments, our aim is to deliver a resilient MVP with core features to our design partners by Q1 next year, with a broader commercial offering expected by the end of 2024," Seetharaman stated.
Though the concept of AI agents infiltrating U.S. secrets may sound like something out of a thriller, it’s wise to prepare for unprecedented challenges in the rapidly evolving AI landscape. We can only hope that the threats Wraithwatch anticipates are still a few years away. For now, investment in proactive defenses is crucial for protecting sensitive information.
Ranking
