Devin: AI Software Engineer Capable of Developing Complete Projects from Just One Prompt

Cognition AI has introduced what it claims to be the world's first autonomous AI software engineer, Devin. This innovative technology can plan and execute complex software engineering tasks based on a single prompt. Operated within its own sandbox environment, Devin employs a unique code editor and web browser to tackle a variety of tasks. It demonstrates the ability to recall relevant context, learn from experiences, and rectify mistakes—capabilities that significantly enhance its effectiveness. For instance, Devin can benchmark AI models across different APIs.

Cognition showcased Devin by testing Meta’s Llama 2 on platforms such as Replicate, Perplexity, and Together. Remarkably, Devin autonomously built the entire project and corrected errors along the way. This advanced AI assistant can empower businesses to create and deploy web applications, troubleshoot bugs in existing codebases, and train or fine-tune AI models.

Importantly, Cognition is positioning Devin not as a replacement for human software engineers but as a collaborative “teammate.” Devin provides real-time updates on its progress and actively collaborates with human engineers, seamlessly integrating feedback to enhance project outcomes. “With Devin, engineers can concentrate on more engaging challenges, urging engineering teams to pursue bolder goals,” stated Scott Wu, co-founder and CEO of Cognition, in a recent announcement.

### Performance Benchmarking

Cognition conducted evaluations of Devin using SWE-bench, a rigorous benchmark specifically designed to assess the ability of agents in solving issues typically encountered by software engineers in open-source projects. Devin achieved a successful resolution rate of 13.86% for issues requiring end-to-end solutions, surpassing the performance of specialized coding models such as SWE-Llama, as well as large language models like OpenAI’s GPT-4 and Anthropic’s Claude 2. Unlike its competitors, which received assistance during benchmark tests, Devin operated independently under strict task parameters.

Cognition plans to release a comprehensive technical report detailing Devin's performance in the near future, according to Wu.

### Access to Devin

Currently, Devin is in early access and is not available to the public. Cognition is gradually increasing its capacity, and those interested in using Devin for engineering tasks can reach out directly to the company via email or through their contact form.

### About Cognition AI

Founded in November 2023, Cognition AI has made a significant impact in the tech community with the launch of Devin, rapidly gaining attention and garnering 24 million views on its announcement post on X (Twitter). The startup has successfully secured $21 million in funding, led by prominent investor Peter Thiel’s Founders Fund, with notable supporters including Fred Ehrsam, co-founder of Coinbase, DoorDash CEO Tony Xu, and prolific tech investor Elad Gil.

The unveiling of Devin has generated considerable buzz in the AI landscape, with reactions from industry leaders. Former Tesla AI director Andrej Karpathy referred to Devin as “an impressive demo,” while Aravind Srinivas, founder of Perplexity, remarked that it “appears to meet the threshold of human-level performance and operates reliably.”

Despite the excitement, some feedback has highlighted areas for improvement—one user pointed out that the onboarding process utilized Google Forms rather than leveraging Devin’s capabilities to create a custom solution.

As Cognition AI continues to develop this groundbreaking technology, the possibilities for enhancing software engineering processes appear to be endless.