Earlier this year, GitHub introduced Copilot Chat, a programming-focused chatbot resembling ChatGPT, available initially for organizations subscribed to Copilot for Business. Recently, it entered the beta phase for individual Copilot customers at $10 per month. Today, GitHub has made Copilot Chat generally available to all users.
As of now, Copilot Chat can be accessed in the sidebar of Microsoft’s integrated development environments (IDEs), Visual Studio Code and Visual Studio. It is included in paid tiers of GitHub Copilot and offered for free to verified educators, students, and maintainers of select open-source projects.
“As a platform for the global developer community, we’ve launched what has become the most widely adopted AI developer tool ever,” said Shuyin Zhao, VP of Product Management at GitHub, in an email interview. “And code completion was just the beginning."
Since the beta, there have been few changes to Copilot Chat. The chatbot continues to leverage GPT-4, OpenAI’s leading generative AI model, specifically fine-tuned for development purposes. Developers can use natural language to interact with Copilot Chat for a variety of real-time assistance, such as explaining concepts, identifying vulnerabilities, or generating unit tests.
Inspired by current trends, it’s essential to note that the underlying GPT-4 model was trained on publicly available data, some of which may be copyrighted or restricted. While GitHub and similar vendors assert that the fair use doctrine protects them from copyright disputes, some developers have initiated class action lawsuits against GitHub, its parent company Microsoft, and OpenAI, alleging violations of open-source licensing and intellectual property rights.
When I inquired whether codebase owners would have a chance to opt-out of training, Zhao responded that no new mechanisms were introduced with the broader launch of Copilot Chat. Instead, she suggested that codebase owners keep their repositories private to avoid being included in future training sets.
It’s understandable that codebase owners may find this suggestion unappealing — maintaining public access to copyrighted code often aids in crowdsourcing bug detection. Yet, GitHub currently seems inflexible on allowing training data opt-outs.
Moreover, generative AI models, including GPT-4, can sometimes generate misleading information, a phenomenon known as "hallucination," which poses risks in coding. A recent study conducted by Stanford revealed that developers who utilize AI coding assistants like Copilot produce less secure code compared to those who do not, largely due to the introduction of buggy or outdated code snippets by these AI tools.
Zhao mentioned that GPT-4 performs "better" in managing hallucinations than its predecessor used by Copilot. She also highlighted features designed to mitigate security risks, such as filters that detect insecure coding patterns, alerting users to vulnerabilities like hardcoded credentials, SQL injections, and path manipulations. However, she reiterated the necessity of thorough human review for any AI-generated code.
"GitHub Copilot is powered by OpenAI’s models, which we consider the best available for our services," Zhao commented. "We are well-positioned to continue equipping developers with the AI tools they need to create secure software efficiently while enjoying the process."
In October, Microsoft CEO Satya Nadella reported that Copilot had reached 1 million paying users and about 37,000 enterprise clients. Nonetheless, GitHub must enhance Copilot’s appeal to avoid losing market share to competitors and financial profitability.
According to a Wall Street Journal report, Copilot is losing an average of $20 per user each month, with certain customers costing GitHub as much as $80 per month. The substantial operational costs of running the underlying AI models have been cited as a major factor—an issue that led the GenAI coding startup Kite to shut down prematurely last December.
While GitHub navigates these challenges to achieve profitability, Amazon has consistently advanced its competitor, CodeWhisperer. In April, Amazon offered CodeWhisperer free for developers without usage restrictions. This was also the month they launched the CodeWhisperer Professional Tier, which included features like single sign-on with AWS Identity and Access Management integration, along with higher thresholds for security vulnerability scanning. An enterprise plan for CodeWhisperer debuted in September, and by early November, Amazon had “optimized” the tool to enhance suggestions for app development on MongoDB, the open-source database management platform.
Beyond CodeWhisperer, Copilot faces competition from emerging startups like Magic, Tabnine, Codegen, and Laredo, in addition to open-source models such as Meta’s Code Llama and Hugging Face's and ServiceNow’s StarCoder.
Ranking
