Anthropic has introduced a groundbreaking update to its Claude AI assistant, enabling it to interact with a PC in a manner similar to a human user. This new feature, called "computer use," entered public beta today. With this capability, Claude can be instructed to perform tasks such as "viewing a screen, moving a cursor, clicking buttons, and typing text," as indicated in the company's announcement.
The integration of a new API allows Claude to perceive and interact with computer interfaces, translating user prompts into actionable commands. This advancement opens up possibilities for developers, who can utilize it to automate repetitive tasks, streamline testing and quality assurance, and conduct exploratory research.
While this development holds significant potential for automating mundane computer tasks, internal testing reveals that the feature is still in its early stages and may be prone to errors. For instance, during testing, Claude unexpectedly paused a coding task to browse images of Yellowstone National Park—a quirk that underscores the human-like nature of the AI but also serves as a reminder of its limitations.
In addition to the "computer use" feature, Anthropic has also launched an upgraded version of its Claude 3.5 Sonnet model and will introduce a new model, Claude 3.5 Haiku, later this month. Recently, Anthropic partnered with OpenAI to collaborate with the US AI Safety Institute, committing to responsible AI development.