Effortlessly Automate Repetitive Tasks: How Twin Labs Uses AI to Control Your Mouse Cursor

Introducing Twin Labs: Revolutionizing Automation for Repetitive Tasks

Meet Twin Labs, a Paris-based startup on a mission to streamline automation for repetitive tasks. Whether it’s onboarding new employees across various internal systems, reordering stock items, downloading financial reports from multiple SaaS platforms, or reaching out to potential clients, Twin Labs is set to simplify your workflows.

“Twin's foundation is rooted in a science-fiction concept. We observed the advancements in large language models (LLMs) and asked ourselves if we could create an AI agent that mimics our task execution,” explained Twin Labs co-founder and CEO Hugo Mercier.

While Twin Labs focuses on enhancing internal processes, the most intriguing aspect is their approach. They utilize multimodal models with vision capabilities, such as GPT-4 with Vision (GPT-4V), to emulate human actions.

Initially, Twin Labs explored developing autonomous agents using traditional LLMs. However, after extensive testing, Mercier concluded, “We discovered that LLMs are often unreliable, frequently leading to incorrect decisions. Ultimately, this means the task remains unfinished.”

The transformative capabilities of GPT-4V stem from its training on diverse software interfaces and the underlying code, which has opened up new possibilities. “When presented with an interface, it comprehends the functionality behind each button,” Mercier noted.

In contrast to automation tools like Zapier, Twin Labs operates similarly to a web browser. Their solution can automatically load web pages, click buttons, and enter text seamlessly.

Imagine a hiring scenario: you need to input a new employee's information into your payroll system, send them a Slack invitation, set up a Google Workspace account, and enroll them with a healthcare provider. Many companies maintain extensive task lists for onboarding, and while these tasks seem straightforward, they must be executed accurately and in the correct sequence. This is why the ability to train Twin Labs' AI assistant using screen recordings and natural language descriptions will be crucial.

Though the startup is not quite there yet, it’s diligently working toward this vision. Co-founders Hugo Mercier and Joao Justi have dedicated the past six months to developing a product prototype. They also secured $3 million in pre-seed funding from Betaworks, Motier Ventures, and Factorial, along with investments from numerous angel investors, including data innovators from respected firms like Dataiku and OpenAI.

Twin Labs faces several challenges with its autonomous agent system. For instance, task completion can be expensive, though the costs associated with APIs and infrastructure in the AI sector are decreasing. In initial phases, Twin Labs plans to launch a product featuring a library of pre-trained tasks to ensure efficiency before expanding its platform to enable clients to create custom tasks.

While many associate AI products with chatbot interfaces, Twin Labs offers a unique perspective that innovatively interacts with AI models. “Our goal is to delve deeper into the everyday responsibilities people face and discover how we can alleviate some of the more tedious tasks,” Mercier emphasized.