Chatbots like Gemini and ChatGPT may only be the beginning of the AI revolution. Indications suggest that the next significant advancement in this field will involve AI agents. These programs are designed to take control of systems and apps to perform a wide variety of tasks. In fact, Google has recently made a substantial move in this direction.
On Wednesday, the tech giant unveiled Project Mariner (formerly known as Project Jarvis). This AI agent is designed to understand what’s displayed on a browser screen and perform actions on behalf of the user. It’s based on Gemini 2.0, the latest version of the company’s family of language models.
A New Way to Use Your Browser
According to Google, Project Mariner can interact with web pages through an experimental extension available in Chrome. First, the system analyzes the user’s instructions, whether typed or spoken. Next, it attempts to fulfill the requests by analyzing pixels, page text, code, images, and even forms.
In the demo video above, a Chrome window displays a spreadsheet with the names of several companies. A member of the Google DeepMind team instructs the agent to look up the websites of these companies and extract contact emails. The agent promptly begins to fulfill the request.
Next, the agent opens the Google search engine, searches for each company, navigates to their About Us sections, and gathers the information. A visual progress report appears in a sidebar of the browser, showing exactly what the agent is doing. Users can stop it at any time.
Google says that its agent can be highly beneficial for automating repetitive tasks and saving time. If a request is unclear, the agent can ask for clarification or additional information from the user, which should help minimize errors. It’s important to note that the company expects some bugs to occur. In the end, this is an experimental version currently available only to a limited number of “trusted testers.”
In October, Anthropic introduced Computer Use, a system aimed at automating tasks within a computer’s operating system. Anthropic’s agent is still in its early stages and has some limitations. For example, it may struggle to complete tasks, exhibit slow responses, and make errors. However, this technology is expected to continue evolving.
Image | Google
View 0 comments