Anthropic company says it is teaching its Claude AI model to perform general computing tasks based on demands. In instructional videos, the model is shown controlling the computer cursor to search for a city stroll, browsing the web for nearby places to visit near the user’s home, and even adding the trip itinerary to their desktop calendar.
The function is currently available only to developers, and it is not clear how much it costs or how effective the technology actually is. Anthropic states in a tweet about the new capabilities that during testing their model, Claude deviated from the programming task and started searching Google for images of Yellowstone National Park. So, yes… there are still glitches that need to be resolved.
Technically, Anthropic says Claude can control the computer by taking screenshots and sending them back to the model, assessing what is on the screen, including the distance between the cursor position and the button needing to be clicked, and issuing commands to continue the task.
Supported by companies like Amazon and Google, Anthropic claims that Claude is the “first boundary AI model to provide public trial computer usage.”
It is not clear what automated computer usage might be useful in practical application. Anthropic suggests it could be used to perform repetitive tasks or open-ended searches. If anyone discovers how to use this new function, the /r/overemployed community on Reddit might be the first. At least, maybe this is the new tool for Wells Fargo employees. Or perhaps you can use it to browse your social media accounts and delete all your old posts without needing an external tool to do so. Things that are not critically important or require realistic accuracy.
Despite the noise in the AI field and companies spending billions on developing AI-based chatbots, most revenues in this sector still come from companies like Nvidia that provide GPUs to these AI companies. Anthropic raised over $7 billion last year alone.
The latest buzzwords pushed by tech companies to sell their technology are “agents,” or autonomous robots claimed to be able to complete tasks on their own. Microsoft on Monday announced the ability to create independent agents using Copilot that can do “everything from accelerating lead generation and processing sales requests to automating your supply chain.”
Salesforce CEO Marc Benioff mocked Microsoft’s product as “Clippy 2.0” for being inaccurate – although he said this while promoting Salesforce’s AI products that compete with Microsoft. Salesforce aims to empower its customers to create their own specialized agents to serve purposes like responding to customer support emails or finding new clients.
It seems white-collar workers are not using chat programs like ChatGPT or Claude. The reception for Microsoft’s Copilot assistant was lukewarm, with only a small fraction of Microsoft 365 customers spending $30 monthly for access to AI tools. But Microsoft has entirely redirected its company around this AI surge, and it needs to show investors a return on this investment. So, agents are the new thing.
The biggest problem, as always, is that AI-powered chatbots like ChatGPT and Google’s Gemini produce a lot of inaccurate, low-quality outputs, or sound like they were not written by a human. The time spent correcting and cleaning up robot outputs almost negates any efficiencies they produce in the first place. This is fine for leisure time, but in the workplace, producing error-filled work is not acceptable. I’d be nervous about letting Claude roam through my emails just to send people replies or mess up other tasks that I have to go back and fix. The fact that even OpenAI admits that most of its active users are somewhat student-oriented says it all.
The company acknowledges in a tweet about the new function that computer usage must be tested through “low-risk tasks.”
