Key Takeaways
- Researchers created a virtual company staffed entirely by AI agents from major tech firms.
- The goal was to see if AI could autonomously run a software company’s daily operations.
- The experiment was largely unsuccessful, described by some reports as a “total disaster.”
- Even the best-performing AI completed less than a quarter of its tasks, slowly and at a significant cost.
- The results suggest AI isn’t ready to replace human workers in complex roles just yet.
What happens when you try to run a company using only artificial intelligence? Researchers at Carnegie Mellon decided to find out by creating “The Agent Company,” a simulated software firm.
They put AI “agents” from companies like OpenAI, Google, Meta, Amazon, and Anthropic into virtual roles like software engineers and financial analysts.
These AI employees were assigned daily tasks similar to those in a real company, such as coding, analyzing files, and even writing performance reviews.
The outcome wasn’t exactly a smooth operation. According to Newser, citing sources like Insider and Futurism, the experiment was described as a “total disaster” and “laughably chaotic.”
Anthropic’s Claude 3.5 Sonnet performed the best, but it still managed to complete less than 25% of its assigned jobs. Each task took about half an hour on average and cost over $6.
At the other end, Amazon’s Nova Pro 1.0 completed a mere 1.7% of its tasks.
Many supporters of AI agents point out that in a real-world scenario, humans would likely work alongside these tools, providing guidance and oversight.
For now, as Futurism noted, it seems “the machines aren’t coming for your job anytime soon.”
This experiment adds to a growing list of instances where AI has stumbled. Live Science has even compiled examples of other high-profile AI errors, reminding us that the technology is still evolving.
