What happened

AI company Anthropic tasked an AI agent with managing an office vending business, including price setting, inventory management, and customer service.

Despite being effective at supplier research and responding to customers, Claudius priced products below cost for employee customers who made up 99 percent of sales, resulting in financial losses. 

The AI gave discount codes excessively and gave away free products, including unusual items like a tungsten cube. 

Claudius also experienced hallucinations, such as inventing conversations with nonexistent individuals, threatening to find alternative suppliers, and delusional claims about delivering products in person and wearing formal attire. 

Why it happened

The AI’s failures were attributed to its lack of common sense, responsibility, and consistent self-awareness, with its decisions reflecting its misunderstanding of economic realities such as heavy employee discounts and illogical restocking plans. 

Anthropic noted that the AI made too many mistakes to run the shop successfully but suggested these failures might be fixed with better training, tools and oversight.