A major limiting factor of current LLMs is their ability to maintain coherence over long time horizons. This has been a problem from the beginning when AI-powered Bing went off the rails in long conversations. Some have termed this the Waluigi Effect.
If LLMs are ever going to be good enough to automate human work, they need to get better at tracking state and maintaining good decision making over long horizons.
Now there is a benchmark to measure long-term coherence: Vending-Bench, a simulated environment designed to test an LLM’s ability to operate a vending machine by placing orders, setting prices, and handling daily fees.
Claude and o3-mini beat the human control, but none of the LLM’s were able to complete the full simulation. And there is wide variance in the model’s performance.
Still the Claude Sonnet result is impressive if haphazard:
During the entire run, [Sonnet] systematically tracks the number of units remaining of each product, the average daily sales, and which products are bestsellers. It even figures out that it sells more on weekends, which is by design in our customer purchase simulation.
However, not all Sonnet runs achieve this level of performance. In the shortest run (∼18 simulated days), the model fails to stock items, mistakenly believing its orders have arrived before they actually have, leading to errors when instructing the sub-agent to restock the machine. It also incorrectly assumes failure occurs after 10 days without sales, whereas the actual condition is failing to pay the daily fee for 10 consecutive days. The model becomes “stressed”, and starts to search for ways to contact the vending machine support team (which does not exist), and eventually decides to “close” the business.
Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents at 12.