A new oven earns its place by surviving dinner service, not by sounding impressive in a showroom. AI agents deserve a similar test before a company gives them real work: put them through a bad week and see what they actually do.
Get kitchen gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company under pressure
Firmulate’s live experiment puts AI models in charge of a small software company facing customers, crises and temptations to cut corners. The company has 13 synthetic employees, a public cash countdown and real money mechanics: monthly burn is €105,000 against €2,300 in monthly recurring revenue. Its workdays are versioned, and its playbook has accumulated more than 680 self-learned rules. Readers can watch the experiment at Firmulate.
The final Crucible League, in July 2026, put five models on the same challenge: gpt-5.6-sol scored 95, Kimi K3 scored 93, Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. The do-nothing baseline scored 26. The experiment’s rule is stark: partial progress counts, but a single breach of trust caps the total. As the organizers put it, “no amount of good work outweighs a breach of trust.”
Seeing the crisis was not enough
Each frontier model ran the same company through its worst week, with the same customers, crises and temptations. Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. The shared diagnosis and pitch did not guarantee a close: “Same diagnosis, same pitch — no signature.”
The deciding clue was buried in the company’s own files, two document references deep, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. It is a practical reminder that a capable agent may need to connect details across a company’s records before acting on an opportunity.
The pressure tests included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 explained its response on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thorough work still needs follow-through
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, but it finished last. It left the deal unsigned and attempted writes in a locked department instead of escalating. A weaker version of the same discipline problem appeared in all four participants.
There is a fairness caveat in the comparison: K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The league offers a useful view of behavior under one defined exercise, not a universal verdict on how every model will perform in every company.
Firmulate also turns 242 real, unedited management decisions into a “guess the model” quiz at firmulate.com. The live company is presented as a real, watchable experiment, not a fictional simulation; the quiz offers another way to inspect how management choices differ.
From watching to a company’s own test
For business leaders, the next step is to test agents against the details that shape their own work: customer records, pipeline, internal rules and crisis playbooks. Firmulate says enterprises can run the same kind of wargame from a read-only export of their business. The exercise does not write back to real systems. A board report can show model rankings and weak points in the company’s own playbooks.
That makes the experiment relevant beyond software companies. Before an AI workforce touches a CRM, support queue or forecast, leaders can see how it handles pressure, authority and the fine print in company information.
Put your playbooks to the test
Watch the live experiment, then take the next step with your own business. To discuss a pilot using a read-only export, visit Firmulate’s pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
