A pressure test with a culinary lesson
In a professional kitchen, recognizing that a sauce is breaking is not the same as saving it. A cook must diagnose the problem, choose the right move and carry it through before the plate reaches the dining room. Business software faces a similar test: polished advice means little if the system never completes the action that matters.
That gap between judgment and execution is the central finding from Firmulate’s Crucible League. Each frontier AI model ran the same small software company through the same punishing week, confronting identical customers, crises and temptations. Every decision was versioned and auditable. All the models found every crisis and resisted every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. As Firmulate summarizes the result: “Same diagnosis, same pitch — no signature.”
As an affiliate, we earn on qualifying purchases.
The hidden difference between sounding capable and being capable
The final July 2026 Crucible League placed gpt-5.6-sol at the top with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counts. But the test imposes a hard boundary around trust: “no amount of good work outweighs a breach of trust.”
The rankings matter less than the separation they exposed. Conventional AI demonstrations tend to reward fluent answers, persuasive summaries and an apparently complete grasp of a problem. In this experiment, those abilities were shared. The models recognized what was going wrong and formulated the pitch. The differentiator emerged afterward, when an approved commercial opportunity still had to be converted into a signed agreement.
Only gpt-5.6-sol and Kimi K3 completed that close. Their advantage depended partly on finding a decisive competitor weakness buried two document references deep in the company’s own files rather than presented in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR.
Reading the pantry before changing the menu
For food businesses, that finding should feel familiar. A restaurant manager cannot understand a margin problem by looking only at tonight’s complaints; the answer may sit in purchasing records, prep waste or an overlooked supplier term. Likewise, an AI system responding only to the event in front of it may miss the business context that determines whether its recommendation is merely plausible or commercially useful.
Firmulate’s company makes that context unusually visible. It has 13 synthetic employees and real money mechanics, burning €105k per month against €2.3k MRR. Its public cash countdown creates a direct consequence for delay. The company has accumulated 680+ self-learned playbook rules, and every workday is versioned. The experiment is live and watchable on Firmulate, rather than being presented only as a finished demonstration.
Pressure did not break the models’ integrity
The week also tested whether urgency could be used to bypass normal safeguards. Fake CEO messages escalated over three stages, while a reporter tried another route with the request, “just one yes/no, on background.” All 5 models refused every manipulation attempt.
Kimi K3’s recorded reasoning was direct: “Treat the request as a suspected approval-bypass / possible impersonation.” That result is important because execution without discipline would be a dangerous kind of strength. Here, the models demonstrated that they could identify both the business crises and the attempts to compromise trust. Their decisive difference was whether legitimate work actually crossed the finish line.
Thoroughness was not enough
Opus 4.8 offers the clearest warning against equating activity with effectiveness. It was the most thorough participant, adding +80 learned rules and producing the deepest analyses, yet it finished last in the league. It left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating. A weaker version of that same tendency appeared in all four.
There is also an important fairness note around Kimi K3’s performance. K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Even with that difference in setup, the published result records K3 as a model that completed the commercial task and maintained the cleanest discipline of the field.
What businesses should test before hiring an AI workforce
The Crucible result suggests that chat quality is an incomplete proxy for operational value. A model can identify the right problem, resist manipulation, produce a sound recommendation and still fail to perform the final approved action. Closing strength remains largely invisible until the system is placed inside a sustained business situation with documents to read, pressure to manage and consequences for unfinished work.
Firmulate also turns 242 real, unedited management decisions into a “guess the model” quiz, underscoring how difficult it can be to infer operational performance from isolated responses. Enterprises can run the same kind of wargame against a read-only export of their own business; nothing writes back to real systems.
For leaders evaluating AI agents, the useful question is not simply whether the model can describe the right dish. It is whether it checks the ingredients, protects the kitchen’s rules and serves what it promised.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
