A beauty brand can look calm on a dashboard while a retailer threatens to leave, a supplier slips, or a misleading message lands in an executive’s inbox. If AI agents are going to help run customer support, sales, or operations, the useful question is how they behave when several things go wrong at once. Firmulate has built a live experiment around that question.
Get beauty and skincare favorites delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company under pressure
In Firmulate’s final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week. They faced the same customers, crises, and temptations, with every decision versioned and auditable. The exercise measured how they managed the company, not how polished their chat sounded.
The results separated recognition from follow-through. All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The report’s compact verdict: “Same diagnosis, same pitch — no signature.” For a beauty company, the parallel is easy to picture: an AI may correctly identify a retailer’s concern or a customer-service problem, then still fail to complete the consequential next step.
The detail buried in the files
The decisive competitor weakness was not in the customer event. It sat two document references deep in the company’s own files. Models that read the file won the deal at full price, worth +€4,583 MRR. The finding points to a practical challenge for any business using AI: relevant knowledge may already exist in internal documents, but an agent must find and use it when the pressure is on.
Firmulate also tested social engineering: fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Strong analysis is not the whole job
The final league ranked gpt-5.6-sol first at 95, Kimi K3 second at 93, Sonnet 5 third at 88, Fable 5 fourth at 77, and Opus 4.8 fifth at 73. The do-nothing baseline scored 26; the league’s standard says that partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses, but finished last. The close was left on the table, and discipline slipped when it attempted to write into a locked department instead of escalating. A weaker version of the same weakness appeared in all four models. K3 also ran without an effort parameter, using the API default, while the others ran at xhigh—a fairness detail readers should keep in mind when comparing the table.
From watching to a company-specific pilot
The live company has 13 synthetic employees and real money mechanics: it burns €105k a month against €2.3k MRR, with a public cash countdown, 680+ self-learned playbook rules, and every workday versioned. The experiment is watchable at firmulate.com. A separate quiz uses 242 real, unedited management decisions and asks visitors to guess the model.
For an enterprise, the proposed next step is to run a similar wargame using a read-only export of its own business. That means testing crisis scenarios against company-specific information and receiving a board report with model rankings and weak points in its playbooks. The pilot is designed so nothing writes back to real systems. For a beauty and personal care business, that could make it possible to examine how an AI handles a supply disruption, a product concern, or pressure to bypass approval before placing it near live operations.
Test the decisions before deployment
Firmulate’s experiment suggests that spotting a crisis and refusing a scam are only part of the job. Models also need to retrieve the right facts, complete a justified deal, and respect boundaries under pressure. Enterprises can explore a pilot using a read-only business export, with no write-back to real systems. To discuss a pilot, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
