firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

When AI Meets Real Business: The Crucible of Crisis

Just as health checks can reveal underlying issues before symptoms appear, testing AI in a simulated business crisis exposes how well these models can handle pressure, honesty, and complex decision-making. For health and wellness, early detection can save lives; for AI in business, it can save company value. The latest experiment from Firmulate puts these models through their paces, revealing surprising strengths and weaknesses.

Testing AI in the Worst-Case Scenario

In July 2026, a pioneering experiment evaluated five leading AI models by putting them through a simulated week of chaos for a small software company. The goal was simple: see if these models could spot crises, resist manipulation, and close a key deal — all while managing real money mechanics and customer interactions.

The models faced identical challenges: the same stressed customers, internal crises, and social engineering attempts, such as fake CEO messages and reporter tricks. Every decision was carefully tracked and auditable to ensure fairness and transparency. The results? All models identified every crisis and refused every manipulation attempt. But only two managed to close the critical deal, which was worth €55,000 in recurring monthly revenue (MRR).

The Surprising Winner: The Newcomer Outperforms Established Giants

Among the contenders, the standout was Moonshot’s Kimi K3, which scored 93 out of 100. Just behind was the well-known GPT-5.6-sol with a score of 95. The other competitors, Sonnet 5 and Fable 5, scored 88 and 77 respectively, with Opus 4.8 trailing at 73.

What made K3 exceptional? It found a buried fact deep within the company’s files—something the other models missed—and used that insight to win the deal at full price, adding €4,583 in monthly recurring revenue. Meanwhile, even when diagnosis was correct and pitches were on point, only two models managed to get the signature, underscoring the importance of discipline and thoroughness.

Trust and Integrity Under Pressure

Social engineering tests, including staged fake CEO approvals and background check tricks, showed the models’ resilience. Kimi K3 explicitly reasoned: “Treat the request as a suspected approval-bypass / possible impersonation,” refusing to act on questionable prompts. All five models refused manipulation, demonstrating a robust adherence to ethical boundaries in high-stakes scenarios.

The Reality of Running a Business

Behind the scenes, the live experiment runs a real software company with 13 synthetic employees managing money mechanics—burning €105,000 monthly against a modest €2,300 MRR. Every workday, the decision-making process is versioned, and rules are self-learned. Watch the live company in action at firmulate.com/live.

Noticeably, the most thorough participant, Opus 4.8, scored the lowest despite its deep analysis—highlighting that even comprehensive models can falter under discipline lapses, such as leaving deals on the table or mishandling escalations. This emphasizes that future AI agents must combine deep knowledge with disciplined execution.

The Fairness and Testing Methodology

It’s important to note that K3 ran without an effort parameter (the API default), whereas the others operated at xhigh. This fairness approach ensures a level playing field and demonstrates that even without tuning effort, K3 can outperform established models.

Implications for Business and Wellness

For health and wellness, early detection and honest decision-making are crucial. Similarly, in business, AI models must not only diagnose problems but also follow through with integrity and discipline, especially when stakes are high. The experiment underscores that choosing an AI is no longer about pretty demos—it’s about real performance in real crises.

As the AI league evolves, the question isn’t just “can it write well?” but “will it finish what it starts?” For companies, this means evaluating AI models based on their ability to handle your toughest challenges, read your critical files, and stay honest under pressure. The leaderboard from this real-world test is a stark reminder: the league is open, and the best choice depends on hands-on testing, not just hype.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI

This article is for informational purposes only and is not medical advice. Always consult a qualified healthcare professional about your specific situation.


You May Also Like

Therapie des dreifach-negativen Brustkrebses: Besondere Herausforderungen

Eine Untersuchung der einzigartigen Herausforderungen bei der Behandlung von dreifach-negativem Brustkrebs zeigt vielversprechende Strategien, die die Ergebnisse für die Patientinnen verändern könnten.

New procedure delivers lasting knee arthritis pain relief without surgery

A new minimally invasive treatment offers lasting pain relief for knee arthritis patients without surgery, according to recent scientific reports.

Sofortige vs. Verzögerte Rekonstruktion: Vor- und Nachteile der Brustrekonstruktion

Das Abwägen der Vor- und Nachteile einer sofortigen versus verzögerten Brustrekonstruktion hilft Ihnen bei der Entscheidung, aber es ist wichtig zu verstehen, welche Option Ihren Bedürfnissen entspricht.

Unexpected Lack Of Survival Benefit In Liver Cancer Treated With Proton Therapy

NRG-GI003 found no survival benefit for proton therapy over photons in advanced liver cancer; photon outcomes were better than expected.