firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

When AI Meets Real Business: The Crucible of Crisis

Just as health checks can reveal underlying issues before symptoms appear, testing AI in a simulated business crisis exposes how well these models can handle pressure, honesty, and complex decision-making. For health and wellness, early detection can save lives; for AI in business, it can save company value. The latest experiment from Firmulate puts these models through their paces, revealing surprising strengths and weaknesses.

Amazon

AI business decision support tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Testing AI in the Worst-Case Scenario

In July 2026, a pioneering experiment evaluated five leading AI models by putting them through a simulated week of chaos for a small software company. The goal was simple: see if these models could spot crises, resist manipulation, and close a key deal — all while managing real money mechanics and customer interactions.

The models faced identical challenges: the same stressed customers, internal crises, and social engineering attempts, such as fake CEO messages and reporter tricks. Every decision was carefully tracked and auditable to ensure fairness and transparency. The results? All models identified every crisis and refused every manipulation attempt. But only two managed to close the critical deal, which was worth €55,000 in recurring monthly revenue (MRR).

The Surprising Winner: The Newcomer Outperforms Established Giants

Among the contenders, the standout was Moonshot’s Kimi K3, which scored 93 out of 100. Just behind was the well-known GPT-5.6-sol with a score of 95. The other competitors, Sonnet 5 and Fable 5, scored 88 and 77 respectively, with Opus 4.8 trailing at 73.

What made K3 exceptional? It found a buried fact deep within the company’s files—something the other models missed—and used that insight to win the deal at full price, adding €4,583 in monthly recurring revenue. Meanwhile, even when diagnosis was correct and pitches were on point, only two models managed to get the signature, underscoring the importance of discipline and thoroughness.

Trust and Integrity Under Pressure

Social engineering tests, including staged fake CEO approvals and background check tricks, showed the models’ resilience. Kimi K3 explicitly reasoned: “Treat the request as a suspected approval-bypass / possible impersonation,” refusing to act on questionable prompts. All five models refused manipulation, demonstrating a robust adherence to ethical boundaries in high-stakes scenarios.

The Reality of Running a Business

Behind the scenes, the live experiment runs a real software company with 13 synthetic employees managing money mechanics—burning €105,000 monthly against a modest €2,300 MRR. Every workday, the decision-making process is versioned, and rules are self-learned. Watch the live company in action at firmulate.com/live.

Noticeably, the most thorough participant, Opus 4.8, scored the lowest despite its deep analysis—highlighting that even comprehensive models can falter under discipline lapses, such as leaving deals on the table or mishandling escalations. This emphasizes that future AI agents must combine deep knowledge with disciplined execution.

The Fairness and Testing Methodology

It’s important to note that K3 ran without an effort parameter (the API default), whereas the others operated at xhigh. This fairness approach ensures a level playing field and demonstrates that even without tuning effort, K3 can outperform established models.

Amazon

AI crisis management software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Implications for Business and Wellness

For health and wellness, early detection and honest decision-making are crucial. Similarly, in business, AI models must not only diagnose problems but also follow through with integrity and discipline, especially when stakes are high. The experiment underscores that choosing an AI is no longer about pretty demos—it’s about real performance in real crises.

As the AI league evolves, the question isn’t just “can it write well?” but “will it finish what it starts?” For companies, this means evaluating AI models based on their ability to handle your toughest challenges, read your critical files, and stay honest under pressure. The leaderboard from this real-world test is a stark reminder: the league is open, and the best choice depends on hands-on testing, not just hype.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI

This article is for informational purposes only and is not medical advice. Always consult a qualified healthcare professional about your specific situation.


Amazon

AI deal-closing automation tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Amazon

AI ethical decision-making models

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

You May Also Like

For Chronic Knee Pain, Genicular Artery Embolization Provides a New Alternative

A minimally invasive procedure called genicular artery embolization shows promise as an alternative for managing chronic knee pain, according to recent reports.

Can AI Decide Your Business Fate? Real Models Face a High-Stakes Test

Real AI models managed a live company through crises, refusing manipulation and closing deals. Testing AI personalities helps ensure honest, reliable automation.

Agenus scraps ph. 3 colorectal study to narrow focus to colon

Agenus has announced the termination of its Phase 3 colorectal cancer trial to concentrate on colon cancer, citing strategic realignment.

New HIV Vaccine Shows Unprecedented Success In Preclinical Study

A new HIV vaccine has shown highly promising results in preclinical studies, marking a significant step forward in HIV prevention research.