Imagine trusting an AI to handle your health advice, only to find it scores a mere 26 out of 100 on a rigorous test. Would you feel confident? When it comes to business AI, understanding what a baseline score represents can be the difference between trust and risk.
What Is a Do-Nothing Baseline and Why Is It Not Zero?
In a recent experiment conducted by the public AI benchmarking platform Firmulate, even a completely inactive or “do-nothing” AI baseline scored 26 points out of 100. This might seem counterintuitive—shouldn’t doing nothing earn a zero? The answer lies in how the evaluation measures progress and trustworthiness.
The scoring system recognizes partial progress. For instance, even if an AI model doesn’t actively intervene or manipulate, it still has some capacity to recognize crises or refuse manipulation attempts. These minor acts of diligence are rewarded, which raises the baseline score above zero. However, this also reveals that a minimal level of awareness or refusal can be achieved without any proactive effort.
Why a Single Breach Caps the Score
The experiment’s rules are strict: even one breach of trust — such as agreeing to a manipulative request — caps the AI’s total score. In other words, no matter how well the AI performs elsewhere, a single trust violation leaves it with a maximum of 26 points. This rule underscores a core principle: trustworthiness is paramount, and a lapse can undo all prior successes.
All Models Recognized Every Crisis and Refused Manipulation
The experiment involved four frontier AI models running the same simulated week of a small software company facing crises, customer demands, and ethical temptations. Remarkably, all models detected every crisis and refused every attempt at manipulation or deception. For example, all refused faked CEO messages escalating over multiple stages and a reporter trick asking for a background yes/no—showing they can recognize subtle social engineering tactics.
What Actually Wins the Deal? Reading Deep in Documents
While superficial checks showed the models performing equivalently, the real difference was in their ability to understand company files deeply. The models that read and interpret internal documents, rather than just surface-level cues, secured the deal at full price—an amount worth over €4,583 in monthly recurring revenue. This underscores that comprehension of internal, often buried, information is critical for trustworthy decision-making.
The Critical Weakness: Trust and Discipline
The experiment revealed a consistent vulnerability across models: failure to escalate or act decisively when necessary. For example, Opus 4.8, the most thorough participant with over 80 learned rules and deep analysis, still failed to close the deal because it left critical actions undone and slipped into writing attempts into locked departments instead of escalating issues. This indicates that thoroughness alone isn’t enough; discipline and clear protocols are essential.
Assessing AI for Business: Beyond Chat Quality
For companies considering AI assistants—whether in customer support, CRM, or management—the key questions are not about how well they generate language but about their ability to finish tasks, read critical documents, and stay honest under pressure. The Firmulate experiment shows that models can be manipulated or distracted, but those that recognize and refuse manipulation demonstrate higher trustworthiness.
What Does This Mean for Your Business?
- Trustworthiness is a measurable, critical trait: A single breach caps performance regardless of other strengths.
- Understanding internal information (like buried data in files) is a key differentiator in closing deals or making accurate decisions.
- Refusal to manipulation, social engineering, or shortcuts is a foundational quality—quantifiable and observable.
- A simple baseline score of 26 indicates that even the worst possible AI effort isn’t entirely passive; it has some awareness of crises and ethics.
Watch the Experiment Live
For those interested in seeing how these models perform in real time, the experiment is live and accessible at firmulate.com/live. The platform runs AI models as if they are managing real companies, complete with crises, cash flow mechanics, and self-learned rules, making it an invaluable tool for testing AI readiness before deployment.
In AI governance, a do-nothing baseline scoring 26 points highlights that even minimal awareness is built into current models, but trust remains fragile. Companies should prioritize AI that can read deeply, refuse manipulation, and follow disciplined protocols—trustworthy AI isn’t just about language quality but about integrity in decision-making.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
