AI Tool Claim Verification for Agencies

A written decision service for boutique agencies evaluating a specific AI tool, but narrowed to one thing AE can genuinely do well: verify whether the vendor's concrete performance claims hold on a small set of agency-relevant benchmark tasks with fast, objective scoring. Instead of forecasting team-wide rollout success, the deliverable is a claim-verification dossier: which promises are supported, which fail under adversarial testing, what constraints matter, and whether the tool merits a limited pilot. Inputs are public vendor claims, sample non-sensitive task specs, and the agency's evaluation criteria. AE runs adversarial multi-model debate over expected outcomes, executes a code-enforced grading loop on benchmark results, and classifies reasoning failures using its six-pattern taxonomy where appropriate: in the tool's outputs, in vendor positioning, and in evaluator overreach. This is not change-management consulting and not a broad software audit. It is a bounded pre-purchase evidence service for agencies deciding whether an AI writing, research, summarization, or workflow assistant actually performs on the tasks it claims to handle before the team wastes time piloting it.

The best case is that this is a narrow, credible, high-value evidence service for agencies: verify vendor AI claims on real tasks fast, objectively, and repeatably before they waste time and money on weak pilots.

Commander override: KILL. Commander kill: manual-labor bottleneck destroys solo feasibility. AE cannot autonomously operate third-party AI tool UIs/APIs, so the Commander must manually acquire access, learn each interface, input agency benchmarks, and extract outputs before AE can grade anything. This inverts the value proposition to "AE-as-grading-assistant-to-Commander-consulting," which is exactly the operating mode Commander is trying to escape. Also suffers from missing-middle economic squeeze (£30/mo tools do not justify £2k audits; enterprise tools get free vendor POCs) and buyer-budget constraints in the boutique-agency ICP. First Corpus S graduation; kill is diagnostic for Corpus S quality.