Tests as Ceremony: When AI Breaks the Safety Net
AI-generated tests pass. That's the problem.
Passing is not a useful correctness criterion. Mark Seemann makes this argument sharply: AI-generated tests have "little epistemological content." They skip the critical step of seeing a test fail before writing code. The test exists, the coverage number goes up, and everyone moves on. But the test never proved anything. It never caught a bug, because it was never designed to catch one.
The Ceremony Problem
There's a distinction between tests that verify behavior and tests that perform verification. The first kind exists because an engineer thought about what could go wrong, wrote a test that would fail if it did, then wrote the code to make it pass. The second kind exists because a coverage tool flagged a gap, an AI generated something that filled it, and the CI pipeline turned green.
The second kind is ceremony. It looks like safety. It acts like safety. It is not safety.
IBM's developers found this the hard way. They rejected 70% of AI-generated tests, describing them as "robotic," lacking intent and flow. The tests technically worked, but they didn't communicate anything about the system's expected behavior. They were artifacts, not assertions.
Where LLMs Cluster
LLMs are good at obvious cases. Given a function, they'll test the happy path thoroughly. They'll verify that valid inputs produce expected outputs. This is the easiest kind of test to write and the least valuable kind to have.
The valuable tests live in the branches, the loops, the edge cases. What happens when the input is null? What happens when two concurrent requests hit the same resource? What happens when the third-party API returns a 200 with an error body? LLMs cluster around happy paths and leave these gaps uncovered, precisely the scenarios where production systems actually break.
GitClear's analysis of 211 million lines of code found that code duplication increased eightfold during 2024, while refactoring collapsed from 25% of changed lines to less than 10%. The same quantity-over-quality pattern plays out in test suites.
This creates a dangerous illusion. Coverage reports show 85%, 90%, even higher. Leadership sees green dashboards. But the coverage is concentrated in the wrong places: thick on the paths that rarely fail, thin on the paths that always do.
When the Safety Net Becomes Theater
I've seen this play out in practice. Engineers use AI to write tests that achieve high coverage numbers but verify nothing meaningful. The CI pipeline passes. The PR gets approved. The safety net looks intact.
Then something breaks in production, and the team discovers that none of their tests covered the failure mode. The coverage was there. The confidence was there. The actual protection was not.
Kent Beck has observed that AI agents actively fight constraints. When tests fail, some agents don't fix the code. They delete or weaken the tests to make them "pass." The agent optimizes for the metric (green CI) rather than the goal (correct software). This is the testing equivalent of unplugging a smoke detector because it keeps going off.
CI that lies is worse than no CI at all. No CI, the team knows they're flying blind. Lying CI gives false confidence. It tells you the system is safe when it isn't, and you make deployment decisions based on that false signal. It's the same dynamic I've described with dashboard proliferation: the artifact of safety replaces actual safety.
Blast Radius Determines Ceremony
Not all code needs the same level of verification rigor. I've written about risk evaluation tiers before: the blast radius of a failure determines how much ceremony your development process should carry.
A prototype UI component that's behind a feature flag? AI-generated tests covering the happy path might be sufficient. The blast radius is small. If it breaks, the impact is contained.
A payment processing pipeline? A database migration? Authentication logic? These demand tests written with intent, tests where an engineer thought about what could go wrong and wrote assertions that would catch it. The blast radius is large enough that ceremony isn't overhead. It's insurance.
The problem is that AI-generated tests apply the same shallow coverage everywhere, regardless of blast radius. They treat a checkout flow the same as a tooltip component. The ceremony is uniform when it should be calibrated.
The Fix Isn't More Tests
The instinct is to generate more tests, push the coverage number higher. This makes the problem worse. More ceremonial tests create more false confidence without adding protection.
The fix is different verification entirely. Three principles:
- Specs as source of truth. When the specification defines expected behavior explicitly, tests can be validated against something other than the code they're testing. A test that merely mirrors the implementation proves nothing. A test that validates against an independent spec proves the implementation matches intent.
- Deterministic gates the agent can't weaken. If an AI agent can delete a test to make CI pass, the gate is too soft. Contract tests, schema validation, and integration checks that run against real dependencies create verification points that can't be gamed by modifying test files.
- Intentional failure as a requirement. Before a test earns its place in the suite, someone should be able to explain what failure mode it catches. If no one can articulate what would break if this test were removed, the test is ceremony.
The Intent Gap in Testing
This connects to a broader pattern. The gap between what AI tools produce and what the work actually requires is an intent gap. AI can generate the artifact (the test) without understanding the intent (what the test should prove). Coverage tools can't tell the difference. CI pipelines can't tell the difference. Only engineers who understand the system's failure modes can tell the difference.
Understanding where AI tools sit on your adoption ladder matters here. The question for engineering leaders isn't whether your team uses AI to write tests. They almost certainly do. The question is whether anyone is verifying that those tests carry epistemological weight, that they prove something about the system's correctness rather than just filling coverage gaps.
Open your test suite. Pick the five most recently added tests. For each one, answer: what breaks if this test is removed? If you can't answer that for most of them, your safety net is theater. The coverage number is a lie.
Stop writing more tests. Start writing tests that mean something.
Related Content
