AI Testing

What Is AI Testing?

AI testing covers two related practices: using AI models to generate, run, and maintain software tests, and testing AI systems themselves, whose outputs vary from run to run and resist traditional pass-fail assertions. Both aim at the same goal, catching defects before users do, but each demands different techniques because deterministic and probabilistic software fail in different ways.

Key Takeaways

• The term has two meanings in practice: AI that writes and maintains tests for your software, and the discipline of verifying AI-powered features, which usually means LLM evals.
• AI-generated tests are cheap to produce and expensive to trust. A suite written from the implementation will happily lock in the bugs that implementation contains.
• The highest-value use is test maintenance, keeping suites green through refactors and updating brittle assertions, because that is the toil humans avoid.
• Testing AI features requires scoring outputs against criteria, not asserting exact strings, since the same input can legitimately produce different valid answers.

How It Works

On the test-authoring side, an AI coding agent reads the code under test, or better, the specification for it, and produces unit tests, integration tests, or end-to-end scripts. Given a failing build, the same agent can read the error output, distinguish a broken test from broken code, and repair whichever is actually wrong. Some teams run agents against the application itself: a browser agent clicks through user flows, tries malformed inputs, and reports what breaks, which is exploratory testing at machine speed. Enterprises moved on this early: in GitHub's 2024 survey of 2,000 enterprise developers across four countries, over 98% said their organizations had experimented with AI for generating test cases, and 92% of U.S. respondents used it at least sometimes [1].

The critical design decision is what the tests are generated from. Tests derived from a spec with acceptance criteria verify intended behavior. Tests derived from the source code only verify that the code keeps doing what it currently does, which is useful as a regression net during refactoring work but says nothing about correctness. Mature teams treat generated tests like any other agent output: they pass through AI code review before anyone relies on them.

Testing AI systems inverts the problem. A feature backed by a language model cannot be asserted with expect(output).toBe(...) because outputs are nondeterministic. Instead, teams build eval suites: fixed input sets, graded outputs, scoring by rules, similarity measures, or a judge model, with thresholds wired into CI so a prompt or model change that degrades quality fails the build.

Example

A team inherits a payments service with 12 percent test coverage. An engineer writes a short brief per module describing the intended behavior, including edge cases like partial refunds and currency rounding, then has an agent generate tests from the briefs rather than the code. The first run surfaces three real bugs where the implementation disagrees with the brief, exactly because the tests were not derived from the implementation. The engineer reviews the generated suite, deletes a handful of tests that assert incidental details, and merges. Coverage lands at 78 percent, and two of the three bugs turn out to be live in production.

What People Get Wrong

The common mistake is reading AI-generated coverage numbers as quality. An agent can take a module to 95 percent coverage in an afternoon, but if the tests were inferred from the implementation, every existing defect is now enshrined as expected behavior, and the suite will actively resist the fix. Coverage measures which lines ran, and nothing about whether the assertions encode intent. A smaller suite generated from a written spec is worth more than a large one generated from the code.

AI unit testing

AI unit testing is the narrowest and most adopted slice of this practice: pointing a model at a single function or class and asking for unit tests. Individual habits lag the enterprise experiments, though: in the 2025 Stack Overflow Developer Survey, 17.9% of developers said they mostly use AI for writing test code and another 27.5% use it partially, while 44.1% have no plans to adopt AI for testing at all [2]. It works well because the scope is small enough to fit cleanly in a context window and the feedback loop is immediate, the tests either compile and pass or they do not. The habits that make it useful carry over from the broader discipline: state the intended behavior in the prompt instead of letting the model infer it, ask for edge cases and failure paths explicitly, and read the assertions before you commit them. Treat the output as a draft written by a fast junior who has never met your users.

FAQ

Can AI fully replace QA engineers? No. Agents excel at generating cases, maintaining suites, and brute-force exploration, but deciding what matters to test, what level of risk is acceptable, and whether a strange behavior is a bug or a feature remains human judgment. QA work shifts toward designing test strategy and reviewing agent output.

How do you test nondeterministic AI features? With evals rather than exact assertions: a fixed dataset of inputs, a scoring method (rules, similarity, or an LLM judge), and a threshold enforced in CI. You measure the distribution of quality instead of asserting a single correct answer.

Should AI-generated tests be reviewed like production code? Yes, and arguably more carefully, because a wrong test creates false confidence that outlives the session that wrote it. Review the assertions for intent, not just the code for style.

Sources

1. GitHub. "Over 98% of surveyed enterprise organizations had experimented with AI for generating test cases; 92% of U.S. respondents used it at least sometimes." https://github.blog/news-insights/research/survey-ai-wave-grows/. Accessed August 2026.
2. Stack Overflow Developer Survey. "17.9% of developers mostly use AI for writing test code, 27.5% use it partially, and 44.1% have no plans to adopt AI for testing." https://survey.stackoverflow.co/2025/ai. Accessed August 2026.