Writing unit tests is where LLM quality gaps show up fast. A model that sounds confident in a chat session can still hallucinate method signatures, generate tests that only exercise the happy path, or miss the one edge case that actually fails in production. The surface is narrow enough that you know quickly whether the model understood the code or just understood the shape of unit tests.
This post ranks five API-level models on what each produces, what each costs per 1,000 tests, and which one a solo dev running a CI budget should actually pick.
The candidates
Pricing verified August 2026 from Anthropic{target="_blank" rel="noopener"}, OpenAI{target="_blank" rel="noopener"}, and Google AI Studio{target="_blank" rel="noopener"}.
| Model | Provider | Input $/M | Output $/M | Tier |
|---|---|---|---|---|
| Claude Fable 5 | Anthropic | $10 | $50 | Frontier |
| GPT-5.5 | OpenAI | $5 | $30 | Frontier |
| Claude Haiku 4.5 | Anthropic | $1 | $5 | Budget |
| Gemini 3.5 Flash | $0.75 | $4.50 | Budget | |
| GPT-4o-mini | OpenAI | $0.15 | $0.60 | Cheapest |
A typical unit test request: 500 tokens of input (the function, its docstring, relevant imports) and 300 tokens of output (a test file with 3 to 6 cases). The cost math scales linearly from there.
What separates useful tests from merely passing tests
Three failure modes make generated tests worthless even when they technically run:
API hallucination: the model invents method names or constructors that don't exist in your codebase. The test file is syntactically valid; it just imports something it invented. You catch it on the first run, but it still wastes cleanup time.
Happy-path only: every test hits the expected success case. No null inputs, no boundary values, no error branches. Coverage metrics look fine until the next bug report arrives from a case no test touched.
Hardcoded expectations: the model guesses the expected return value and embeds it as a literal. When the underlying logic changes, the test breaks, and you're now maintaining two copies of the same assumption.
The models that avoid these failure modes share one trait: they read the full function and trace what each branch actually returns before generating assertions. That takes both enough context window to load the function plus its dependencies, and enough reasoning quality to follow conditional paths. Budget models handle short, flat functions well. The gap widens on anything with multiple early-return paths or shared state.
How each model performs
This ranking draws on our code review benchmarks, Claude vs GPT-5.5 coding head-to-head results, and known model capabilities on code-adjacent tasks. Note: a unit test generation tournament is not included here (the LLMTest runner is offline pending a credit top-up), so no test-specific win rates are cited. Only what published data supports.
Claude Fable 5 is the pick when test quality is the constraint. In our code review benchmark, Claude models consistently caught subtle reasoning errors: the race condition that only surfaces under concurrent access, the authorization gap that exists on every request but nobody tested. That same careful reading of control flow applies to test generation. Fable 5 produces assertions that target branching logic rather than just the expected success case. The tradeoff is cost: at $10/$50 per million tokens, 1,000 tests cost $20. That's reasonable for payment or authentication logic; it's expensive for scaffolding coverage across new code.
GPT-5.5 is competitive, particularly on TypeScript-heavy codebases. It tends to produce shorter, less exploratory tests than Fable 5. Edge case coverage is solid but narrower; you get null checks and basic boundary values but rarely the second-order effect when two conditions interact. At $5/$30, a 1,000-test run costs $11.50. Worth running against your specific codebase before committing; some TypeScript projects see GPT-5.5 match Fable 5 output quality at nearly half the price.
Claude Haiku 4.5 is the value pick for most solo devs. In our code review benchmark, Haiku 4.5 beat GPT-4o 5-0 on finding real bugs in production code at one-tenth the per-call cost. That reasoning quality carries into test generation: it reads enough of the function to avoid the worst hallucinations and generally includes at least one non-happy-path case per function. A 1,000-test run costs $2.00. For a CI pipeline that generates tests on new functions with every PR, that's a sustainable budget.
Gemini 3.5 Flash is the budget pick when test volume is the constraint. At $0.75/$4.50 per million tokens, a 1,000-test run costs $1.73. Flash produces structurally sound test files reliably, but more often stops at the happy path, especially on functions with multiple early-return branches. If you're generating tests to push coverage from 20% to 60%, Flash does the job at a cost that scales. For the harder push from 75% to 90% (the edge cases that matter most), Haiku produces more complete test sets for an extra $0.27 per 1,000 tests.
GPT-4o-mini at $0.15/$0.60 per million costs $0.26 per 1,000 tests and is the right tool for one specific job: scaffolding. It reliably generates a test file with the correct imports, a properly structured describe block, and stub test names based on the function's documented behaviors. The gap shows up on assertion quality. Treat it as a starting point you then fill in manually, or pass to a better model for the actual assertions.
Cost per 1,000 tests
At 500 input tokens and 300 output tokens per test:
| Model | Cost per 1,000 tests | 10,000 tests/month |
|---|---|---|
| Claude Fable 5 | $20.00 | $200 |
| GPT-5.5 | $11.50 | $115 |
| Claude Haiku 4.5 | $2.00 | $20 |
| Gemini 3.5 Flash | $1.73 | $17.30 |
| GPT-4o-mini | $0.26 | $2.60 |
At 10,000 generated tests per month (a reasonable volume for a solo dev automating coverage on new code), Haiku 4.5 and Flash are both under $20. The step down to GPT-4o-mini saves $17.40 per month but trades away edge case coverage. The step up to Fable 5 adds $180 per month over Haiku, which is only worth it for code paths where a missed edge case means a production incident.
Subscription vs API
| Provider | Model | API cost/1k tests | Subscription |
|---|---|---|---|
| Anthropic | Claude Fable 5 | $20.00 | Claude Pro $20/mo, Max $100-200/mo{target="_blank" rel="noopener"} |
| OpenAI | GPT-5.5 | $11.50 | ChatGPT Plus $20/mo, Pro $200/mo{target="_blank" rel="noopener"} |
| Anthropic | Claude Haiku 4.5 | $2.00 | Included in Claude Pro |
| Gemini 3.5 Flash | $1.73 | Google One AI Premium $19.99/mo{target="_blank" rel="noopener"} | |
| OpenAI | GPT-4o-mini | $0.26 | Included in ChatGPT Plus |
The subscription is the right default for interactive use: generating tests file by file through Cursor, Claude Code, or a browser session. The API makes sense when you're automating the generation step, specifically a script that runs on every new function or a CI job that generates tests for files landing below a coverage threshold.
For interactive use at typical solo volumes (under a few hundred functions per month), any $20/month subscription covers the usage comfortably. For a CI pipeline generating tests automatically, even a light setup pays back the API approach over a subscription within a few weeks. At Haiku 4.5's rate, 10,000 auto-generated tests per month costs $20 via the API vs. $20/month for a subscription that also has to cover all your other Claude usage.
Which to use
Coverage scaffolding on new code: Claude Haiku 4.5. At $2 per 1,000 tests, it covers the happy path plus edge cases reliably, without hallucinated imports. The default for any automated test generation pipeline.
Payment, auth, and critical-path logic: Claude Fable 5. The quality ceiling matters when a missed edge case means a production incident. Limit Fable 5 to the functions where that trade-off pays.
TypeScript-heavy projects: test GPT-5.5 against your specific codebase first. At $11.50 per 1,000 tests it sometimes matches Fable 5 output quality on TypeScript-native patterns.
High-volume coverage from scratch: Gemini 3.5 Flash. Best for pushing from 20% to 60% coverage at $1.73 per 1,000 tests. Gets you there cheaply; switch to Haiku for the edge-case-dependent next 20%.
Test scaffolding and structure only: GPT-4o-mini. Use it to generate the file skeleton; fill in the assertions with a better model or by hand.
To wire the generation step into your CI pipeline (running a test generator on new functions before merging), the LLM eval setup in Node.js covers the pattern for running any LLM quality check on every PR. Route test generation calls from a single endpoint with per-call cost tracking at LLMTest, or run a free benchmark against your own functions to see which model's output your CI actually needs.