Testing AI-Generated Code

AI agents generate code fast, but that code needs the same (or more) testing rigor as human-written code. Here's how.

The Trust Hierarchy

Testing Strategies

1. Test-First Development

Write tests before asking the agent to implement:

"Write a function that scores articles by keyword matching.
Here are the tests it should pass: [paste test file]"

The agent implements to satisfy the tests. You verify the tests are correct, not the implementation.

2. Property-Based Testing

Define invariants the code must satisfy:

• "The filter should never reject an article with a Tier 1 keyword match"
• "The score should always be between 0 and 10"
• "The output array should never be longer than the input"

3. Snapshot Testing

For formatters and renderers, snapshot tests catch unexpected changes:

expect(formatDigestMessage(articles, '2026-01-01', 'https://test.com')).toMatchSnapshot();

4. Integration Tests

Test the boundaries between components:

• Does the ingestion layer produce valid articles?
• Does the filter accept what it should?
• Does the formatter produce valid HTML/Block Kit?

What to Watch For

Hallucinated APIs

The agent may use functions that don't exist or have wrong signatures. Always check imports and types.

Off-by-One Errors

Pagination, array slicing, date calculations — classic AI mistakes. Test boundary conditions explicitly.

Missing Error Handling

Agents often generate the happy path. Explicitly ask: "What happens if the API returns 500? What if the input is empty?"

Subtle Logic Errors

The code looks right but has inverted conditions, wrong comparison operators, or missing null checks. Type-strict TypeScript (noUncheckedIndexedAccess, exactOptionalPropertyTypes) catches many of these at compile time.

Minimum Testing for AI-Generated Code

1. Every new function needs at least one happy-path test
2. Every bug fix needs a regression test that fails before the fix
3. Every data transformation needs input/output validation
4. Every external API call needs error handling tests
5. Build must pass before committing any AI-generated code