Reviewing AI-Generated Code
AI agents can produce a complete, tidy-looking pull request in minutes. That speed shifts the hard work to the reviewer. AI code fails in different ways from human code: it's rarely sloppy, but it can be confidently, plausibly wrong. This guide builds on Conducting Effective Code Reviews with what to check specifically when a machine wrote the code.
Step 1: Hold It to the Same Bar, with More Skepticism
- The same standards apply. AI-written code meets every rule a human's would: architecture, tests, style, security.
- A person is accountable. Whoever ran the agent or accepted the suggestion owns the pull request as if they'd typed every line. "The AI wrote it" is never an explanation.
- Polish isn't evidence. Clean formatting, confident comments, and a thorough description say nothing about whether the code is correct. Read it as carefully as you would a new contributor's first PR.
- Say how it was made. Authors note in the description when a PR was substantially AI-generated, so reviewers know to apply this guide.
Step 2: Check the Scope First
Before reading any code, compare the list of changed files with the issue. Agents eagerly "improve" things they pass on the way, and an unrelated change buried in a big diff is easy to miss.
gh pr diff 123 --name-only # every changed file
gh pr view 123 # the description and linked issue- Files changed that the issue never mentions.
- Whole-file reformatting, reordering, or renaming that hides the real change.
- Edits to configuration, CI workflows, lint rules, or dependency versions.
- New abstractions or "helper" layers nobody asked for.
Ask for out-of-scope changes to be removed, not just justified. They can become their own issues.
Step 3: Know the AI Failure Modes
| Failure mode | What it looks like |
|---|---|
| Invented APIs | Calls to methods, parameters, or config options that don't exist, or that existed in an older version of the library. |
| Plausible but wrong logic | Code that reads naturally but has an off-by-one error, inverted condition, wrong time zone, or wrong rounding. |
| Tests that don't test | Assertions that can't fail, tests that mock the very thing under test, or expected values copied from the code's own output. |
| Making the checks pass instead of the code work | Weakened or deleted assertions, skipped tests, broadened catch blocks, disabled lint rules, or special cases for test inputs. |
| Swallowed errors | try/catch that logs and carries on, or default values that hide failures. |
| Reinventing existing code | A new helper that duplicates a utility already in the codebase, because the agent didn't find it. |
| Missing context | Code correct in isolation that breaks a project invariant: no tenant filter, wrong layer, ignores the architecture document. |
| Misleading comments | Comments or PR descriptions that claim something the code doesn't do. |
Step 4: Prove the Tests Actually Test
A test only has value if it fails when the code is wrong. The quickest check: put the old version of the code back while keeping the new tests, and run them. If they still pass, they aren't testing the change.
gh pr checkout 123
git restore --source=origin/Dev -- src/Inventory/ItemSorter.cs # old implementation, new tests
dotnet test --filter "FullyQualifiedName~ItemSorterTests" # these should now FAIL
git restore -- src/Inventory/ItemSorter.cs # put the PR's version backAlso read the test names and assertions against the issue's acceptance criteria. Each criterion should have a test that would catch it being broken, including the edge cases the issue lists.
Step 5: Verify Every New Dependency
AI tools sometimes suggest packages that don't exist, are abandoned, or have names close to a popular package. Attackers publish malicious packages under exactly those names. For every new or upgraded dependency in the diff, including lockfile changes:
- Find the package on its official registry (NuGet, npm, the Unity registry) and confirm it's the real, well-known one: publisher, download count, source repository.
- Check it's actively maintained and its license is compatible with the project.
- Ask whether it's needed at all. A dependency for ten lines of code is rarely worth it.
Step 6: Feed Problems Back to the Agent
Leave review comments exactly as you would for a person: specific, with the reason and the expected fix. Then decide how to proceed:
- Small fixes: have the agent address the comments on the same branch, then review the new commits.
- Wrong approach: close the pull request, improve the issue, and start again. Patching a fundamentally wrong design costs more than redoing it.
- The same mistake keeps happening: fix the source. Add the rule to the repository's
CLAUDE.md(see Writing Effective CLAUDE.md & Agent Rules) so every future session knows it.
Reviewer checklist for AI-generated pull requests:
- [ ] Changed files match the issue's scope; no unrelated edits
- [ ] Every API and method used exists in our versions of the libraries
- [ ] Logic checked against the acceptance criteria, including edge cases
- [ ] Tests fail when the change is reverted
- [ ] No weakened tests, skipped checks, or broadened catch blocks
- [ ] Errors are handled, not swallowed
- [ ] Reuses existing utilities; no duplicate helpers
- [ ] Project invariants respected (tenant scoping, layers, ARCHITECTURE.md)
- [ ] New dependencies verified as real, maintained, and needed
- [ ] Comments and PR description match what the code actually does