How to review AI-generated pull requests without becoming the bottleneck

Writing code is cheap. Reviewing it is as slow as it ever was. An agent can open pull requests faster than you can read them, and that gets less exciting when the description says "implemented the requested changes," the diff touches unrelated files, and nobody can tell you which checks ran.

The review starts with the task brief. If you ask for a bounded change and specify how to verify it, the agent can prepare most of the material you need while it works. Your review can then concentrate on the decisions that deserve a person.

Ask for a small PR with evidence

A useful completion request includes the intended behavior, the scope of the diff, and the evidence you expect. For a hypothetical retry bug:

Fix duplicate sends when the notification request times out. Keep the public API unchanged. Add a regression test for a timeout after the server accepted the request. In the PR description, explain the failure, summarize the fix, and list the checks you ran. Call out any uncertainty about the provider's behavior. Open a draft PR and don't merge it.

The timeout detail prevents a superficial fix that retries every failed response. The uncertainty request gives the agent somewhere honest to put a question it cannot answer from the repository.

Describe the goal, constrain the change, and define verification. A brief with all three also reduces how much context you have to reconstruct later.

Read the explanation before the patch

Start by checking whether the PR solves the task you assigned. Compare its description with the original behavior and acceptance criteria. If the description is vague, send it back for clarification before reading several hundred lines of code.

Then scan the changed-files list. An unexpected lockfile change may be legitimate. A rewritten authentication module in a notification fix needs an explanation. Ask the agent to separate unrelated cleanup so it can receive its own review.

A changed-files tree and per-file patches are enough for this pass. If your tool records the agent's commands, that log helps when you need to check how it reached a result. You shouldn't have to read the entire conversation to understand the PR.

Match the checks to the risk

Read the regression test as a claim about behavior. Would it fail on the original code? Does it exercise the failure that matters? A test that only confirms the new helper returns its own hardcoded value provides very little reassurance.

For the notification example, check duplicate prevention and the behavior of a legitimate second request. For a UI change, inspect the running preview. For a data migration, review how existing records behave and what happens if the migration stops partway through.

Keep these review questions nearby:

• Does the implementation satisfy the acceptance criteria without expanding the task?
• Do the tests exercise a realistic failure and the expected success path?
• Which permissions, persisted data, or external effects can this change affect?
• Did verification run against the current revision?
• What remains unverified, and does it prevent merging?

A passing test suite is useful evidence. It doesn't settle a product decision the tests never encoded.

Let a tool do the first pass

Several products run this first pass for you. Greptile and CodeRabbit read every PR, leave inline comments, and work the day you install them. They have limits. The review runs on the vendor's models and pricing, not the ChatGPT or Claude subscription your team already pays for, and you customize what they check through the settings and config files the vendor exposes. For many teams that is a fine trade. A reviewer that catches the obvious problems on every PR beats the one you meant to build.

If you want the first pass to run your checklist, in your words, against a checked-out copy of the branch, run it yourself. A Hoplite automation is a saved prompt with a webhook. A GitHub Actions step on pull_request that calls the webhook starts a run in a fresh sandbox of the project; put the pull request number from the event in the request so the prompt can tell the agent to check that pull request out (the number resolves a fork or an updated branch; a branch name alone does not). From there the reviewing agent can run the tests and open the preview instead of only reading the diff. Put the five questions above in the prompt, tell it to post one comment, and pick the model. Runs can bill to a connected ChatGPT or Claude subscription or your own API keys.

Start with a hosted tool if you have nothing today. Write your own prompt when you find yourself wanting to change what it checks.

Write feedback the agent can act on

"This feels fragile" may start a useful human conversation, but it leaves the agent guessing. Identify the condition and the desired behavior instead:

If the provider accepts the notification and the client times out, this code sends it again. Preserve the request identity across retries and add a test for that sequence. Explain how the provider handles a repeated identity before changing the retry policy.

Send that feedback back to the same agent session that produced the PR, not a fresh one. The original task context is still loaded there, so the revision needs one condition from you instead of the whole assignment again.

If you don't want to shuttle comments by hand, Hoplite's PR review loop feeds review comments and failing checks back to the agent that opened the PR. Keep auto-merge off until you've watched a few corrections land.

After the revision, review the new diff and rerun the relevant checks. A small correction can invalidate an earlier screenshot or change the assumptions behind a test.

Record who approved and why

Review completion should say whether the change is ready to merge, needs a specific correction, or cannot proceed without a decision. Avoid leaving a thread in a vague state where the agent keeps editing while the reviewer is waiting for an answer.

For your next agent PR, ask for a description that names the original failure, explains the changed behavior, and links the verification. If that makes the first review pass easier, turn the requirement into a project instruction.