Beyond the first Sentry automation: dedupe, prioritize, and verify fixes

Wiring Sentry to a coding agent takes an afternoon. Then you have an agent that investigates every error, including the ones that don't matter. A recurring error can produce a lot of agent activity without producing a fix. The same stack trace arrives again, another investigation starts, and two branches end up changing the same handler. Meanwhile the error that affects checkout is waiting behind an exception from a browser extension.

The first Sentry automation gets an error into a coding workflow. The next improvement is deciding which errors deserve a run and what evidence that run needs to produce.

Investigate before patching

Start with read access to the error tracker. Reading issues, events, and releases is enough for an investigation. Commenting and resolving are separate permissions to grant later, if at all.

Begin with a prompt that reads the supplied error, checks the repository, and reports a proposed next step. Let it edit only after it has a reproducible case or a specific, reviewable explanation of the cause.

For example, a missing customer identifier in a request might indicate a frontend regression, an old client, or malformed traffic. The stack trace alone doesn't tell you which. Ask the agent to identify the affected release and execution path before selecting a patch.

Check whether someone is already fixing it

An error identifier gives a run something concrete to match. Include it in the automation input and in the resulting thread summary. Ask the agent to inspect relevant open pull requests and recent changes for the same issue when those tools and permissions are available.

That check helps avoid duplicate work, but it isn't a concurrency lock. Two webhook deliveries can start before either run has published a result. For a high-volume alert source, handle deduplication in the event-delivery system or use a controlled scheduled sweep. Most automation systems start a new run for every trigger; don't assume yours deduplicates equivalent error reports for you.

A useful first assignment is:

Inspect this error and look for an existing PR or recent fix that addresses the same failure. Report the evidence you compared. If relevant work already exists, summarize it and stop before editing. Do not resolve the Sentry issue or post a comment.

Have the agent report findings in its own run log. That works without granting permission to change your incident tracker.

Give the agent your priorities

Frequency helps, but frequency alone doesn't establish impact. A noisy analytics failure and a rare payment failure deserve different responses. Give the agent your actual priority rules, including which routes matter and when to escalate to a person.

For a small initial workflow, restrict eligibility to one application, one error family, and a known release window. Ask it to report missing impact information. Avoid an open instruction to "fix the most important errors" when importance only exists in someone's head.

Keep production data out of reproduction fixtures. Prefer synthetic values that preserve the relevant shape of the failing request. If a failure depends on data the agent cannot safely access, the useful output is a precise request for that missing condition.

Reproduce the failure first

For a hypothetical null-handling bug, a regression test should exercise the missing value and show the expected response. Ask for the original failure and the result after the change using the same test. Then inspect whether the patch suppresses the exception by discarding work that the application still needs to perform.

A suggested implementation prompt:

Reproduce the reported failure with synthetic data and add a regression test. Make the smallest change that preserves the documented behavior. Run the relevant tests and describe any checks you couldn't complete. If the intended behavior is ambiguous or the failure won't reproduce, stop with an investigation summary. Open a draft PR for a verified patch; leave merging and Sentry resolution for review.

Keep auto-merge off for this workflow and use branch protections to require the intended review. Check the repository's merge settings before enabling the automation.

Review the automation's run history

After the first few runs, inspect duplicate investigations, unresolved approvals, and findings that never produced a patch. You need the run record for failures too, because a provider limit or a broken environment looks like a missed diagnosis if all you see is "no PR." If you don't want to build that record yourself, Hoplite's automation run history keeps failed runs and labels the cause.

If you haven't built the first version yet, the original Sentry walkthrough covers it end to end; scope that one to a single error family. For an existing setup, start by adding one check for prior work and one explicit stop condition for failed reproduction. Review the next run's evidence before allowing it to handle a broader group of errors.