Can an AI agent open a pull request that fixes a production error, and how do I keep it safe?
Yes, for small, well-localized bugs. Give the assistant the stack trace and the crash-site file, ask for a failing test first and the smallest possible diff, and let it open a pull request, never a direct push. Run your CI, have a human review it, link the issue, and never auto-merge. Multi-file refactors and data fixes need a person.
Why it happens
A lot of production errors are small: a missing null check, an unhandled empty list, a wrong assumption about an API response. The stack trace points at the line, and the fix is a few lines. That is the shape of bug an assistant handles well, because everything it needs is in one or two files and the trace.
The risk is not that the assistant writes bad code occasionally, since people do too. The risk is a fix that looks right and is wrong: it silences the exception without addressing the cause, changes behavior elsewhere, or passes because no test covers it. A fix that swallows the error makes the dashboard quiet while the user-facing bug remains. Process, not trust, is what keeps this safe: a test that fails before the change, a diff small enough to read in a minute, CI, and a human who is accountable for the merge.
Where the fix lives matters too. If the cause is in several modules, in infrastructure, or in bad data, a patch to the crash site is a symptom fix at best. Those cases need a person to decide.
How to have an AI agent open a fix PR safely (works without any particular tracker)
Give it the trace and the crash-site file, nothing more
Paste the full stack trace, the error message, the release and the file that contains the top in-app frame. A narrow context keeps the change narrow. Do not give it credentials, production data or a broad instruction to fix everything it finds. If the trace is minified, resolve it with source maps first, or the assistant will be guessing.
Ask for a failing test first
Before any fix, ask for a test that reproduces the error and fails on the current code. Run it yourself. If the assistant cannot produce a failing test, it does not understand the bug yet, and the fix will be a guess. A passing-before test proves nothing.
Here is the stack trace and the file. First write a test that reproduces this error and fails on the current code. Do not change the source yet.Ask for the smallest diff
Instruct the assistant to change only what the failing test requires, and to explain the cause in a sentence. Reject diffs that add a broad try/catch, delete the failing line or touch unrelated files. The test should go from failing to passing because of a real change in behavior.
Open a pull request, not a push
Have the agent work on a new branch and open a pull request against your default branch. Protect that branch so a direct push or a merge requires a review and passing checks. Include in the PR description the issue link, the stack trace, the hypothesis and how the test proves it.
git switch -c fix/issue-<id> # agent edits, adds test git push -u origin fix/issue-<id> gh pr create --fillRun CI and review as you would for a colleague
Let CI run the whole suite, not only the new test. Read the diff for side effects, check that the test would have caught the original bug, and ask whether the change fixes the cause or hides the symptom. Request changes freely.
Never auto-merge, and verify after release
A human merges. After deployment, watch that the issue stops recurring in the release that contains the fix. If it comes back, reopen it and treat the fix as incomplete.
| Situation | Suitable? | Why |
|---|---|---|
| Single file, clear stack frame, reproducible with a unit test | Yes | Everything needed is in one place and a test can prove the fix |
| Error depends on a race, timing or load | Usually no | Hard to reproduce in a test, so the agent cannot show the fix works |
| Cause spans several modules or needs a refactor | No | Too large to review safely as an agent diff |
| Bad data, a migration or an infra setting | No | The code is not the problem; a person must decide |
| Third-party outage or network failure | No | Add handling deliberately; do not patch around it blindly |
| No repository access or no tests | Weak | Without a test, review is the only safety net |
A judgment checklist written for this page, not a vendor comparison.
When to use Bugsly
Bugsly has an opt-in AI fix pipeline. When a GitHub repository is connected, Bugsly fetches the crash-site files, at most two, and an AI drafts a unified diff. Strict validation follows: the diff may only modify the files it was shown and must apply exactly. If it passes, Bugsly pushes a new branch named bugsly/fix-issue-<id> off your default branch and opens a pull request. Nothing is merged automatically, and an existing branch of that name is never overwritten. Treat the PR as a proposal to review with the steps above.
The other path is your own coding agent. Connect it to the Bugsly MCP server, have it read the issue with get_issue and its events with get_issue_events, then let it open the pull request itself in your repository using your own tools and your review rules. That path keeps the model, the prompt and the test requirements under your control. On the hosted server, get_issue_events output is size-capped and omits user email, IP address, username and extra data. Bugsly's AI analysis returns a root cause, a suggested fix and a confidence number between 0 and 1, which is a hypothesis and can be wrong.
When Bugsly is not the right answer
If the fix needs a multi-file refactor, an infrastructure or configuration change, or a data repair, an automated pull request is the wrong tool: Bugsly's pipeline touches at most two files, and a patch to the crash site would only hide the symptom. If you have no GitHub repository, the Bugsly pipeline does not apply; use the manual steps above with your own host.
Ask your AI assistant to do it
With the Bugsly MCP server connected (https://bugsly.dev/api/mcp; setup for ChatGPT, Claude, Cursor and Gemini CLI is on /mcp), paste this. The assistant reads first and reports back; you decide what happens next.
Assign to Bugsly: use get_issue on my newest unresolved issue and get_issue_events with limit 3 to read the stack trace, then find the crash-site file in my repository. Draft a failing test first, then the smallest fix, as a diff plus a pull request description that links the issue. Do not push, open the pull request, resolve the issue or merge anything until I approve; show me the diff so I can approve it.
Frequently asked questions
Will an AI-opened pull request merge itself?
It should not. Protect your default branch so a merge needs review and passing checks. Bugsly's pipeline also never merges automatically.
Why ask for a failing test before the fix?
A test that fails first proves the assistant reproduced the real bug. Without it, a fix that merely silences the exception looks identical to a correct one.
How many files can Bugsly's AI fix change?
It fetches at most two crash-site files and may only modify those files, and the diff must apply exactly or it is rejected.
What if the AI fix is wrong?
Close the pull request. Nothing was merged, and the issue stays open and will keep recurring until it is truly fixed.
Sources
- Claude Code docs: Connect Claude Code to tools via MCP (read 2026-11-13)
Related
- Let an AI agent triage production errors safely
- Explain any error or stack trace
- Bugsly for AI agents
- All problems and solutions
Sentry is a trademark of its owner; Bugsly is not affiliated with or endorsed by Sentry. “Sentry-compatible” only means Bugsly accepts events sent by Sentry's open SDKs.