All posts

89,499 production errors from 16 apps: what our own error tracker showed us

First-party data from 16 of our own apps: 1,195 issues, 12 of them 79% of the volume, 39% seen once, and how much an AI agent could fix from a stack trace.

TL;DR

  • Between 2026-03-13 and 2026-11-17, 16 of our own apps sent 89,499 error occurrences to Bugsly. They grouped into 1,195 issues. Twelve issues (1%) made up 78.7% of the volume; 465 issues (38.9%) happened exactly once.
  • The biggest categories were upstream HTTP 4xx/5xx errors (32.5% of volume), plain code bugs such as type and null errors (21.4%) and ClickHouse limits (15.6%). Postgres connection and pool errors were 0.8%; Redis was 0.2%.
  • I read the 12 largest issues by hand. Two (19.4% of all volume) needed only a code change. Six (48.3%) needed config, infrastructure or a migration that a human applies. Four (11.0%) carried no usable signal.

We build Bugsly, a hosted error tracker that accepts Sentry SDKs, and run it on our own products. This is what that data says; the queries to reproduce every number are published at bugsly.dev/data/production-errors-16-apps-queries.md.

One caveat first. 56,302 of the 89,499 occurrences (62.9%) came from a single project: Bugsly's own Python backend (the project is still called Watchdog). Read this as what happens in a small portfolio of products, not as an industry benchmark.

What we counted

The scope is one organization (ours) with 17 projects; 16 reported at least one error: 11 Next.js, 2 Node, 2 Ruby and 1 Python. The Python project is the 62.9% outlier above; Next.js apps contributed 27.1% (24,223) of the occurrences across 955 issues.

Totals use each issue's occurrence counter, which has run since 2026-03-13. The raw events in ClickHouse expire after 90 days, so they are used only for the latency numbers further down. All other organizations are excluded, including one external tenant with a much larger volume.

Where the volume went

I bucketed every issue by pattern-matching its title. The buckets are heuristic; the rules are in the published queries.

CategoryIssuesOccurrencesShare
HTTP 4xx/5xx from an upstream API9329,04932.5%
Plain code bugs (type, null, import)9019,18921.4%
ClickHouse limits and connectivity3213,95315.6%
Network failures (fetch, DNS, socket)379,09110.2%
Schema drift (missing column or table)204,5975.1%
Framework, proxy and ORM misc903,5183.9%
Auth, tokens, unverified identities553,1693.5%
Quota, credits, rate limits632,3492.6%
React hydration mismatch1321,0601.2%
Postgres connection or pool486730.8%
Redis82110.2%
Other (long tail)5272,6402.9%

Volume is concentrated, and a lot of issues are one-offs

The top issue is 24.5% of all occurrences; the top 3 are 50.7%, the top 10 are 75.9%. At the other end, 465 issues were seen once: 0.5% of the volume but 38.9% of the list you would have to triage.

The biggest source was one of our own services refusing another

Twenty-seven issues name our own email API in the title. They total 23,725 occurrences, 26.5% of everything. By volume, nearly all of it is the backend's billing and notification tasks getting a 403 from that API. The issue data does not record why, so I will not guess. What it does show is an uncomfortable pattern: the alerting path itself was the loudest failure.

The classic suspects were rare

Postgres connection and pool issues: 48 issues, 673 occurrences. Redis: 8 issues, 211. Issues whose title names OpenAI: 8 issues, 81 occurrences. Any AI or media provider (OpenAI, Gemini, Veo, TTS and similar): 55 issues, 5,474 occurrences (6.1%).

Grouping was the bigger problem. React hydration errors produced 132 separate issues for one class of bug, because the minified message carries variable arguments. Normalizing digits and cutting titles to 70 characters collapses all 1,195 issues into 556 title families. Fix grouping before you automate anything.

Do issues come back?

Of 1,195 issues, 1,069 are resolved, 59 (4.9%) are regressions (marked resolved, then seen again), 44 are ignored and 23 are unresolved. The regressions account for 13.6% of occurrences. "Resolved" here means someone clicked resolve; we do not record whether a code change shipped, so this is not a fix rate.

How fast does an error arrive?

I can measure one part of "time to alert" and not the other.

Measured: the gap between the SDK's event timestamp and the moment Bugsly received the event, for 15,190 stored events. Median 108 ms, 90th percentile 17.2 s, 99th percentile 154.4 s. The timestamp comes from the client, so offline queues in mobile apps and clock skew inflate the tail.

Not measured: the time from event to alert email. Alert delivery is not timestamped in our data, so I will not publish a number. (32 active rules cover these projects: 16 new-issue, 16 regression, all email.)

What could an AI agent fix?

We have not measured an agent's fix rate on this data. What follows is my own triage of the 12 largest issues (70,418 occurrences, 78.7% of the total), judged from the title and stack frame. "Code only" means the stack trace is enough to write the patch.

#IssueOccurrencesWhat closes itBucket
1Our email API returned 403 to a billing task21,926Fix a credential or sender settingConfig
2TypeError: object of type 'bool' has no len() in an event insert12,348Add a type guard in one functionCode only
3ClickHouse memory limit exceeded (code 241)11,123Tune the query or raise limitsInfra
4Browser Failed to fetch5,371Nothing to fix in codeNo signal
5Video API returned 400 for an unsupported parameter4,984Remove the parameterCode only
6Celery KeyError: task not registered3,541Align worker and scheduler versionsConfig
7Postgres column does not exist3,147Apply the missing migrationMigration
8AWS SES InvalidSignatureException2,273Check credentials and regionConfig
9Browser Failed to fetch (second source)1,742Nothing to fix in codeNo signal
10Browser Failed to fetch (injected script)1,483Nothing to fix in codeNo signal
11Unknown Error with no frames1,245Add instrumentation firstNo signal
12Same 403 as #1, in the notification dispatcher1,235Same fix as #1Config

Code only: #2 and #5, 17,332 occurrences (19.4% of all volume). Config, infra or migration: #1, 3, 6, 7, 8 and 12, 43,245 (48.3%). No signal: #4, 9, 10 and 11, 9,841 (11.0%).

My read: an agent that can read a stack trace and your repository helps on the one-fifth of volume that is a code change. For the rest, the useful output is a diagnosis ("this migration was never applied") that a human still has to act on.

How we wired it

Every app uses a stock Sentry SDK. The only change per app was the DSN:

Sentry.init({
  dsn: "https://<public_key>@bugsly.dev/<project_id>",
});

Stored events show the mix: sentry.javascript.nextjs (2,690), sentry.python.fastapi (1,058), sentry.ruby.sidekiq (709), sentry.ruby (706), sentry.dart (613). Guide: bugsly.dev/migrate-from-sentry.

For agents, Bugsly has an MCP server. In Claude Code the hosted version is one command, followed by an OAuth consent page:

claude mcp add --transport http bugsly https://bugsly.dev/api/mcp

The local package takes a token instead: claude mcp add bugsly -e BUGSLY_TOKEN=<token> -- npx -y @bugsly/mcp-server. Triage uses list_issues, get_issue_events, search_errors and resolve_issue; the hosted server caps event size and omits user email, IP and username. A prompt that works: "Use Bugsly to read the most frequent unresolved issue, find the failing line in this repository, and propose a fix. Do not change the issue's status." Bugsly does not write the patch; your editor's agent proposes it and you review it.

Frequently asked questions

What are the most common production errors?

In our data: upstream HTTP 4xx/5xx errors (32.5% of occurrences), plain code bugs such as type and null errors (21.4%) and ClickHouse limits (15.6%). This is a case study of 16 apps, not a benchmark.

How much of the error volume is noise?

465 of 1,195 issues (38.9%) were seen exactly once, but they are only 0.5% of occurrences. Four of the 12 largest issues (11.0% of all occurrences) had no usable signal: browser Failed to fetch errors and an Unknown Error with no stack frames.

How quickly does an error tracker receive an error?

For 15,190 stored events the median gap between event timestamp and receipt was 108 ms, the 90th percentile 17.2 s and the 99th 154.4 s. The tail is likely client-side (offline queues, clock skew). This is ingest latency, not time to alert.

Can an AI agent fix production errors on its own?

Partly. In my manual triage of the 12 largest issues, 19.4% of all occurrences needed only a code change a stack trace is enough to write, and 48.3% needed configuration, infrastructure or a migration that a person must apply. I did not run an agent on this data, so this is not a measured fix rate.

Do I need to rewrite my instrumentation to try Bugsly?

No. Bugsly accepts Sentry SDK envelopes, so you change the DSN and keep the SDK. Test the SDK features you rely on before moving production traffic. Sentry is a trademark of its owner; Bugsly is not affiliated with or endorsed by Sentry.

Methodology and data

  • Read-only snapshot, 2026-11-17 around 09:40 UTC, from production Postgres (issue counters) and ClickHouse (ingest latency).
  • Included: organization 1 (our own), 17 projects, 16 with at least one error, 1,195 issues, 89,499 occurrences, first seen 2026-03-13.
  • Excluded: every other organization, including one external tenant.
  • Categories are regular expressions on issue titles; the rules and every query are in the published queries. Hand triage of the top 12 is my judgment, not an agent run.
  • Ingest latency: events where receive time is after event time and within one day (15,190 of 15,291 stored events); older than 90 days is deleted.
  • Disclosure: we build and sell Bugsly. It is hosted and closed source.

Try Bugsly free, no card

Free plan, no card: 1 project, 100 new issues and 50,000 events a month, 7-day retention, 10 AI explanations a month. Paid plans from $19/month, 7-day money-back, cancel anytime.

Start free, no card

From the same team