What Is Distributed Tracing?

What Is Distributed Tracing?

Distributed tracing tracks requests as they flow through multiple services in a distributed system, giving you end-to-end visibility.

The Problem

In a microservices architecture, a single user request might touch 5-10 services. When something is slow or fails, you need to know which service caused the issue. Logs from individual services don't show the complete picture.

How Tracing Works

Every request gets a unique trace ID that follows it through every service:

User Request → API Gateway → Auth Service → Order Service → Payment Service → Database
     ├─── trace_id: abc123 ─────────────────────────────────────────────────────┤
     ├─ span: gateway (50ms) ─┤
                               ├─ span: auth (20ms) ─┤
                                                      ├─ span: order (150ms) ────┤
                                                                ├─ span: payment (80ms) ─┤
                                                                        ├─ span: db (15ms) ─┤

Key Concepts

• Trace — represents the entire request journey, identified by a trace ID
• Span — a single operation within a trace (e.g., one service call or database query)
• Context propagation — passing trace/span IDs between services via HTTP headers

Implementation

Most frameworks support the W3C Trace Context standard:

# Incoming request headers
# traceparent: 00-abc123-def456-01

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("process_order") as span:
    span.set_attribute("order.id", order_id)
    result = process(order_id)
    span.set_attribute("order.total", result.total)

What Tracing Reveals

• Latency bottlenecks — which service is the slowest in the chain
• Error propagation — where an error originated vs. where it surfaced
• Service dependencies — actual runtime dependencies (not just what's documented)
• Retry storms — cascading retries that amplify failures

When You Need Tracing

• Running more than 3 services
• Debugging latency issues across service boundaries
• Understanding request flow in complex architectures
• SLA monitoring for end-to-end request processing

Tracing + Error Tracking

Distributed tracing shows the path; error tracking shows what went wrong. Bugsly connects errors to traces, so when an exception occurs, you see not just the stack trace but the entire request journey that led to the failure. This combination dramatically reduces mean time to resolution for distributed system issues.