Configuration: tracing

This page describes the tracing system of Ferron. The system covers W3C Trace Context propagation, internal trace spans, trace sampling, and trace ID response headers.

W3C Trace Context⁠#

Ferron 3 supports W3C Trace Context (traceparent and tracestate) and W3C Baggage (baggage) propagation and generation. This enables end-to-end observability by carrying trace identifiers and application-defined context across service boundaries.

Incoming trace context⁠#

By default, Ferron discards incoming traceparent, tracestate, and baggage headers. It generates a new trace ID for each request. Each request in the Ferron boundary then starts with a fresh server-generated trace identity.

When the trace block enables trust_request, Ferron parses the incoming traceparent and tracestate headers. It uses them as the parent for the internal ferron.request span. Ferron creates a local request span with the same trace ID and a new span ID. It reuses that span context for upstream propagation, access logs, and request-scoped OTLP logs. If the request has no trace context, Ferron can still generate a new one when generate is active.

With trust_request enabled, Ferron also parses the incoming baggage header and attaches it to the local request span context. Ferron then propagates baggage to upstream services and includes it in OTLP span exports. This lets application-defined key-value pairs flow through the entire request path.

Trace configuration⁠#

These directives go inside the http block.

W3C Baggage⁠#

Ferron 3 propagates the W3C Baggage header (baggage) alongside trace context headers. Baggage carries application-defined key-value pairs (for example, tenant ID, user segment, request flags) across service boundaries with no explicit configuration.

How baggage propagation works⁠#

1. By default, Ferron discards incoming baggage headers. With trust_request, Ferron reads the incoming baggage header from the request instead.
2. Ferron stores the baggage string (when available) in the request trace context.
3. When Ferron forwards the request to an upstream service, it includes the baggage header alongside traceparent and tracestate. It does this only when the trace context carries non-empty baggage values.
4. When Ferron exports via OTLP, it parses baggage and attaches it to the OpenTelemetry span context as OpenTelemetry baggage.

Baggage header format⁠#

The baggage header follows the W3C Baggage specification. Multiple items are comma-separated:

baggage: userId=alice,serverNode=5;props;otherKey=otherValue

Each item is a key=value pair with optional semicolon-separated properties. Values use URL encoding.

Baggage promotion to telemetry attributes⁠#

You can also promote specific baggage keys into OpenTelemetry attributes on telemetry signals (logs, metrics, traces). The baggage sub-directive inside each observability backend block configures this promotion:

{
    observability {
        provider otlp

        traces "https://collector:4317/v1/traces" {
            protocol "grpc"
        }

        baggage {
            key "tenant.id" {
                attribute "tenant.id"
                signals traces logs
                max_distinct 1000
            }
        }
    }
}

Examples⁠#

With default settings (trust_request false):

A client sends trace headers, but Ferron discards them and generates a new trace ID:

GET /api/data HTTP/1.1
Host: example.com
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
baggage: userId=alice,tenantId=acme

Ferron ignores the incoming traceparent and baggage. It generates a fresh trace ID and a new span ID. No baggage goes upstream unless a module adds it explicitly.

With trust_request true:

http {
    trace {
        generate
        trust_request
    }
}

With trust_request enabled, Ferron reads the incoming traceparent, tracestate, and baggage headers. It stores baggage in the request trace context and passes the trace headers to upstream services. With the OTLP provider, Ferron attaches the baggage to the span context, and the observability backend sees it.

Trace ID response header⁠#

Ferron can inject the trace ID of the current request into HTTP response headers. Clients can then correlate their requests with the server-side traces and logs.

trace_id_header⁠#

The trace_id_header directive configures whether and how Ferron injects the trace ID into response headers.

example.com {
    trace_id_header {
        header_name "X-Trace-Id"
    }
}

Configuration example (default behavior):

example.com {
    trace_id_header
}

Ferron injects the trace ID into the X-Ferron-Trace-Id response header for every response, including error responses.

Configuration example (custom header name):

example.com {
    trace_id_header {
        header_name "X-Request-Trace-Id"
    }
}

Ferron injects the trace ID into the custom X-Request-Trace-Id header.

Configuration example (conditional injection):

example.com {
    trace_id_header {
        reflect_request
    }
}

Ferron injects the trace ID only when the incoming request includes X-Ferron-Trace-Reflect: 1. This is useful for development or debugging when you want trace IDs on demand.

Configuration example (disabled):

example.com {
    trace_id_header false
}

Ferron explicitly disables trace ID injection.

Behavior⁠#

• Ferron takes the trace ID from the trace context of the current request. This is the W3C traceparent context if present, or the generated trace ID.
• Ferron injects the header into custom responses (for example, from reverse proxy, static file serving). It also injects it into built-in error responses, such as 404 and 500.
• With reflect_request enabled, Ferron injects the trace ID only when the request carries the X-Ferron-Trace-Reflect: 1 header.

Trace spans⁠#

Each HTTP request generates a root trace span and multiple nested spans for pipeline execution.

Root request span⁠#

• StartSpan("ferron.request") emits when the request enters the handler.
  • Attributes: http.request.method, url.full, url.scheme, server.address, server.port, client.address
  • For HTTPS connections: tls.protocol.version (for example, "TLSv1.3"), tls.cipher_suite (for example, "TLS_AES_256_GCM_SHA384")
• EndSpan("ferron.request", error) emits when the request completes.
  • Attributes: http.response.status_code, http.route (if applicable), error.type (if status >= 400)

Pipeline execution span⁠#

• ferron.pipeline.execute wraps the entire pipeline execution, including all forward and inverse stages. The span is a child of ferron.request.

File resolution span⁠#

• ferron.pipeline.file_resolve wraps static file path resolution with a root directive. The span is a child of ferron.pipeline.execute.
  • Attributes: ferron.file_resolve.request_path, ferron.file_resolve.root_path, ferron.file_resolve.outcome
  • On success: ferron.file_resolve.resolved_path
  • On error: ferron.file_resolve.last_candidate_path

Per-stage spans⁠#

Each pipeline stage and file-serving stage generates a forward span (ferron.stage.<stage_name>) and an inverse span (ferron.stage.<stage_name>.inverse). These are children of ferron.pipeline.execute, which enables flame graph analysis. Every per-stage span carries a ferron.stage.name attribute.

Error pipeline span⁠#

• ferron.pipeline.execute_error wraps error pipeline execution when Ferron generates error responses.
  • Attributes: http.response.status_code

Observability backends that support tracing (for example, OTLP) consume the trace events. All spans from one request share the same trace_id. Access logs carry the matching request span context when available.

Trace sampling⁠#

The trace_sampling directive (in the http block) controls which traces Ferron samples and exports. Sampling reduces the volume of trace data that Ferron sends to the collector while keeping representative coverage.

Configuration example:

example.com {
    http {
        trace {
            generate
        }

        # Sample 10% of root spans, respect parent for child spans
        trace_sampling "parentbased_traceidratio" {
            ratio 0.1
        }
    }
}

Ratio-based sampling⁠#

The traceidratio and parentbased_traceidratio modes accept a ratio sub-directive (a float between 0.0 and 1.0):

example.com {
    http {
        trace_sampling "parentbased_traceidratio" {
            ratio 0.05   # 5% of root spans
        }
    }
}

Use parentbased_traceidratio instead of bare traceidratio in distributed systems. It makes sampling decisions consistent across service boundaries. With traceidratio, a child span may get sampled even when the parent is not, which produces partial traces.

Attribute-based sampling⁠#

The attribute_based mode samples spans from the attributes that exist when Ferron creates the span. Configure rules inside a rules block:

example.com {
    http {
        trace_sampling "attribute_based" {
            # What to do with spans that don't match any rule
            default_action "sample"

            rules {
                # Always sample spans with http.request.method == "POST"
                rule "exact" "http.request.method" "POST"

                # Sample spans where url.path starts with "/api/"
                rule "prefix" "url.path" "/api/"

                # Sample spans that have an "error.type" attribute (any value)
                rule "exists" "error.type"
            }
        }
    }
}

Each rule takes 2 or 3 arguments:

Any matching rule samples the span. When no rule matches, the default_action directive controls the outcome:

See also⁠#

• W3C Trace Context: incoming trace context parsing, trace configuration, Baggage propagation, and trace ID response headers
• Trace sampling: trace sampling modes and configuration
• OTLP observability: export traces via OpenTelemetry Protocol
• Prometheus metrics: native Prometheus metrics export
• Access logging: access log configuration