Grafana reference dashboard
Ferron ships a reference Grafana dashboard (dashboards/ferron-3-reference.json in the repository). It adapts to different operational roles from a single layout. Maintaining separate dashboards for a CDN edge, an API gateway, and a shared-hosting box is a lot of work. The reference template uses template variables and collapsible rows. One structure serves all three.
The dashboard targets a Prometheus-compatible datasource (such as Prometheus metrics exported by Ferron or scraped through Mimir/Cortex). It expects OpenTelemetry-style metric names from Ferron converted to Prometheus snake_case. Counters gain a _total suffix.
Design: symptoms on top, causes on the bottom#
The dashboard splits along the classic RED versus USE boundary. This layout lets a user move from symptom to root cause during an incident:
| Dashboard section | Methodology | Reports | Best used for |
|---|---|---|---|
| Top half (always expanded) | RED (Rate, Errors, Duration) | User-facing symptoms: p99 latency spikes, exploding 5xx rates | High-level triage and SLA verification |
| Bottom half (collapsible rows) | USE (Utilization, Saturation, Errors) | Internal proxy causes: pool exhaustion, drained retry budget, circuit trips | Root-cause analysis after an alert fires |
Template variables#
Top-level variables let a single dashboard adapt to any deployment without editing panels:
| Variable | Selects | Source label |
|---|---|---|
Job | Scrape job (default ferron) | job |
Instance | Scrape target | instance |
Upstream backend | Reverse-proxy backend | ferron_proxy_backend_url |
Host (tenant) | SNI / tenant (TLS metrics only) | ferron.host |
Cache zone | Named cache zone | ferron.cache.zone |
Rate-limit zone | Named rate-limit zone | ferron.ratelimit.zone |
Ferron does not expose a per-route (request-path) metric label. The sketch $route variable from generic dashboard designs has no matching signal. Filter by upstream backend or by cache/rate-limit zone instead. If you need per-route granularity, promote a bounded route attribute with baggage promotion. Add it as a variable, but cap max_distinct to avoid label explosion.
Rows#
Row 1 — Core L7 Matrix (RED, always expanded)#
The row every deployment reads first:
- Traffic rate by status class:
sum by (http_response_status_code) (rate(ferron_http_server_request_count_total{...})) - Active requests:
sum(http_server_active_requests) - 5xx / 4xx error ratio: error-class request rate divided by total request rate
- Request latency (p50/p95/p99):
histogram_quantileoverhttp_server_request_duration_seconds_bucket. With native exponential histograms enabled, tail latencies (p99/p99.9) stay sharp without manual bucket tuning.
Row 2 — Resiliency primitives (collapsible)#
Maps the token-bucket retry budget and rate limiting:
- Retry budget exhausted /s:
rate(ferron_proxy_retry_budget_exhausted_total) - Retry budget tokens available:
ferron_proxy_retry_budget_tokens_available - Retry final failures:
ferron_proxy_retry_final == 0 - Rate-limit rejected /s (by zone):
rate(ferron_ratelimit_rejected_total) - Rate-limit allowed vs throttled /s:
rate(ferron_ratelimit_allowed_total)/rate(ferron_ratelimit_throttled_total)
An API gateway engineer keeps this row open to watch JWT-adjacent throttling and endpoint rejection spikes. A CDN operator can leave it collapsed.
Row 3 — Infrastructure saturation (USE, collapsible)#
Connection-pool and host-pressure panels, crucial for multi-tenant edges and service meshes:
- Pool saturation:
ferron_proxy_pool_outstandingagainstferron_proxy_pool_local_limitandferron_proxy_pool_global_limit - Pool wait time p95:
histogram_quantileoverferron_proxy_pool_wait_time_seconds_bucket - Pool waits /s:
rate(ferron_proxy_pool_waits_total)(exhaustion events) - Pool hit ratio: hits divided by hits + misses
- Top backends by outstanding connections: a table answering “who is the noisy neighbor” directly
- Circuit breaker state & unhealthy backends:
ferron_proxy_circuit_state,rate(ferron_proxy_backends_unhealthy_total),ferron_proxy_lb_active_connections - Host saturation:
process_cpu_utilization_ratio,process_unix_file_descriptor_count,process_memory_usage_bytes
Row 4 — Edge cache (collapsible)#
- Cache hit ratio:
rate(ferron_cache_requests_total{...,result="hit"})/ total - Cache entries:
ferron_cache_entriesby zone - Cache evictions /s (by reason):
rate(ferron_cache_evictions_total)split byferron.cache.reason - Egress bandwidth:
sum(rate(ferron_static_bytes_sent_sum))(static-file and PHP-accelerator egress. See the gap below) - DNS cache TTL remaining:
ferron_proxy_dns_cache_ttl_remaining_seconds(min/avg/max via theaggregationlabel) and DNS hit ratio
A CDN or PHP-accelerator operator keeps this row pinned. An API gateway user can ignore it.
Deploying the dashboard#
Import the JSON#
- In Grafana, Dashboards → New → Import, then upload
dashboards/ferron-3-reference.json. - Pick your Prometheus/Mimir datasource.
Or provision it#
Mount the JSON under a Grafana file provisioner dashboards path, or POST it to the HTTP API:
curl -u admin:admin -X POST \
http://localhost:3000/api/dashboards/db \
-H "Content-Type: application/json" \
-d @dashboards/ferron-3-reference.jsonRequired Ferron configuration#
The dashboard reads metrics that Ferron emits only when you configure observability:
example.com {
observability {
provider prometheus
endpoint_listen "127.0.0.1:8889"
}
}Enable the features whose rows you care about:
- Row 2 (retry budget, rate limiting): configure
rate_limitand a proxy withretrysettings. - Row 3 (pools, circuit): configure a
proxywith upstreams. - Row 4 (cache, DNS cache): configure
cacheand strict/SRVdns_servers.
Known gaps#
- Per-route filtering is unavailable without baggage promotion.
- Egress bandwidth exists only as
ferron.static.bytes_sent(static files, PHP accelerator). Reverse-proxy upstream egress has no bytes metric today. You must measure proxy-dominated CDN egress at the load balancer or add it to Ferron later. - TLS /
ferron.hostmetrics (ferron_tls_*) appear only when you configure HTTPS. Those panels render empty on HTTP-only instances. - Rows whose Ferron modules are not active show empty panels by design. The dashboard degrades gracefully rather than erroring.
See also#
- Configuration: metrics: full metric catalog and alerting patterns
- Prometheus metrics: exporter setup and native histograms
- OTLP observability: OpenTelemetry export alternative