Enable tracing & metrics
Turn on OTLP trace export and Prometheus metrics, and point them at the VictoriaMetrics stack.
Ductor emits OpenTelemetry traces and Prometheus metrics out of the box. This guide turns on trace export and confirms metrics are scraping. For the full production observability topology, see Observability.
Metrics — already on
Ductor always exposes Prometheus metrics on the metrics listener at /metrics:
curl -s http://localhost:9090/metrics | headThe address is server.metrics_addr (default :9090). To scrape it, point your
Prometheus-compatible agent at ductor:9090 with metrics_path: /metrics. In
the repo's compose stack, vmagent does this for you (job ductor, 15s
interval) and remote-writes to VictoriaMetrics.
On Kubernetes, enable the bundled ServiceMonitor so Prometheus Operator
discovers the target:
# Helm values
serviceMonitor:
enabled: true
interval: 30s
scrapeTimeout: 10sTraces — turn on OTLP export
Tracing is off by default (tracing.enabled=false). Enable it and point it
at an OTLP collector. Two transports are supported — HTTP (default) and gRPC.
The compose observability profile runs VictoriaTraces with an OTLP gRPC listener
on :4317. Match the compose defaults:
export DUCTOR_TRACING_ENABLED=true
export DUCTOR_TRACING_ENDPOINT=victoriatraces:4317
export DUCTOR_TRACING_GRPC=true
export DUCTOR_TRACING_INSECURE=true
export DUCTOR_TRACING_SAMPLE_RATE=1.0 # sample everything in devLeave tracing.grpc off (the default) to use the HTTP exporter. The default
endpoint targets VictoriaTraces' HTTP OTLP path:
export DUCTOR_TRACING_ENABLED=true
# default endpoint when grpc=false:
export DUCTOR_TRACING_ENDPOINT="http://victoriatraces:10428/insert/opentelemetry/v1/traces"
export DUCTOR_TRACING_SAMPLE_RATE=0.1 # sample 10% in productionThe equivalent serve flags exist too: --tracing-enabled,
--tracing-endpoint, --tracing-grpc, --tracing-insecure,
--tracing-sample-rate, --tracing-backend.
Tracing config reference
Prop
Type
insecure is dev-only
tracing.insecure=true disables TLS on the trace exporter — fine inside a
trusted compose network, but set it to false and use a TLS endpoint when
exporting across a network boundary.
Verify traces are flowing
With the observability profile up, open Grafana at http://localhost:3000
(admin / admin) and query the VictoriaTraces datasource, or hit
VictoriaTraces directly on :10428. Drive a request through Ductor (create a
pool, run a workflow) and you should see spans appear.
What you get
- Metrics — HTTP/gRPC request metrics, routing decision metrics, and DAG-runtime metrics (coordinator ticks, wakeups, attempts).
- Traces — spans across the transport → application → infrastructure path, including routing pipeline stages and coordinator ticks.
- Logs — structured
log/slogoutput (JSON by default) that the Vector pipeline ships to VictoriaLogs in the compose stack.