Skip to main content
Every Hilt backend service exposes a /metrics endpoint in the standard Prometheus text exposition format (text/plain; version=0.0.4). You can scrape these into your own observability stack — Prometheus, Grafana Agent / Alloy, VictoriaMetrics, the OpenTelemetry Collector, Datadog, or any tool that speaks OpenMetrics. The exporter covers process and runtime telemetry (CPU, memory, GC, file descriptors). Application-level metrics such as request latency or queue depth are not currently exported.

Services with /metrics

The endpoint shares each service’s existing HTTP port — no additional container port or security-group rule is required. On the backend, the endpoint is mounted ahead of the authentication middleware, so scrapes do not require a session or cookie.

Labels

Every metric series carries three labels: These are stamped by the exporter at emission time. If your scrape config also adds an environment label, set honor_labels: true to preserve the exporter’s value.

Configuration

All services share the same environment-variable contract:
The endpoint is enabled by default. Set METRICS_ENABLED=false on any service where you do not want to expose it.

Metric families

CPU

  • process_cpu_seconds_total — cumulative CPU time
  • process_cpu_utilization_ratio — instantaneous CPU usage (0–1)
  • process_cpu_throttled_seconds_total — cgroup throttle time
  • process_cpu_throttled_periods_total — cgroup throttle period count
  • process_cpu_core_usage_ratio{core} — per-core usage (Linux only)

Memory

  • process_resident_memory_bytes — RSS
  • process_virtual_memory_bytes — virtual memory
  • process_heap_bytes / process_heap_sys_bytes — Go heap (Go services only)
  • system_memory_total_bytes / system_memory_available_bytes — host memory
  • process_memory_usage_ratio — RSS vs cgroup limit

Runtime

  • process_uptime_seconds — seconds since start
  • process_open_fds / process_max_fds — file descriptors
  • go_goroutines, go_threads, go_gc_duration_seconds — Go runtime (Go services)
  • python_gc_collections_total, python_info — Python runtime (Detection ML)
cgroup and /proc-derived metrics (throttling, per-core, system memory) are emitted only on Linux. On other platforms they are silently omitted.

Scrape configuration

Prometheus / VictoriaMetrics

A minimal static-target config:
Replace the static targets with your service discovery mechanism (Kubernetes pod annotations, ECS Cloud Map, Consul, etc.).

Kubernetes pod annotations

For in-cluster Prometheus with pod-based service discovery, annotate your pods:

Grafana Agent / Alloy

Use the prometheus.scrape component pointing at the same targets shown above.

Datadog Agent

Use the OpenMetrics check:

OpenTelemetry Collector

Use the prometheus receiver with the same scrape_configs block.

Grafana dashboard

A starter Grafana dashboard is available with panels for:
  • CPU utilization ratio and CPU seconds rate
  • Resident memory (RSS) and memory usage ratio
  • CPU throttling rate
  • Goroutines and GC pause (Go services)
  • Open file descriptors
  • Uptime
The dashboard includes a $service template variable for filtering by service. Import the JSON into your Grafana instance and select your Prometheus data source.

Relationship to System Status

The System Status page in the Hilt dashboard provides a curated health view of backend components using internal health checks. Prometheus metrics complement this with lower-level, time-series resource telemetry suitable for alerting, capacity planning, and long-term trend analysis.
Last modified on October 8, 2026