Data collection & handling - Hilt Documentation

Hilt collectors observe system activity at the metadata level. They record what ran, what connected, and what was read or written, and they never capture packet payloads or process memory. File contents are not captured unless you explicitly enable content inspection on a host. This page describes the data Hilt handles, where it lives, and how long it is kept.

What is collected

The collector captures execution and system metadata for each event:

Timestamps
Process and user identifiers (PID, UID)
Command name, binary path, and arguments
File paths (for file events)
Network details such as the connection 5-tuple and byte counts (for network events)
Container and cgroup identifiers
The configured node label

This metadata lets activity be correlated across hosts and workloads. For the full schema, see Event data model.

What is never collected

By default the collector is not a content inspection tool. It never captures:

Category	Detail
File contents	Only the path and operation type, never the bytes in the file — unless content inspection is enabled on that host
Packet payloads	Only the connection 5-tuple and byte counts, never packet data
Memory and process memory	No reads of process memory or memory regions
Environment variables	Process environment is not captured
Credentials	No passwords, keys, or token values
Anything beyond standard tools	Nothing past what `ps`, `lsof`, or `ss` would expose

The macOS collector follows the same design: it does not read file contents, log keystrokes, capture screen content, or capture packet payloads. Telemetry is overwhelmingly NOTIFY; the only AUTH path is agent self-protection (deny unauthorized kill/delete/rename of the collector). See macOS Collector. One documented exception on macOS: the clipboard sensor is on by default and stores copied content. Raw content is admin-only to read, truncated, and expires in 30 days rather than 90. It can be reduced to metadata-only or turned off entirely by managed preference. See Clipboard and data movement for exactly what is captured and how to scope it.

Content inspection

Content inspection is off by default. Hosts that never set DAG_CONTENT_INSPECT keep the default guarantee: the collector does not read file bytes. When you set DAG_CONTENT_INSPECT=1 on a Linux host whose profile is not latency_critical:

On a DATA_FLOW event to a public destination (not RFC1918, loopback, link-local, or IPv6 ULA), the agent copies the source file.
Copy is capped at 100 MiB. Larger files are skipped (size is recorded; the file is not hashed or uploaded).
Bytes go to in-tenant object storage via a presigned PUT. They never ride Kafka, enrichment, or the event shipper.
Stored objects expire after 30 days, matching the dag_content_objects table TTL.
Global admins can view (text, images, PDF) or download the file from File Detail. Other roles see that content exists, not the bytes.

DAG_HOST_PROFILE=latency_critical never starts the capture worker, even if DAG_CONTENT_INSPECT is set. This slice stores the file as-is. It does not classify content or produce summaries.

Filtering and minimization

The collector reduces raw kernel telemetry to a security-relevant stream before anything is stored. On a busy host, raw syscall volume can exceed 100k events per second; a two-tier eBPF and userspace filter typically drops well over 90% of that as OS housekeeping noise. Two properties matter for data handling:

No content inspection in filtering. Filters operate on metadata only (path prefix, process name, CIDR), never on file or packet contents. (Separate, opt-in content inspection copies a file after a public DATA_FLOW; it is not used to filter events.)
Security-relevant events are preserved. Process executions, file writes, container lifecycle events, and reads of sensitive paths (credential files, tokens, secrets) always pass the filter.

For the filtering architecture and configuration, see Agent filtering.

Where data lives

Each data class stays inside your cloud account.

Data class	Location
Raw events in flight	Collector host buffer, then the private ingest endpoint
Buffered events	Streaming buffer (Kafka or Event Hubs) in your account
Normalized telemetry and findings	Analytics database (ClickHouse) in your account
Captured file content (opt-in)	Object storage in your account (30-day retention; never Kafka)
Dashboard accounts, teams, and audit records	Dashboard database (PostgreSQL) in your account
Reports and exports	Object storage (S3, Cloud Storage, Blob, or OSS) in your account

No copy of this data is sent to or stored by Hilt. See Architecture and data flow for the component map.

Collector privilege

The collector requires elevated privileges to read kernel telemetry. The exact requirement depends on the deployment path (root for systemd, a privileged DaemonSet on Kubernetes). These privileges grant observation only; the collector does not modify host state, terminate processes, or block arbitrary activity. The privilege model is documented in Networking and security.

Retention

Telemetry retention is set per deployment. The analytics database and object storage are sized to your agreed retention window, and you control retention because the storage lives in your account. Default retention windows are shown per dataset on the Storage and ClickHouse tables pages; confirm the configured retention period for your deployment with your Hilt representative. When you delete a dashboard user, their account and sessions are removed; audit history keeps entries under the username so the record of past actions is preserved. See User Management.

Anonymized data export

The only path by which any data leaves your account is the anonymized export pipeline, a core component of every deployment. It exports aggregate numeric features, derived statistics, and abstracted activity tokens, never file paths, IP addresses, hostnames, usernames, cloud resource IDs, or detection rule logic. Identifying values are stripped or remapped before export, exports are rejected below a minimum host threshold, and — when approval is enabled (the default) — you review a manifest and sample and approve each transfer. For the full data catalog, anonymization method, and customer controls, see Anonymized data export.