> ## Documentation Index
> Fetch the complete documentation index at: https://hilt.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Data collection & handling

> What Hilt collectors capture, what they never capture, how events are filtered, where data lives, and retention.

Hilt collectors observe system activity at the metadata level. They record what ran, what connected, and what was read or written, and they never capture packet payloads or process memory. File contents are not captured unless you explicitly enable content inspection on a host. This page describes the data Hilt handles, where it lives, and how long it is kept.

***

## What is collected

The collector captures execution and system metadata for each event:

* Timestamps
* Process and user identifiers (PID, UID)
* Command name, binary path, and arguments
* File paths (for file events)
* Network details such as the connection 5-tuple and byte counts (for network events)
* Container and cgroup identifiers
* The configured node label

This metadata lets activity be correlated across hosts and workloads. For the full schema, see [Event data model](https://hilt.ai/docs/linux-collector/event-data-model.md).

***

## What is never collected

By default the collector is not a content inspection tool. It never captures:

| Category | Detail |
| :- | :- |
| **File contents** | Only the path and operation type, never the bytes in the file — unless [content inspection](#content-inspection) is enabled on that host |
| **Packet payloads** | Only the connection 5-tuple and byte counts, never packet data |
| **Memory and process memory** | No reads of process memory or memory regions |
| **Environment variables** | Process environment is not captured |
| **Credentials** | No passwords, keys, or token values |
| **Anything beyond standard tools** | Nothing past what `ps`, `lsof`, or `ss` would expose |

The macOS collector follows the same design: it does not read file contents, log keystrokes, capture screen content, or capture packet payloads. Telemetry is overwhelmingly NOTIFY; the only AUTH path is agent self-protection (deny unauthorized kill/delete/rename of the collector). See [macOS Collector](https://hilt.ai/docs/macos-collector/layout.md).

**One documented exception on macOS:** the clipboard sensor is on by default and stores copied content. Raw content is admin-only to read, truncated, and expires in 30 days rather than 90. It can be reduced to metadata-only or turned off entirely by managed preference. See [Clipboard and data movement](https://hilt.ai/docs/macos-collector/layout.md#clipboard-and-data-movement) for exactly what is captured and how to scope it.

***

## Content inspection

Content inspection is **off by default**. Hosts that never set `DAG_CONTENT_INSPECT` keep the default guarantee: the collector does not read file bytes.

When you set `DAG_CONTENT_INSPECT=1` on a Linux host whose profile is not `latency_critical`:

* On a `DATA_FLOW` event to a **public** destination (not RFC1918, loopback, link-local, or IPv6 ULA), the agent copies the source file.
* Copy is capped at **100 MiB**. Larger files are skipped (size is recorded; the file is not hashed or uploaded).
* Bytes go to **in-tenant object storage** via a presigned PUT. They never ride Kafka, enrichment, or the event shipper.
* Stored objects expire after **30 days**, matching the `dag_content_objects` table TTL.
* Global admins can view (text, images, PDF) or download the file from File Detail. Other roles see that content exists, not the bytes.

`DAG_HOST_PROFILE=latency_critical` never starts the capture worker, even if `DAG_CONTENT_INSPECT` is set.

This slice stores the file as-is. It does not classify content or produce summaries.

***

## Filtering and minimization

The collector reduces raw kernel telemetry to a security-relevant stream before anything is stored. On a busy host, raw syscall volume can exceed 100k events per second; a two-tier eBPF and userspace filter typically drops well over 90% of that as OS housekeeping noise.

Two properties matter for data handling:

* **No content inspection in filtering.** Filters operate on metadata only (path prefix, process name, CIDR), never on file or packet contents. (Separate, opt-in content inspection copies a file after a public `DATA_FLOW`; it is not used to filter events.)
* **Security-relevant events are preserved.** Process executions, file writes, container lifecycle events, and reads of sensitive paths (credential files, tokens, secrets) always pass the filter.

For the filtering architecture and configuration, see [Agent filtering](https://hilt.ai/docs/linux-collector/agent-filtering.md).

***

## Where data lives

Each data class stays inside your cloud account.

| Data class | Location |
| :- | :- |
| Raw events in flight | Collector host buffer, then the private ingest endpoint |
| Buffered events | Streaming buffer (Kafka or Event Hubs) in your account |
| Normalized telemetry and findings | Analytics database (ClickHouse) in your account |
| Captured file content (opt-in) | Object storage in your account (30-day retention; never Kafka) |
| Dashboard accounts, teams, and audit records | Dashboard database (PostgreSQL) in your account |
| Reports and exports | Object storage (S3, Cloud Storage, Blob, or OSS) in your account |

No copy of this data is sent to or stored by Hilt. See [Architecture and data flow](https://hilt.ai/docs/trust-security/architecture.md) for the component map.

***

## Collector privilege

The collector requires elevated privileges to read kernel telemetry. The exact requirement depends on the deployment path (root for systemd, a privileged DaemonSet on Kubernetes). These privileges grant observation only; the collector does not modify host state, terminate processes, or block arbitrary activity. The privilege model is documented in [Networking and security](https://hilt.ai/docs/linux-collector/networking-security.md).

***

## Retention

Telemetry retention is set per deployment. The analytics database and object storage are sized to your agreed retention window, and you control retention because the storage lives in your account. Default retention windows are shown per dataset on the [Storage](https://hilt.ai/docs/administration/storage.md) and [ClickHouse tables](https://hilt.ai/docs/administration/clickhouse-tables.md) pages; confirm the configured retention period for your deployment with your Hilt representative.

When you delete a dashboard user, their account and sessions are removed; audit history keeps entries under the username so the record of past actions is preserved. See [User Management](https://hilt.ai/docs/administration/user-management.md).

***

## Anonymized data export

The only path by which any data leaves your account is the anonymized export pipeline, a core component of every deployment. It exports aggregate numeric features, derived statistics, and abstracted activity tokens, never file paths, IP addresses, hostnames, usernames, cloud resource IDs, or detection rule logic. Identifying values are stripped or remapped before export, exports are rejected below a minimum host threshold, and — when approval is enabled (the default) — you review a manifest and sample and approve each transfer.

For the full data catalog, anonymization method, and customer controls, see [Anonymized data export](https://hilt.ai/docs/on-prem/data-export-overview.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.