> ## Documentation Index
> Fetch the complete documentation index at: https://hilt.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Export Schema

> Schema of the anonymized Parquet export file.

The exported Parquet file contains exactly the columns listed below. Anything not listed here is excluded at the query level.

## Temporal and grouping

| Column | Type | Notes |
| :- | :- | :- |
| `epoch_ts` | `String` | 5-minute bucket timestamp; offset by a random per-export shift (±0–48h) to prevent calendar correlation |
| `host_id_anon` | `String` | Deterministic remap: `host-0001`, `host-0002`, etc. Consistent within one export |
| `uid_anon` | `UInt32` | Sequential remap: 0, 1, 2, … per host |
| `phase` | `String` | Scoring phase label (e.g. `cold_start`, `baseline`, `scoring`) |
| `is_active` | `UInt8` | Whether the epoch had meaningful activity |
| `is_anomaly` | `UInt8` | Binary anomaly flag |
| `topk_sum` | `Float64` | Sum of top-K anomaly channel scores |
| `volume_topk` | `Float64` | Volume-weighted top-K score |

## Feature vectors

| Column | Type | Notes |
| :- | :- | :- |
| `numeric_features` | `List(Float64)` | 24-element vector of aggregate activity counts per epoch |
| `derived_features` | `List(Float64)` | 32-element vector of statistical transforms (ratios, entropies, wavelets) |

Feature vectors are packed as Arrow List columns. Individual feature semantics are documented in internal engineering references.

## Activity token sequences

| Column | Type | Notes |
| :- | :- | :- |
| `token_ids` | `List(UInt16)` | Closed vocabulary IDs: no raw strings, filenames, or IPs |
| `token_sub_epochs` | `List(UInt8)` | Sub-epoch index per token |
| `token_counts` | `List(UInt16)` | Occurrence count per token |
| `token_importances` | `List(Float32)` | Per-token anomaly importance weight |
| `token_depths` | `List(UInt8)` | Process tree depth |
| `token_roles` | `List(UInt8)` | Token role enum |
| `total_events` | `UInt32` | Total raw events in the epoch (before tokenization) |
| `tokens_before_topk` | `UInt16` | Token count before top-K pruning |

## Labels (when available)

| Column | Type | Notes |
| :- | :- | :- |
| `ml_label` | `String` | Attack phase label or empty |
| `ml_source` | `String` | Label provenance (`hitl`, `synthetic`, `replay`) |

## Explicitly excluded

Every column not in the allowlist above is excluded. The export queries only aggregated scoring and token tables. It has no access to raw event data or application databases.

Excluded categories include:

* Raw event evidence (filenames, argv, addresses)
* Free-text scorer explanations
* Human identity strings
* Original host IDs (replaced with anonymized versions)
* Boot IDs and lifecycle identifiers
* Detection rule provenance (rule names, match conditions, severity labels)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.