Export Schema - Hilt Documentation

The exported Parquet file contains exactly the columns listed below. Anything not listed here is excluded at the query level.

Temporal and grouping

Column	Type	Notes
`epoch_ts`	`String`	5-minute bucket timestamp; offset by a random per-export shift (±0–48h) to prevent calendar correlation
`host_id_anon`	`String`	Deterministic remap: `host-0001`, `host-0002`, etc. Consistent within one export
`uid_anon`	`UInt32`	Sequential remap: 0, 1, 2, … per host
`phase`	`String`	Scoring phase label (e.g. `cold_start`, `baseline`, `scoring`)
`is_active`	`UInt8`	Whether the epoch had meaningful activity
`is_anomaly`	`UInt8`	Binary anomaly flag
`topk_sum`	`Float64`	Sum of top-K anomaly channel scores
`volume_topk`	`Float64`	Volume-weighted top-K score

Feature vectors

Column	Type	Notes
`numeric_features`	`List(Float64)`	24-element vector of aggregate activity counts per epoch
`derived_features`	`List(Float64)`	32-element vector of statistical transforms (ratios, entropies, wavelets)

Feature vectors are packed as Arrow List columns. Individual feature semantics are documented in internal engineering references.

Activity token sequences

Column	Type	Notes
`token_ids`	`List(UInt16)`	Closed vocabulary IDs: no raw strings, filenames, or IPs
`token_sub_epochs`	`List(UInt8)`	Sub-epoch index per token
`token_counts`	`List(UInt16)`	Occurrence count per token
`token_importances`	`List(Float32)`	Per-token anomaly importance weight
`token_depths`	`List(UInt8)`	Process tree depth
`token_roles`	`List(UInt8)`	Token role enum
`total_events`	`UInt32`	Total raw events in the epoch (before tokenization)
`tokens_before_topk`	`UInt16`	Token count before top-K pruning

Labels (when available)

Column	Type	Notes
`ml_label`	`String`	Attack phase label or empty
`ml_source`	`String`	Label provenance (`hitl`, `synthetic`, `replay`)

Explicitly excluded

Every column not in the allowlist above is excluded. The export queries only aggregated scoring and token tables. It has no access to raw event data or application databases. Excluded categories include:

Raw event evidence (filenames, argv, addresses)
Free-text scorer explanations
Human identity strings
Original host IDs (replaced with anonymized versions)
Boot IDs and lifecycle identifiers
Detection rule provenance (rule names, match conditions, severity labels)

Last modified on October 8, 2026