Ali Cloud - Hilt Documentation

Hilt runs as a single-tenant platform inside your own Ali Cloud account. A managed ACK cluster hosts ingest, enrichment, classification, and the dashboard; collectors on your hosts stream telemetry over HTTPS to {name}-ingest.hilt.ai and {name}-edr.hilt.ai, which feed Message Queue for Apache Kafka; ClickHouse Cloud is the analytics database; ApsaraDB RDS (PostgreSQL) holds dashboard auth. Product images are pulled from Hilt’s Container Registry (ACR Enterprise Edition) with a short-lived cross-account credential. All customer telemetry stays in your account.

Services

Service	Role
ACK	Managed Kubernetes for all platform services (ingest, enrichment, classification, dashboard)
Message Queue for Apache Kafka	Event streaming between ingest and enrichment (professional edition, cloud-storage topics)
ClickHouse Cloud	Analytics database, reached over the public internet through your NAT Gateway
ApsaraDB RDS (PostgreSQL)	Dashboard auth/HITL database (private address only)
OSS	Terraform remote state, app configuration, and data-export staging
NAT Gateway + EIP	Outbound path for image pulls and ClickHouse Cloud
nginx ingress behind a load balancer	HTTPS entry for `{name}.hilt.ai` (dashboard), `{name}-ingest.hilt.ai`, and `{name}-edr.hilt.ai`
KMS	Generated database and admin secrets
Hilt ACR Enterprise Edition	Product images, pulled cross-account (`hilt-release-registry.ap-southeast-1.cr.aliyuncs.com/hilt`)
NAS (File Storage NAS)	Shared RWX volume for the on-prem ML model artifact store (optional)

Network topology

You bring an existing VPC with at least two private VSwitches in different zones
The VPC needs a NAT Gateway with SNAT for those VSwitches. If it has none, set create_nat: true and the stack creates and owns one
Collectors reach {name}-ingest.hilt.ai and {name}-edr.hilt.ai over HTTPS; the dashboard is served at https://{name}.hilt.ai
dashboard_ingress_cidr_blocks restricts who can reach the dashboard; ingest_internal, dashboard_internal, and edr_collector_internal keep each endpoint on a private address instead
RAM roles with RRSA (RAM Roles for Service Accounts) provide pod-level identity
Outbound internet is the NAT Gateway, for image pulls and ClickHouse Cloud
Optional DashScope PrivateLink for private LLM inference

Features

Feature	Description
Enrichment pipeline	Kafka consumer writes enriched telemetry to ClickHouse with GNN edge support
ML classification	Anomaly detection scoring with automated drift monitoring (CronJob every 6 hours)
On-prem ML training	Optional autonomous TRACE model retraining with NAS-backed artifact store
Data export	Scheduled anonymized data export to OSS with admin approval workflow
Observability	In-cluster Prometheus and Grafana for platform metrics
Dashboard	Frontend, backend, and query service behind the ingress with path-based routing
Auto-scaling	HPA on enrichment (CPU/memory targets) with configurable min/max replicas

Before you start

Item	Notes
Account readiness	A new Ali Cloud account must complete the console account profile and activate the services below once. Until then calls fail with `BASIC_INFO_UNCOMPLETED` or `UserDisable`, usually first on OSS.
Services to activate	OSS, ACK, RDS, Message Queue for Apache Kafka, NLB, ALB, NAT Gateway / EIP, KMS, RAM
RAM identity	A RAM user or assumed role, not the root account login. It needs broad rights on the services above for the first apply.
Network	VPC with at least two private VSwitches in different zones
ClickHouse Cloud	A service reachable from the internet; you supply its hostname and password
Enrollment	Hilt enrolls your account ID and deployment name on dns-broker (`*.hilt.ai`) and grants your account pull access to the Hilt registry

Image delivery

Hilt publishes product images to ACR Enterprise Edition in a Hilt release account:

hilt-release-registry.ap-southeast-1.cr.aliyuncs.com/hilt/<service>:<tag>

When Hilt enrolls your account, it allows your account to assume the hilt-acr-customer-pull role. The ACK ACR credential helper in your cluster assumes that role through RRSA and attaches a pull secret to the workload service accounts; no static registry password is stored. A customer-owned ACR mirror (Personal Edition) is optional. The installer image the pipeline runs from is pulled from Hilt’s shared registry; see the starter README for the exact reference.

Provisioning

Hilt provisions the data plane with Terraform via hilt-deploy (cloud: alibaba), run from a GitLab pipeline in your own environment:

You provide the account, VPC, two private VSwitches, and a ClickHouse Cloud hostname; Hilt enrolls the account and deployment name
You import the Alibaba starter into a new GitLab project, add the CI variables, and fill in values.json
The pipeline runs preflight automatically (RAM identity, registry access, enrollment, network contract), then manual plan and apply
apply creates ACK, RDS, Kafka, the load balancers, and the workloads, publishes the public DNS records, applies the ClickHouse and Postgres schemas in-cluster, and runs a health check
Collectors are deployed to your hosts using the Linux and macOS collector guides

What to expect on the first apply

A from-scratch apply takes roughly 40 to 60 minutes
It is safe to re-run apply. A fresh cluster can fail one early step once (node pool and addon timing, or the ingress admission webhook not yet ready) and succeed on the retry
Pods that start before the registry pull secret exists show ImagePullBackOff briefly and recover on their own
Public TLS (Let’s Encrypt through dns-broker) is best effort. If a certificate cannot be issued, the apply still completes, the ingress serves a self-signed certificate, and the health check reports the HTTPS checks as failed. Re-run apply after the cause is fixed. If you see this, contact Hilt