Deployment Guide

Insights is a single Go binary with independently toggleable subsystems. How you configure them determines the deployment topology. See Architecture for what each subsystem does.

The flags shown below are a subset chosen for the deployment scenarios on this page. For the full reference (every flag, env var, and YAML key) see Configuration.

Deployment Topologies

Demo (Simulator)

Zero-config mode for evaluation. Starts an embedded NATS simulator with traffic workloads. All data is ephemeral.

./insights --simulator.enabled

Everything runs in-process: simulator, embedded sink, scraper, indexer, web UI. No external NATS required.

Choose a profile to control deployment size:

./insights --simulator.enabled --simulator.profile super-medium

Available profiles: core-{small,medium,large}, js-{small,medium,large}, super-{small,medium,large}, leaf-{small,medium,large}, super-leaf-{small,medium,large}.

All-in-One (Ephemeral)

Monitor a real NATS deployment. Requires system account credentials for $SYS access.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds

Or with a NATS context:

./insights --sys.context my-system

Data is stored in-memory (DuckDB) and a temp directory (JetStream). Both are lost on restart.

All-in-One (Persistent)

Same as above with durable storage that survives restarts.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds \
  --data-dir /data/insights

The default web.session-seed is "insights", which preserves sessions across restarts out of the box. For production, set it to a unique secret:

--web.session-seed "your-secret-seed"

For all --sys.* authentication options (creds, context, basic auth, NKey, JWT, TLS, SOCKS proxy), see Configuration > sys.*.

External NATS Sink

Disable the embedded NATS server and use an external NATS cluster with JetStream for the scrape stream. This decouples the scraper from the indexer and enables distributed topologies.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds \
  --sink.embed=false \
  --server nats://sink-cluster:4222 \
  --creds /path/to/sink.creds \
  --data-dir /data/insights

The top-level client connection flags (--server, --creds, --context, --tls-ca-cert, …) are unprefixed and nats-CLI-style. They configure the connection used by the API, indexer, and scraper sink (unless overridden). The pre-rename --nats.* names remain as hidden aliases, and the YAML nats: section still maps here. When sink.embed is false, the JetStream stream must already exist or be created on the external server.

You can override the NATS connection independently for the indexer or scraper. The fallback order is scraper.nats → indexer.nats → nats (top-level).

nats:
  server: nats://api-cluster:4222

indexer:
  nats:
    server: nats://data-cluster:4222

scraper:
  nats:
    server: nats://data-cluster:4222

Headless (No Web UI)

Run without the web UI. The NATS micro API remains available for direct nats req queries.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds \
  --web.enabled=false

Query via the NATS CLI:

nats req '$INS.db.query' '{"sql": "SELECT * FROM hx.servers LIMIT 5"}'

For programmatic and AI-agent access, see the AI Agents guide.

Indexer-Only

Consume from an existing scrape stream without running the scraper. Useful when a separate instance or process handles scraping.

./insights \
  --scraper.enabled=false \
  --sink.embed=false \
  --server nats://data-cluster:4222 \
  --data-dir /data/insights

Split Web Tier

Run the web UI as a pure NATS client of a remote indexer/scraper node. The insights web subcommand holds no local state — no database, no license, no scraper or indexer — and reaches the backend entirely over NATS, discovering its capabilities, retention, and license from the backend via $INS.ops.info.

./insights web \
  --server nats://data-cluster:4222 \
  --creds /path/to/api.creds

Point it at the indexer/scraper node with --server (or --context); every query and control flows over NATS. The same connection backs the other client surfaces: insights mcp (a read-only Model Context Protocol server) and insights http (a JSON/CSV gateway over the query API).

Operational Tuning

Scrape interval and timeout

--scraper.interval 1m     # Time between scrape cycles (default: 1m)
--scraper.timeout 30s     # Per-request timeout (default: 30s)

Scrape filter

By default every discovered server is scraped. Restrict the scrape set by exact match on server name, cluster, or tag; excluded servers receive no monitoring requests. Deny takes precedence over allow.

--scraper.filter.allow.clusters us-east,us-west   # Scrape only these clusters
--scraper.filter.deny.tags edge                   # Never scrape servers with this tag

Retention

Limit how much historical data is kept in DuckDB:

--db.retention.duration 24h  # Keep last 24 hours (default: 768h)
--db.retention.interval 10m  # Sweep interval (default: 10m)

Web binding

--web.hostname 0.0.0.0    # Bind to all interfaces (default: 127.0.0.1)
--web.port 8080            # Bind port (default: 8080)

To serve HTTPS, enable TLS and provide an explicit certificate and key. Enabling TLS without both files is an error.

--web.tls \
--web.tls-cert /path/to/cert.pem \
--web.tls-key /path/to/key.pem

Notifications

Deliver check findings to external systems. Notification endpoints are configured in the YAML file only (there are no equivalent flags): webhook endpoints receive the full payload, and Alertmanager endpoints receive a v2-compatible alert array. Each endpoint can subscribe to specific check codes.

notifications:
  webhook:
    - name: ops-relay
      url: https://hooks.example.com/insights
      subscriptions:
        - SERVER_001
        - JETSTREAM_001
  alertmanager:
    - name: alertmanager
      url: http://alertmanager:9093/api/v2/alerts

Config File Example

A persistent deployment monitoring a real NATS system with all features enabled:

# /etc/insights/config.yaml
log-level: info

data-dir: /data/insights

sys:
  server: nats://nats.internal:4222
  creds: /etc/insights/sys.creds

web:
  hostname: 0.0.0.0
  port: 8080
  session-seed: 'your-stable-secret'

db:
  retention:
    duration: 72h

scraper:
  interval: 20s # faster than the 1m default; adjust based on cluster size

./insights --config /etc/insights/config.yaml

For all available keys, env-var equivalents, and the full precedence order, see Configuration.