Deployment Guide

Insights is a single Go binary with independently toggleable subsystems. How you configure them determines the deployment topology. See Architecture for what each subsystem does.

The flags shown below are a subset chosen for the deployment scenarios on this page. For the full reference (every flag, env var, and YAML key) see Configuration.

Deployment Topologies

Demo (Simulator)

Zero-config mode for evaluation. Starts an embedded NATS simulator with traffic workloads. All data is ephemeral.

./insights --simulator.enabled

Everything runs in-process: simulator, embedded sink, scraper, indexer, web UI. No external NATS required.

Choose a profile to control deployment size:

./insights --simulator.enabled --simulator.profile super-medium

Available profiles: core-{small,medium,large}, js-{small,medium,large}, super-{small,medium,large}, leaf-{small,medium,large}, super-leaf-{small,medium,large}.

All-in-One (Ephemeral)

Monitor a real NATS deployment. Requires system account credentials for $SYS access.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds

Or with a NATS context:

./insights --sys.context my-system

Data is stored in-memory (DuckDB) and a temp directory (JetStream). Both are lost on restart.

All-in-One (Persistent)

Same as above with durable storage that survives restarts.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds \
  --data-dir /data/insights

The default web.session-seed is "insights", which preserves sessions across restarts out of the box. For production, set it to a unique secret:

--web.session-seed "your-secret-seed"

For all --sys.* authentication options (creds, context, basic auth, NKey, JWT, TLS, SOCKS proxy), see Configuration > sys.*.

External NATS Sink

Disable the embedded NATS server and use an external NATS cluster with JetStream for the scrape stream. This decouples the scraper from the indexer and enables distributed topologies.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds \
  --sink.embed=false \
  --nats.server nats://sink-cluster:4222 \
  --nats.creds /path/to/sink.creds \
  --data-dir /data/insights

The nats.* config is the top-level connection used by the API, indexer, and scraper sink (unless overridden). When sink.embed is false, the JetStream stream must already exist or be created on the external server.

You can override the NATS connection independently for the indexer or scraper. The fallback order is scraper.nats → indexer.nats → nats (top-level).

nats:
  server: nats://api-cluster:4222

indexer:
  nats:
    server: nats://data-cluster:4222

scraper:
  nats:
    server: nats://data-cluster:4222

Headless (No Web UI)

Run without the web UI. The NATS micro API remains available for direct nats req queries.

./insights \
  --sys.server nats://target:4222 \
  --sys.creds /path/to/sys.creds \
  --web.enabled=false

Query via the NATS CLI:

nats req '$INS.db.query' '{"sql": "SELECT * FROM hx.servers LIMIT 5"}'

For programmatic and AI-agent access, see the AI Agents guide.

Indexer-Only

Consume from an existing scrape stream without running the scraper. Useful when a separate instance or process handles scraping.

./insights \
  --scraper.enabled=false \
  --sink.embed=false \
  --nats.server nats://data-cluster:4222 \
  --data-dir /data/insights

Operational Tuning

Scrape interval and timeout

--scraper.interval 1m     # Time between scrape cycles (default: 1m)
--scraper.timeout 30s     # Per-request timeout (default: 30s)

Retention

Limit how much historical data is kept in DuckDB:

--db.retention.duration 24h  # Keep last 24 hours (default: 768h)
--db.retention.interval 10m  # Sweep interval (default: 10m)

Web binding

--web.hostname 0.0.0.0    # Bind to all interfaces (default: 127.0.0.1)
--web.port 8080            # Bind port (default: 8080)

Config File Example

A persistent deployment monitoring a real NATS system with all features enabled:

# /etc/insights/config.yaml
log-level: info

data-dir: /data/insights

sys:
  server: nats://nats.internal:4222
  creds: /etc/insights/sys.creds

web:
  hostname: 0.0.0.0
  port: 8080
  session-seed: 'your-stable-secret'

db:
  retention:
    duration: 72h

scraper:
  interval: 20s # faster than the 1m default; adjust based on cluster size

./insights --config /etc/insights/config.yaml

For all available keys, env-var equivalents, and the full precedence order, see Configuration.