Skip to main content

CLI

The insights binary is a single command with several subcommands. Running insights with no subcommand starts the server. web, mcp, http and prometheus each run their own long-lived process. The client subcommands (db, checks, ops, system list, node list) talk to an already-running node over NATS and exit. The config subcommands are the exception: they read the configuration and the local data directory and need no running server.

The collector build, released as the separate insights-collector binary, carries only serve, ops pprof, config check, config migrate, system list and node list. Its serve is the collector and is the default subcommand, as the server is in the full build; see insights-collector serve. In that build the config subcommands take the collector's flags rather than the server's. See Federated Deployment.

Synopsis​

insights [flags] [subcommand] [subcommand-flags]
SubcommandPurpose
(default) serve (alias run)Start the Insights server.
webRun only the web tier; connect to an indexer/scraper node over NATS.
prometheusServe one Prometheus /metrics endpoint for every node in the deployment.
mcpRun a Model Context Protocol server exposing the read-only db and checks tools to AI agents.
httpRun a stateless HTTP gateway exposing the read-only DB query API over JSON/CSV.
db queryRun a SQL query against the embedded DuckDB over the NATS API. query is a hidden alias.
db explainPlan a query with EXPLAIN; --analyze executes and profiles it.
db schemas / db tables / db columns / db macrosDiscover the queryable schemas, tables, columns, and macros.
checks listList all available checks grouped by category.
checks info <CODE>Show full metadata for a single check.
checks findings <CODE>Run one check and return its findings.
ops backup saveBuild a local copy of the database by streaming it table by table. save is the default, so ops backup [flags] runs it.
ops backup view <file>Serve the web UI against a backup file, opened read-only.
ops sizingRecommend a DuckDB memory_limit / threads pair for the running instance.
ops memoryShow the live DuckDB memory profile (current and peak by component).
ops pprofCapture a Go runtime profile (goroutine, heap, ...) from the running instance.
config checkLoad the configuration serve would load, report its problems, and report state this data directory holds that the configuration no longer claims.
config migrateMove state onto the ids the configuration now uses, or forget what it has walked away from.
system listDiscover the systems the deployment holds over $INS.ping.
node listDiscover the reachable Insights nodes over $INS.ping.
--versionPrint the binary version and exit.

Configuration precedence​

All flags can be supplied via any of three sources. They are applied in the following order, with later sources overriding earlier ones:

  1. Default values (defined in code)
  2. YAML config file (--config / -c, INSIGHTS_CONFIG, or a default search path)
  3. Environment variables (INSIGHTS_*)
  4. Command-line flags

Environment variables mirror the flag path: dots become underscores, everything is uppercased, and the INSIGHTS_ prefix is added. For example, --web.port becomes INSIGHTS_WEB_PORT, and --sys.server becomes INSIGHTS_SYS_SERVER.

A subcommand's own flags follow the same rule without the subcommand's name, so --format reads INSIGHTS_FORMAT and --timeout reads INSIGHTS_TIMEOUT on every subcommand that has them. insights <subcommand> --help prints each flag's variable. The top-level connection flags are the exception: they read INSIGHTS_NATS_*.

The config file is loaded from --config/-c, then INSIGHTS_CONFIG, then the default search paths <user-config-dir>/insights/config.yaml and ./insights.yaml. See Configuration → Config File for the full search order and a client/server example.

Top-level flags​

These flags apply to every subcommand.

FlagEnvDefaultDescription
--config / -cINSIGHTS_CONFIG(search paths)Path to a YAML config file. Applies to every subcommand, including db query and the other API clients.
--log-levelINSIGHTS_LOG_LEVELinfoLog level: debug, info, warn, error.
--serverINSIGHTS_NATS_SERVER(the selected context's, else nats://127.0.0.1:4222)NATS server URL of the Insights API: where the client subcommands and the web, mcp, http and prometheus processes send their requests.
--credsINSIGHTS_NATS_CREDS(empty)Path to a NATS credentials file.
--contextINSIGHTS_NATS_CONTEXT(empty)NATS CLI context name (resolved via ~/.config/nats/).
--user / --passwordINSIGHTS_NATS_USER / INSIGHTS_NATS_PASSWORD(empty)Basic auth.
--nkey / --jwtINSIGHTS_NATS_NKEY / INSIGHTS_NATS_JWT(empty)NKey seed and user JWT. A JWT on its own is sent as a bearer token.
--tls-cert / --tls-key / --tls-ca-certINSIGHTS_NATS_TLS_CERT / INSIGHTS_NATS_TLS_KEY / INSIGHTS_NATS_TLS_CA_CERT(empty)Client certificate, key, and CA certificate paths.
--tls-firstINSIGHTS_NATS_TLS_FIRSTfalsePerform the TLS handshake before the NATS protocol handshake.
--socks-proxyINSIGHTS_NATS_SOCKS_PROXY(empty)SOCKS proxy URL.
--inbox-prefixINSIGHTS_NATS_INBOX_PREFIX(empty, meaning _INBOX)Request/reply inbox prefix, for an account not granted _INBOX.>. When set, the account must allow subscribing to <prefix>.>.

These connection flags mirror the nats CLI. Each also parses as a hidden --nats.* alias (e.g. --nats.server). See Configuration → nats.* for the complete reference (TLS, auth, socks proxy); the YAML nats: section maps onto these flags.

Client subcommands resolve the currently selected NATS CLI context when no context name is supplied. Select it once with nats context select <name>, then run ordinary insights db, insights checks, insights ops, insights system list, insights node list, insights mcp, insights http, and insights prometheus commands. Use --context only to override that selection for one invocation. Long-running insights serve infrastructure does not implicitly inherit the selected workstation context; configure its URLs and credentials explicitly (or name a context explicitly when that is intentional).

Selecting a system or node​

A node can host several systems (multi-system deployments), each with its own database and a scoped API subtree ($INS.sys.<id>.*). Every client subcommand that reads a system's data takes a shared flag to say which:

FlagEnvYAMLDescription
--system <id>INSIGHTS_SYSTEMsystem.idSystem to address. On db, checks, ops backup save, ops sizing, and ops memory. The http and mcp gateways take it per request instead.
--node <id>INSIGHTS_NODEnode.idInsights node (process) to address, for process diagnostics. On ops pprof.

Resolution order is flag → environment → config file. The YAML keys are the same system: / node: identity blocks a single-system node is configured with, so a shared config file drives both sides.

With nothing naming a target, the CLI runs one $INS.ping discovery round before dispatch. When at least one current Insights node answers, the collection waits its full two-second window so every responder has time to reply; those unnamed commands therefore add roughly two seconds of discovery latency. An entirely unbound discovery subject can finish immediately with no responders. This is load-bearing: two servers predating the scoped subjects can both bind the same flat subject on one NATS, where sending flat first would queue-balance the request onto an arbitrary system or node. Discovery makes the choice explicit:

  • exactly one system can answer queries → the request goes to its canonical subject automatically;
  • several can → an error lists them (the same table as insights system list) so you pick one with --system;
  • no catalog was advertised → the flat subject is tried once for compatibility with a pre-discovery server.

The compatibility fallback cannot distinguish multiple pre-discovery servers that share one NATS account; their flat aliases retain the original arbitrary queue behavior. Upgrade those servers before combining them on one account.

A named target skips discovery and never falls back: if --system core-prod gets no answer, that silence is the error you need to see. Set --system/--node (or their env/config forms) for the lowest-latency repeated invocation. Unnamed discovery requires permission to publish $INS.ping and receive replies; explicitly named calls need only their canonical endpoint permissions. Run insights system list / insights node list at any time to see what is out there.

The two long-running gateways (http, mcp) take no --system at all. Each request names its own, so one gateway serves every system and none of them can be answered by the wrong one: on http the system is a path segment (/sys/<id>/db/query); on mcp it is a required system argument on every tool. Neither caches topology, so neither can route on a stale view, and adding a system to a deployment changes nothing for clients already addressing another.

That leaves resolving which system to the client, which is where the two differ. An HTTP client is configured: a dashboard interpolates a template variable populated from GET /systems into the path. An MCP client is an agent and can work it out: the systems tool tells it how, and one result is the answer while several means ask the user rather than choose for them.

insights config check​

Validates the configuration and diffs it against the identity snapshot, so a rename or a decommission is settled before a restart rather than found in the log after one.

It talks to nothing. ops acts on runtime state (a live NATS connection, JetStream, a running instance's profiles), while the config commands read the config file and the local data directory and nothing else. The split is what a command reaches for, not how long its process lives. That is also why these can run while the instance is stopped, and must: moving a catalog means no process may hold it open.

It carries the same flags serve does and resolves them from the same file, envs and flags, so what it validates is what serve would run.

insights config check -c /etc/insights/config.yaml
configuration is valid

config /etc/insights/config.yaml
node node-a
systems production
data dir ./data
disabled production: account-012

unclaimed state

system "prod" is no longer configured
rename insights config migrate --rename system:prod=<new-id>
prune insights config migrate --prune system:prod

State is reported by id, not by the streams and files that hold it. Which assets those are is a detail of the running version, the decision is the same either way, and naming them only invites someone to move them by hand.

An invalid configuration exits non-zero. Unclaimed state does not, because leaving a decommissioned system's data in place is a legitimate end state. Pass --strict to fail on it, which is what makes this usable as a deploy gate.

A check code matching no registered check is an error, in disabled-checks and in check-thresholds alike, and serve refuses to start on one. Silently ignoring the entry is the worse outcome: a dropped disabled-checks code leaves the check on, so an operator who had silenced a noisy check gets flooded by it, and a dropped check-thresholds key reverts to the compiled-in default. An unknown or renamed parameter name under check-thresholds is an error too. A retired check's code is the exception: it is accepted with a warning that the check is no longer evaluated, and its thresholds are ignored. Every offending code is reported in one run, attributed to where it was written, and a code that is only spelled the old way is named with its replacement:

configuration is not valid: invalid check configuration:
- disabled-checks: check code "ACCOUNT_012" was renamed to "account-012"
- systems[edge-west].disabled-checks: unknown check code "stream-999"
run 'insights checks list' for the valid codes and 'insights checks info <code>' for a check's parameters

Checking is not side-effect free: it runs serve's license check, so a trial build fails the same way when no license is configured, and it provisions the data directory exactly as serve would. A temporary directory it invented because --data-dir was unset is removed again rather than left behind on every CI run.

Nothing here deletes anything: a rename moves state onto the new id, a prune drops it from the snapshot, and both leave the data where it is.

A node id change is reported without a command. Nothing on disk carries a node's name, its endpoints are rebuilt from config on the next start, and its delivery history expires on its own, so there is nothing to rename onto the new id and no stale node record to prune.

insights config migrate​

Performs what config check reports. It moves each catalog whose name changed onto the name the configuration now uses, and records the result, so the id now in use owns the history the old one accumulated.

The process must be stopped. This renames database files, and a running instance holds them open.

Run with no flags, it migrates the naming scheme: every system whose id held still while the names beneath it moved. That is the upgrade case, where nothing an operator wrote changed and the state would otherwise be orphaned by the binary rather than by them.

A rename and a decommission-plus-onboard are indistinguishable from the snapshot alone ("east, west" becoming "east, north" is equally either), so a change of ids has to be declared:

insights config migrate -c /etc/insights/config.yaml --rename system:west=north
insights config migrate -c /etc/insights/config.yaml --prune system:west

Both flags repeat, so several systems migrate in one run:

insights config migrate -c config.yaml --rename system:west=north --rename system:staging=south
moved
systems/staging.db -> systems/south.db
systems/west.db -> systems/north.db

not carried over
system "staging" scrape buffer
system "west" scrape buffer

--prune drops the record and nothing else: the catalog stays on disk, so a system pruned by mistake is recovered by putting its id back in the configuration. Only a system can be pruned: the configuration names exactly one node, and it is whichever one is running, so a data directory never holds a spare node record to forget.

What "not carried over" means: the scrape stream is dropped rather than moved. It is a bounded buffer and the catalog is the record; the indexer is rarely more than an epoch behind it, so what goes unindexed is at most that epoch, which the next scrape derives again. Webhook delivery history is left under the old name for the same reason: it ages out on its own, and the endpoints themselves are rebuilt from config on the next start.

Every move is checked before any of them runs, and a move onto an existing file is refused rather than performed: a half-migrated data directory is the state this whole mechanism exists to prevent, and a catalog is the one thing here that cannot be re-derived.

insights serve​

Default subcommand. Starts the Insights server: scraper, indexer, web UI, sink, and the NATS API endpoints. run is a backward-compatible alias for serve.

insights serve [flags]

Because serve is the default, you can omit the subcommand name:

insights --config /etc/insights/config.yaml

Flags​

serve owns every server-only subsystem flag. Rather than duplicate each of them here, see the Configuration reference; each subsystem (scraper, indexer, web, sink, simulator, updater, database, prometheus, license, etc.) has its own section there.

Because these flags are scoped to serve, a bare insights --help lists only the global flags; run insights serve --help to see the full server set (including all --sys.* and --scraper.*).

Each subsystem is toggled with a shorthand on/off flag (--web, --indexer, --scraper, --simulator, --prometheus, --updater, --telemetry) or the matching environment variable, INSIGHTS_WEB, INSIGHTS_UPDATER, and so on. The verbose --<subsystem>.enabled / INSIGHTS_<SUBSYSTEM>_ENABLED form is also accepted but hidden from --help: --web=false is the same as --web.enabled=false, and the shorthand wins if both are given. (In YAML the section is a map, so use its enabled: key; see Subsystem Toggles.)

insights web and insights-collector serve expose only the shorthands that apply to them: --prometheus, --updater and --telemetry on web, --updater and --telemetry on the collector. Each forces its own tier on (the web server and the scraper respectively), so there is no --web or --scraper toggle there.

Frequently used flags:

FlagEnvDefaultDescription
--data-dirINSIGHTS_DATA_DIR(temp dir)Base directory for DuckDB and JetStream data.
--simulatorINSIGHTS_SIMULATORfalseRun against the built-in simulator instead of a real NATS system. A trial build skips license validation when enabled.
--sys.serverINSIGHTS_SYS_SERVER(empty)NATS URL of the target system to scrape.
--sys.contextINSIGHTS_SYS_CONTEXT(empty)NATS CLI context name for the target system (resolved via ~/.config/nats/); an alternative to --sys.server/--sys.creds.
--sys.credsINSIGHTS_SYS_CREDS(empty)NATS credentials file for the target system.
--web.hostnameINSIGHTS_WEB_HOSTNAME127.0.0.1Web UI bind address.
--web.portINSIGHTS_WEB_PORT8080Web UI bind port.
--license.tokenINSIGHTS_LICENSE_TOKEN(empty)License JWT string, for a trial build.
--license.fileINSIGHTS_LICENSE_FILE(empty)Path to a file containing the license JWT, for a trial build.

Example​

insights \
--config /etc/insights/config.yaml \
--web.hostname 0.0.0.0

insights web​

Runs only the web tier, with no indexer, scraper, or simulator, and no embedded NATS server, so the process holds no local state and serves the UI entirely from data and control it reaches over NATS. Because it is a pure client of the API, it exposes only the flags a web tier uses: the web server (--web.*), the Prometheus endpoint (--prometheus.*), the updater (--updater.*), --telemetry (whether what people do in its UI is passed to the nodes that report it; see Telemetry), and the root connection flags (--context, --server, …). The full server configuration lives on insights serve.

Point it at the node running the indexer/scraper with --server (or --context); that node owns the DuckDB and serves the $INS.db.* / $INS.ops.* endpoints this tier consumes. The instance's capabilities (geo enrichment, the realtime advisory feed, the scrape interval, retention, and the license) are discovered from that node over $INS.ops.info, and version info, NATS metrics, and scraper controls are sourced over NATS too, so none of it need be configured locally. A web-only node validates no local license; it renders the backend's.

insights web [flags]

Example​

# Node A — indexer + scraper, no web, embedded NATS on a fixed port:
insights --web=false --sink.port=4222 --data-dir /var/lib/insights

# Node B — stateless web tier pointing at Node A:
insights web --server=nats://node-a:4222 --web.hostname 0.0.0.0 --web.port 8080

insights-collector serve​

Runs a collector: it scrapes one NATS system and writes each epoch to a JetStream stream, serving only the parts of that system's $INS.sys.<id>.* API that need no database (scraper control, diagnostics, events) and $INS.node.<id>.ops.pprof. It has no DuckDB, no indexer, no web tier, no Prometheus endpoint and no embedded NATS server. A full Insights node elsewhere mirrors the stream, indexes it, and serves queries and the UI for the system. It is the default command of the insights-collector binary and is not part of insights. Federated Deployment covers the leafnode setup, JetStream domains, and the central node's mirror.

insights-collector [serve] [flags]

The stream lives on an external NATS server, typically a leafnode local to the collector, named with the top-level connection flags (--server, --context, …) or --scraper.nats.*. With none of them set the command refuses to start. --sink.embed is listed in --help but always forced off.

Flags​

The collector takes the subset of the full server's flags it uses: --data-dir, --node.id, --system.id, --sys.* (the scraped system), --scraper.*, --sink.*, --license.*, --updater.* and --telemetry.*, plus the --updater and --telemetry toggles. Each is described in the Configuration reference; run insights-collector serve --help for the full list.

Example​

# --server/--creds: the local NATS that holds the stream and serves the API.
# --sys.*: the monitored system's $SYS-account connection, which is scraped.
insights-collector \
--server nats://localhost:4222 \
--creds /etc/insights/collector.creds \
--system.id edge-west \
--sys.server nats://monitored-system:4222 \
--sys.creds /etc/insights/sys.creds

insights mcp​

Run a Model Context Protocol server that exposes the read-only db and checks capabilities as MCP tools for AI agents. Like db query, it is a NATS client that forwards each tool call to a running Insights node and returns the reply; it owns no database and is read-only by construction. For client setup (Claude Desktop, Claude Code, Cursor, and HTTP), see the AI Agents (MCP) guide.

insights mcp [flags]

Flags​

FlagEnvDefaultDescription
--transportINSIGHTS_MCP_TRANSPORTstdiostdio for a client-spawned subprocess, or http for a long-running Streamable HTTP service.
--httpINSIGHTS_MCP_HTTP127.0.0.1:8090Listen address when --transport http.
--timeoutINSIGHTS_MCP_TIMEOUT2mPer-tool NATS request timeout.
--http-tlsINSIGHTS_MCP_HTTP_TLSfalseServe the http transport over HTTPS. Requires --http-tls-cert and --http-tls-key. Ignored for stdio.
--http-tls-cert / --http-tls-keyINSIGHTS_MCP_HTTP_TLS_CERT / INSIGHTS_MCP_HTTP_TLS_KEY(none)Paths to the TLS certificate and private key (PEM). Both required when --http-tls is set.

Plus the top-level connection flags (--context, --server, …) for reaching the server. The http transport binds loopback by default and applies no authorization of its own. Keep it local or front it with your own auth.

Example​

# Local stdio server, launched by an MCP client (Claude Desktop, Cursor, …).
# Uses the selected NATS context; add --context to override it for this process:
insights mcp

# Long-running HTTP service over TLS:
insights mcp --transport http --http-tls --http-tls-cert cert.pem --http-tls-key key.pem

insights http​

Runs a stateless HTTP gateway that exposes the read-only db query API over plain HTTP, so an HTTP-only client (a BI/dashboard datasource, a script, another service) can query Insights without a NATS client. Like web, it is a long-running process that owns no database: each HTTP request is translated into the matching $INS.db.* NATS request and the reply is returned verbatim, so the read-only guard, the row cap, and per-account NATS auth all stay enforced upstream. See the HTTP gateway section of the API reference for the route map and error-status mapping.

Point it at the node serving the $INS.db.* endpoints with --server (or --context), exactly like db query. It adds a bearer token at its own edge and refuses to start without one unless --allow-unauthenticated is given.

insights http [flags]

Flags​

FlagEnvDefaultDescription
--listenINSIGHTS_HTTP_LISTEN127.0.0.1:8080HTTP listen address. Use 0.0.0.0:<port> to accept remote connections (e.g. behind a TLS-terminating ingress).
--auth-tokenINSIGHTS_HTTP_AUTH_TOKEN(empty)Bearer token required on every request except /healthz (Authorization: Bearer <token>). Required unless --allow-unauthenticated.
--allow-unauthenticatedINSIGHTS_HTTP_ALLOW_UNAUTHENTICATEDfalseRun without a bearer token. Only for trusted, network-isolated deployments, never for public exposure.
--timeoutINSIGHTS_HTTP_TIMEOUT1mPer-request NATS request timeout.
--max-body-bytesINSIGHTS_HTTP_MAX_BODY_BYTES1048576Maximum request body size in bytes.

Plus the top-level connection flags (--context, --server, …) for reaching the node that serves $INS.db.*.

Example​

# Run the gateway pointed at an Insights node, bound for a TLS-terminating
# ingress, with a bearer token:
export INSIGHTS_HTTP_AUTH_TOKEN=$(openssl rand -hex 32)
insights http \
--server nats://insights.example.com:4222 \
--creds ~/.config/nats/query.creds \
--listen 0.0.0.0:8080

# Query it over HTTP:
curl -s localhost:8080/sys/prod-us/db/query \
-H "Authorization: Bearer $INSIGHTS_HTTP_AUTH_TOKEN" \
-d '{"sql":"SELECT name, cpu FROM hx.servers ORDER BY cpu DESC LIMIT 10"}'

insights prometheus​

Serves one Prometheus /metrics endpoint for every Insights node in the deployment. On each scrape it discovers the nodes over $INS.ping, asks each for its metrics, and serves the merged set with a source label naming the node each series came from. It holds no database and runs no indexer: it serves what a node with prometheus.aggregate: true serves, for when Prometheus can reach this process but not the nodes themselves. See Metrics. Scrape exactly one aggregating endpoint, or every series is reported once per aggregator.

insights prometheus [flags]

Flags​

FlagEnvDefaultDescription
--hostnameINSIGHTS_HOSTNAME127.0.0.1Bind hostname for the /metrics server.
--portINSIGHTS_PORT9091Bind port for the /metrics server.
--timeoutINSIGHTS_TIMEOUT2sBudget for one collection: node discovery, then the per-node requests. Must be shorter than the scrape timeout of whatever reads this endpoint.

Plus the top-level connection flags (--context, --server, …) for reaching the deployment's NATS.

Example​

insights prometheus --server nats://hub:4222 --hostname 0.0.0.0 --port 9091

insights db query​

Send a SQL query to the running Insights server over NATS and write the result to stdout. See the Search reference for the Insights query language; use db query for arbitrary DuckDB SQL against the hx schema.

insights db query [flags] [SQL]

insights query is a hidden alias for db query: it takes the same arguments and flags but is not listed in --help. Prefer db query in scripts and documentation.

The SQL statement can be passed as a positional argument or piped on stdin. If both are omitted, the command exits with an error.

Results are streamed from the server as Apache Arrow chunks and rendered locally, so arbitrarily large results work without buffering the whole result in memory on either side (the command falls back to the legacy buffered endpoint against an older server). Output formatting is canonical: timestamps are RFC3339, NULL is an empty CSV field / JSON null, and BLOB is base64. See the streaming query API.

--format text renders the result as a table for reading at a terminal:

insights db query --format text "SELECT name, cluster, connections, subscriptions, cpu, memory FROM hx.servers WHERE epoch = (SELECT max(epoch) FROM hx.server_stats) ORDER BY name"
Query Result (3 rows)
╭─────────────┬───────────┬─────────────┬───────────────┬───────┬───────────╮
│ NAME │ CLUSTER │ CONNECTIONS │ SUBSCRIPTIONS │ CPU │ MEMORY │
├─────────────┼───────────┼─────────────┼───────────────┼───────┼───────────┤
│ simulator-1 │ simulator │ 6 │ 257 │ 5.175 │ 648232960 │
│ simulator-2 │ simulator │ 0 │ 253 │ 5.175 │ 648232960 │
│ simulator-3 │ simulator │ 5 │ 257 │ 5.175 │ 648232960 │
╰─────────────┴───────────┴─────────────┴───────────────┴───────┴───────────╯

Numeric columns are right-aligned off the result's Arrow schema, so a column of digits that is not a number (a port, an account ID) still reads as text.

Unlike csv and json, text cannot be written incrementally (a table's column widths are only known once the last row has arrived), so it is the one format that buffers the whole result on the client. It is capped at 10,000 rows and fails with an error naming --format csv past that, rather than printing a truncated table that would read as a complete answer. csv is the default.

Flags​

FlagEnvDefaultDescription
--format / -fINSIGHTS_FORMATcsvOutput format: csv, json, or text. text renders a table and is capped at 10,000 rows.
--timeoutINSIGHTS_TIMEOUT1mPer-message receive timeout while streaming. Must exceed the server's stream batch timeout.
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Plus the top-level connection flags (--context, --server, …) for reaching the server.

Examples​

Inline SQL:

insights db query "SELECT name, cpu FROM hx.servers ORDER BY cpu DESC LIMIT 10"

SQL via stdin:

cat query.sql | insights db query --format json

Against a remote Insights server:

insights --server nats://insights.example.com:4222 \
--creds ~/.config/nats/ops.creds \
db query "SELECT COUNT(*) FROM hx.conns"

insights db​

Inspect the database without writing a full query: validate/plan (or profile) a statement, and discover the queryable schemas, tables, columns, and macros. These subcommands sit alongside db query under db, mirror the $INS.db.* discovery endpoints (see the API reference), and are scoped to the allowlisted schemas (hx, main, checks, contexts). The discovery commands scan no data: they read catalog metadata only; explain --analyze is the one exception, as it executes the query to gather runtime timing.

Discovery output defaults to a text table; pass --format json for the raw endpoint response (which also includes each object's entity-level table_comment) or --format csv.

insights db schemas
Schemas (4 rows)
╭──────────┬────────┬───────┬────────┬─────────────────────────────────────────────────────────────╮
│ SCHEMA │ TABLES │ VIEWS │ MACROS │ DESCRIPTION │
├──────────┼────────┼───────┼────────┼─────────────────────────────────────────────────────────────┤
│ checks │ 0 │ 0 │ 176 │ Checks framework: one table macro per check code │
│ │ │ │ │ (server-001 is checks.server_001) returning (code, entity, │
│ │ │ │ │ ...) finding tuples, plus the shared helper macros they │
│ │ │ │ │ call. │
├──────────┼────────┼───────┼────────┼─────────────────────────────────────────────────────────────┤
│ contexts │ 0 │ 0 │ 35 │ Context macros behind the evidence panels shown with a │
│ │ │ │ │ finding: cluster peer comparisons, raft peers, top streams │
│ │ │ │ │ and similar supporting views. │
├──────────┼────────┼───────┼────────┼─────────────────────────────────────────────────────────────┤
│ hx │ 40 │ 13 │ 0 │ Historical entity tables (*_ident/*_opts/*_stats) and the │
│ │ │ │ │ pre-joined views (servers, accounts, streams, consumers, │
│ │ │ │ │ ...) built on them. Each *_stats/view row is per entity per │
│ │ │ │ │ epoch, so ALWAYS scope by epoch — e.g. WHERE epoch = │
│ │ │ │ │ (SELECT max(epoch) FROM <entity>_stats) — and/or add a │
│ │ │ │ │ LIMIT; an unscoped SELECT * scans all retained history. │
│ │ │ │ │ Fully commented. │
├──────────┼────────┼───────┼────────┼─────────────────────────────────────────────────────────────┤
│ main │ 1 │ 0 │ 0 │ Default (unqualified) schema. Holds the geo-IP enrichment │
│ │ │ │ │ table `ips`. │
╰──────────┴────────┴───────┴────────┴─────────────────────────────────────────────────────────────╯

Every text table the CLI prints goes through one renderer (these discovery commands, checks, query --format text, and the deployment listings (system list, node list)), so the rules below hold wherever a table appears, in the collector build as well as the full one.

A table is laid out to fit the terminal it is printed to: the columns carrying prose (a schema description, a column comment) wrap at word boundaries so the box stays whole, and the columns holding a whole value (a name, a count, a type) keep their width. Output that is not going to a terminal is laid out to 100 columns. Table headers, and the severity column of checks findings, are colored when stdout is a terminal. Redirecting or piping the output, or setting NO_COLOR, makes it plain, so anything parsing the text sees no escape sequences. FORCE_COLOR=1 colors it regardless.

insights db explain [flags] [SQL]
insights db schemas [flags]
insights db tables [flags]
insights db columns [flags]
insights db macros [flags]

Flags​

FlagDefaultDescription
--format / -ftextOutput format. explain: text or json; the discovery commands also accept csv.
--analyze / -afalse(explain) Execute the query with EXPLAIN ANALYZE and include per-operator runtime timing.
--timeout30sNATS request timeout.
--schema(none)(tables, columns, macros) Restrict to one schema. Omit to list across all allowlisted schemas.
--table(none)(columns) Restrict to one table or view. Omit to span every object in scope.
--system(discovered)Target system on a multi-system deployment (env INSIGHTS_SYSTEM). See Selecting a system or node.

Plus the top-level connection flags (--context, --server, …) for reaching the server. explain takes the SQL as a positional argument or on stdin, like insights db query.

Examples​

Discover schemas, then drill into one:

insights db schemas
insights db tables --schema hx
insights db columns --schema hx --table servers
insights db macros --schema checks

Plan a query before running it:

insights db explain "SELECT name, cpu FROM hx.servers WHERE epoch = (SELECT max(epoch) FROM hx.server_stats)"

Execute and profile it with EXPLAIN ANALYZE to see per-operator runtime timing:

insights db explain --analyze "SELECT name, cpu FROM hx.servers WHERE epoch = (SELECT max(epoch) FROM hx.server_stats)"

Machine-readable column metadata (includes table_comment):

insights db columns --schema hx --table servers --format json

insights checks list​

List every check the server knows about, grouped by category. The command takes no SQL; it enumerates the registered checks. Default output is one text table per category. --format json prints the endpoint's response as-is, an array of categories each with a label and its checks; --format csv flattens the grouping.

insights checks list [flags]

When the addressed system has any check listed under disabled-checks, the text tables gain a Disabled column marking them; the column is absent otherwise. The CSV schema is fixed at code,name,disabled regardless. In JSON, a disabled check carries "disabled": true and an enabled one omits the field.

Flags​

FlagEnvDefaultDescription
--format / -fINSIGHTS_FORMATtextOutput format: text, csv, or json.
--timeoutINSIGHTS_TIMEOUT30sNATS request timeout.
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Example​

insights checks list --format json | jq '.[].label'

insights checks info​

Show full metadata for a single check: description, remediation, scope, severity, category, configurable parameters, and any extra detail columns.

insights checks info [flags] <CODE>

The text output always includes a Disabled line: whether the addressed system turned this check off, in which case it still runs but is left out of the health grades and never delivered as a notification.

Flags​

FlagEnvDefaultDescription
--format / -fINSIGHTS_FORMATtextOutput format: text or json.
--timeoutINSIGHTS_TIMEOUT30sNATS request timeout.
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Example​

insights checks info server-001

insights checks findings​

Run one check by code over a time window and return the resulting findings.

insights checks findings [flags] <CODE>

<CODE> is a check code such as server-001 or stream-010; see the Checks Reference for the full catalogue.

Flags​

FlagEnvDefaultDescription
--format / -fINSIGHTS_FORMATtextOutput format: text, csv, or json. The JSON response includes pagination metadata.
--timeoutINSIGHTS_TIMEOUT30sNATS request timeout.
--durationINSIGHTS_DURATION(empty)Relative window as a DuckDB interval (e.g. 1 hour, 15 minutes, 7 days).
--startINSIGHTS_START(empty)Absolute start time (UTC, YYYY-MM-DDTHH:MM:SS). When omitted, derived from --duration.
--endINSIGHTS_END(empty, meaning "live")Absolute end time (UTC).
--pageINSIGHTS_PAGE1One-based page number when results exceed one page.
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Example​

Run server-003 (High CPU Usage) against the last hour:

insights checks findings server-003 --duration "1 hour"

The text table carries only the columns the check declares, and right-aligns the ones it declares numeric. Severity appears as a column only for checks that compute it per finding, where it is colored by level at a terminal; for every other check it is a property of the check itself, shown by checks info.

Findings for account-005: No Subscription Interest (4 rows)
╭────────┬─────────────────────────────┬────────────────┬─────────────────────────────╮
│ ENTITY │ SUBJECT │ SOURCE ACCOUNT │ LOCAL SUBJECT │
├────────┼─────────────────────────────┼────────────────┼─────────────────────────────┤
│ $G │ $SYS.REQ.ACCOUNT.$G.CONNZ │ SYS │ $SYS.REQ.ACCOUNT.PING.CONNZ │
│ $G │ $SYS.REQ.USER.$G.INFO │ SYS │ $SYS.REQ.USER.INFO │
│ APP1 │ $JS.API.> │ SYS │ $JS.API.> │
│ APP1 │ $SYS.REQ.ACCOUNT.APP1.STATZ │ SYS │ $SYS.REQ.ACCOUNT.PING.STATZ │
╰────────┴─────────────────────────────┴────────────────┴─────────────────────────────╯

A check this system has disabled is marked [disabled] after its name in the title. Its rows are real findings, but the check is not graded and nothing it finds is delivered, so the marker is the only cue at the terminal. The JSON output carries the same fact as disabled: true; the CSV output has no marker.

Page through findings as JSON:

insights checks findings stream-002 --format json --page 2

insights ops backup​

ops backup has two subcommands: save builds a backup and view serves one in the web UI. save is the default, so insights ops backup [flags] and insights ops backup save [flags] are the same command.

ops backup save builds a local copy of a system's Insights database. The client streams each table over $INS.db.backup.stream and inserts it into a DuckDB file it creates. The server writes nothing, so the disk the backup needs is on the host running the command, not on the server. The result is a complete database (schema, views, macros and comments come from the insights binary running the command), so it can be restored as a --data-dir, or read where it lies with insights ops backup view. To look at one, use the viewer: a --data-dir is opened read-write, and the server migrates and prunes what it finds there.

The output must be a real file: the copy writes into a seekable DuckDB database, so the backup cannot be piped to another process until it is complete.

insights ops backup save [flags]

ops backup save always prints a plan summary (time range, epoch coverage, estimated file size) to stderr. When stdin is a terminal, it then prompts for confirmation before running. Pass --force / -f to skip the prompt; --force is required when running non-interactively (e.g. cron, CI).

Each table's row count is printed as it completes. On a terminal, the table being copied also reports its progress against the plan's estimate on a line rewritten in place, worth having because the largest table is minutes of work on a big instance. That line is only drawn on a terminal, so logs and CI transcripts get the per-table completion lines alone; --no-progress turns it off there too.

The latest epoch is resolved once, before the first table, and every table is filtered to it. That is what keeps tables copied minutes apart consistent with each other, and why a scrape landing mid-run cannot leave one table holding an epoch another table missed.

A retention sweep during the run is the one thing the client cannot pin. It only ever deletes the oldest epochs, so the tables copied first keep rows the ones copied last have already lost. The command warns, names the epoch below which the tables disagree, and keeps the file: the rows in question are ones the server itself has just deleted, and the rest of the range is unaffected. For an exact range, pass a --start above the retention horizon, where a sweep cannot reach the window at all. A window that changes in a way retention cannot explain still fails.

Metadata reads retry transient NATS timeouts and unavailable connections up to three attempts, with short backoffs. --timeout bounds each attempt as well as each message of the table copy. If the final window re-read remains unavailable, the command warns that it could not verify source-window changes and keeps the completed backup. A successful re-read that detects an unexplained change still fails, as do rejected queries and malformed responses.

--start bounds history, not identity. An entity's identity row is written once, when it is first seen, so a windowed copy keeps the identity and configuration its rows refer to regardless of age; otherwise the file's stats would name entities it cannot resolve. It verifies this before saving and fails rather than write a file that looks complete. An identity row the server itself no longer holds is the one exception: the copy carries the same gap the server has, so it is kept and reported as a warning naming the tables and keys.

Run it with the same Insights version as the server. A schema difference is refused up front, naming each column that has nowhere to go or whose type changed: an INSERT would cast such a column rather than reject it, storing something other than what the server held. A column this binary has and the server lacks is a warning instead: it is filled with its declared default, as upgrading that server would fill it, and the warning names the value.

Flags​

FlagEnvDefaultDescription
--output / -oINSIGHTS_OUTPUTinsights-backup-<system>-<timestamp>.dbLocal output path. The file is built alongside it as <output>.part and renamed on success.
--startINSIGHTS_START(no lower bound)Start time: absolute (YYYY-MM-DD, RFC3339, or YYYY-MM-DDTHH:MM:SS UTC) or relative duration (7d, 1h, 30m).
--endINSIGHTS_END(latest epoch)End time: same formats as --start.
--force / -fINSIGHTS_FORCEfalseSkip the confirmation prompt. Required for non-interactive use.
--no-progressINSIGHTS_NO_PROGRESSfalseDon't report per-table progress while copying.
--timeoutINSIGHTS_TIMEOUT10mHow long to wait for each metadata request attempt or message of a table's backup stream.
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Examples​

Back up the last seven days, prompting for confirmation:

insights ops backup save --start 7d --output weekly-backup.db

Non-interactive full backup to a dated filename:

insights ops backup save --force --output "insights-$(date +%Y%m%d).db"

insights ops backup view​

Serve the web UI against a backup file. The file is opened in DuckDB's read-only access mode and nothing that writes is started: no scraper, no indexer, no webhook delivery, no retention sweep, no updater, and no connection to your NATS system. Browsing a backup cannot change it.

insights ops backup view <file> [flags]

The alternative, moving a backup into a data directory and starting the server over it, opens the operator's only copy read-write, migrates it to the running binary's schema, and leaves retention live enough to prune it. This command is what that reaches for.

A NATS server does start, embedded and bound to loopback on an ephemeral port, because the web tier reaches its own database over the query API rather than through a handle of its own. Its JetStream store is a temporary directory removed on exit.

The pages mount under /systems/backup; insights prints the URL once it is serving. A backup carries no system identity of its own, so the id is that fixed name rather than the system the data came from.

Views describing the serving process rather than the backed-up system (the Operations page's NATS panel, the node inventory) report this process. The Operations page offers no scraper controls, since nothing scrapes into a backup, and its Process panel lists the scraper as None. The time controls have no live end to follow: the toolbar reads OFFLINE and pins to the backup's last epoch. Their presets and the charts' resolution follow the cadence the data was scraped at, read from the spacing of the file's epochs since a backup records no interval.

Open the file with the same Insights version that wrote it. A backup carries its own views, macros, and comments, and a read-only handle cannot refresh them, so a mismatched binary would read them as something they are not. The command compares the file's recorded migrations against its own and refuses to open on any difference, naming what differs.

Flags​

FlagDefaultDescription
--hostname127.0.0.1Web server bind hostname.
--port8080Web server bind port.

Example​

insights ops backup view ./insights-backup-prod-20260816-101500.db --port 8123

insights ops sizing​

Recommend a memory_limit / threads configuration for the running instance, grounded in DuckDB's per-thread guidance plus the un-spillable floor (the largest write transaction + the resident ART/metadata footprint).

Sizing is a live operation. The report is computed server-side over the $INS.ops.sizing endpoint against the live database handle, and the host capacity (cores, RAM) reflects the real host the limits apply to. That is what makes it actionable: run it against the instance that is actually under load (for example when investigating a performance issue), then decide whether to retune the limits or add host memory. (Like db query, checks, and ops backup save, this talks to an already-running server over NATS.)

When the memory profiler is enabled (the default), the write-path and resident floors are taken from measured peaks observed under real load rather than estimates; with no samples they fall back to conservative constants and the report flags that. The floors do not scale with retention; they track per-epoch and retention-prune transactions. The report separately projects steady-state disk size at full retention and flags when that size exceeds host RAM (a cache/performance signal, not an OOM floor).

The biggest single write is usually a retention prune, which runs far less often than the profiler's rolling window. Use --observe <duration> to subscribe to the live profile for long enough to span a prune so that peak is captured; otherwise the write floor reflects only per-scrape commits.

insights ops sizing [flags]

Flags​

FlagEnvDefaultDescription
--per-threadINSIGHTS_PER_THREAD2GiB per thread used in the query budget (DuckDB recommends 1–4).
--threadsINSIGHTS_THREADS0Threads to size for (0 = recommend min(cores, 4)).
--observe, -oINSIGHTS_OBSERVE0Observe the live memory profile for this long and size from the peaks seen (0 = one-shot).
--timeoutINSIGHTS_TIMEOUT30sNATS request timeout.
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Examples​

Size the running instance against its real host:

insights ops sizing

Size for a specific thread count and a join-heavy per-thread budget:

insights ops sizing --threads 8 --per-thread 4

Observe live load for an hour (ideally spanning a retention prune) and size from the observed peaks:

insights ops sizing --observe 1h

insights ops memory​

Show the running instance's live DuckDB memory profile: current and peak resident bytes per component (duckdb_memory() tag), over the profiler's rolling window. This is the ground truth behind ops sizing's measured floors, and a quick way to watch what the database is actually doing under load.

An always-on background sampler in the server process records duckdb_memory() every --db.memory-profile.interval (default 2s) into a --db.memory-profile.window (default 15m) window; ops memory reads that retained snapshot over the $INS.ops.memory endpoint; it does not sample on demand. The same readings are exported as the insights_duckdb_memory_bytes{tag=...} Prometheus gauge for charting and alerting.

Key components: BASE_TABLE (buffered table data), ART_INDEX (resident indexes), TRANSACTION (the in-flight write/commit), and query work (HASH_TABLE, ORDER_BY, …) which spills to disk. The un-spillable floor is ART_INDEX + TRANSACTION.

A summary table gives the magnitudes (current / peak / peak time), followed by a small line chart per component. Each chart is self-scaled to its own min–max range (so the shape is visible regardless of absolute level; a flat line means steady), with the y-axis labelled in IEC bytes. Components are listed in a fixed order (un-spillable ones first, then spillable), so the table and charts stay put across --live refreshes instead of reordering as values change:

tag current peak peak at
ART_INDEX 4.0 MiB 4.0 MiB 08:11:37
ALLOCATOR 25.2 MiB 69.0 MiB 08:11:37
TRANSACTION 0 B 1.4 GiB 08:21:37
BASE_TABLE 1.1 GiB 1.1 GiB 08:26:35
TOTAL 1.1 GiB 2.4 GiB 08:23:43

Trends (last 15m — each chart self-scaled to its own min–max):

TRANSACTION
1.4 GiB ┤─╮ ╭╮ ╭╮ ╭─╮ ╭╮ ╭╮ ╭╮ ╭─╮
┤ │ ││ ││ │ │ ││ ││ ││ │ │
┤ │ ││ ││ │ │ ││ ││ ││ │ │
0 B ┤ ╰──╯╰──╯╰──╯ ╰──╯╰──╯╰──╯╰──╯ ╰

BASE_TABLE
1.1 GiB ┤ ╭─────
┤ ╭─────────╯
┤ ╭─────────╯
905.1 MiB ┤──────╯
insights ops memory [flags]

Flags​

FlagEnvDefaultDescription
--timeoutINSIGHTS_TIMEOUT30sNATS request timeout.
--live, -lINSIGHTS_LIVEfalseContinuously re-render the profile as the server samples it (Ctrl-C to stop).
--systemINSIGHTS_SYSTEM(discovered)Target system on a multi-system deployment. See Selecting a system or node.

Examples​

insights ops memory

With --live, the command renders an immediate frame from the request/reply, then subscribes to the $INS.feed.memory broadcast and re-renders in place on each sampler tick (one per --db.memory-profile.interval) until interrupted:

insights ops memory --live

If the profiler is disabled (--db.memory-profile.enabled=false), the command reports that no samples are available (and --live receives no updates, since nothing is published).

insights ops pprof​

Capture a Go runtime profile from the running instance over the $INS.ops.pprof endpoint. The server does not expose the net/http/pprof HTTP handlers, so this is the way to reach the standard runtime/pprof snapshots, most usefully a goroutine dump to see where the process is parked when something has stalled (e.g. a scrape/index loop that stopped advancing).

insights ops pprof [flags]

Flags​

FlagEnvDefaultDescription
--typeINSIGHTS_TYPEgoroutineProfile to capture: goroutine, heap, allocs, threadcreate, block, mutex.
--debugINSIGHTS_DEBUG0runtime/pprof debug level: 0 = binary protobuf for go tool pprof; 1 = legacy text with aggregated counts; 2 = full per-goroutine stacks (only differs from 1 for --type goroutine; identical to 1 for every other profile).
--output, -oINSIGHTS_OUTPUT(stdout)Write the profile to this file instead of stdout.
--timeoutINSIGHTS_TIMEOUT30sNATS request timeout.
--nodeINSIGHTS_NODE(discovered)Target Insights node on a multi-node deployment. See Selecting a system or node.

A binary profile (--debug 0) is refused when stdout is a terminal. Pass --output to write it to a file, or --debug 2 for a readable text dump.

Examples​

# read a goroutine dump directly
insights ops pprof --type goroutine --debug 2

# capture a heap profile and open it in pprof
insights ops pprof --type heap --output heap.pprof
go tool pprof heap.pprof

block and mutex return an empty profile unless their sampling was enabled at server startup. The whole profile is returned in a single NATS reply, so a profile larger than the server's max payload fails with a 413 error rather than being truncated.

insights system list​

Discover the systems the deployment holds: broadcast one $INS.ping round and print the replies as a table: the same data the --system fallback and its ambiguity error are built from.

insights system list [flags]

One row per logical system, not per ping reply. Every Insights process answers ping (collectors included, advertising their scrape side), so a system whose scrape and catalog sides live on different nodes still appears once here, listed against the nodes holding its active data source. ROLES names that source: scrape whenever any node is scraping the system, catalog for a retained catalog whose scraper is gone. The physical placement of every participation is what node list shows.

$ insights system list

╭───────────┬────────┬────────────────┬─────────╮
│ SYSTEM │ ROLES │ NODE │ VERSION │
├───────────┼────────┼────────────────┼─────────┤
│ core-prod │ scrape │ hub-us-east │ 1.2.0 │
│ edge-west │ scrape │ edge-collector │ 1.2.0 │
╰───────────┴────────┴────────────────┴─────────╯

2 systems on 2 nodes.

When any system carries metadata labels (system.metadata), a METADATA column lists them as key=value pairs, as reported by the node scraping that system. A node that holds a system's catalog but cannot open it is named below the table, one line per node, as ! <system>: catalog unavailable on node <node> — <reason>; the system still lists, because the node still holds the role.

Nothing answering prints "no insights node responded" and exits zero: an empty deployment is a valid answer to "what is out there", not an error.

Flags​

FlagEnvDefaultDescription
--format / -fINSIGHTS_FORMATtextOutput format: text or json. JSON emits the raw ping replies for scripting.
--timeoutINSIGHTS_TIMEOUT2sHow long to collect ping replies.

insights node list​

The node-major view of the same discovery round: one row per responding Insights process (full nodes and collectors alike) with the systems each participates in. Scrape-only participations are annotated, since they cannot answer queries:

$ insights node list

╭────────────────┬─────────┬──────────────────────╮
│ NODE │ VERSION │ SYSTEMS │
├────────────────┼─────────┼──────────────────────┤
│ edge-collector │ 1.2.0 │ edge-west (scrape) │
│ hub-us-east │ 1.2.0 │ core-prod, edge-west │
╰────────────────┴─────────┴──────────────────────╯

2 nodes.

A node carrying metadata labels (node.metadata) adds a METADATA column, as on system list. Use it to find the id for ops pprof --node. Flags are the same as system list.

--version​

Print the binary version and exit. Recognised only as the sole argument:

insights --version
1.2.0

Official release binaries and container images print the product version bare, as above; unstamped binaries print dev.