Traces

Tracing captures the raw NATS wire protocol exchanged between a client and the gateway. Every protocol operation — CONNECT, INFO, PUB, HPUB, SUB, UNSUB, MSG, HMSG, PING, PONG — is recorded with timestamps and direction. This is analogous to a tcpdump capture but at the NATS protocol level.

Trace files serve two purposes:

• Forensic analysis — inspect exactly what a client sent and received during an incident
• Bundle testing — replay captured traffic against bundles to validate that rules behave correctly

Trace profiles

Connections come and go with changing connection IDs, so traces are managed through profiles. A profile defines matching criteria — when a connection matches, the gateway captures its protocol exchange from the first byte.

Creating a profile

$ protect admin --context admin trace add --source-ip 127.0.0.1/32 --max-duration 1m
Trace Profile

qkI9g1V850igJiW2XeKvqG Profile:

            ID: qkI9g1V850igJiW2XeKvqG
      Max Time: 1m0s

   TCP Connection Properties:

     Source IP: 127.0.0.1/32

The profile ID is used to manage and delete the profile.

Matching criteria

Multiple criteria can be combined — all must match for a connection to be traced:

$ protect admin --context admin trace add --port-name clients --user admin --max-duration 5m --max-bytes 10000000

Limits

Always set at least one limit. Traces without limits will grow until the connection closes.

Listing profiles

$ protect admin --context admin trace list
Trace Profiles

qkI9g1V850igJiW2XeKvqG Profile:

            ID: qkI9g1V850igJiW2XeKvqG
      Max Time: 1m0s

   TCP Connection Properties:

     Source IP: 127.0.0.1/32

Deleting a profile

Removing a profile stops any active traces for that profile:

$ protect admin --context admin trace delete qkI9g1V850igJiW2XeKvqG
Deleted trace profile qkI9g1V850igJiW2XeKvqG

Existing connections

Profiles match already-connected clients, not just new connections. When a profile is created, all current connections are checked against it.

Trace file format

Trace files are written to the client_trace_dir configured in the audit section. Each file is a JSON lines file with three parts: a header, protocol messages, and a footer.

Header

The first line contains connection metadata:

{
  "version": 1,
  "device": "my_gateway",
  "ts": "2026-04-06T16:32:16.678623Z",
  "cuuid": "qkI9g1V850igJiW2XeKxbl",
  "port": "clients",
  "src": "127.0.0.1",
  "spr": 63127,
  "dst": "192.168.1.88",
  "dpt": 63129,
  "protocol": "client",
  "profile": { "uuid": "qkI9g1V850igJiW2XeKvqG" }
}

Protocol messages

Each subsequent line records one NATS protocol operation:

{"ts":"2026-04-06T16:32:16.678749Z","id":"...-1","dir":"backend","msg":"CONNECT","dat":"<base64>"}
{"ts":"2026-04-06T16:32:16.678778Z","id":"...-2","dir":"backend","msg":"PING","dat":"<base64>"}
{"ts":"2026-04-06T16:32:16.730411Z","id":"...-3","dir":"client","msg":"PONG","dat":"<base64>"}
{"ts":"2026-04-06T16:32:16.730433Z","id":"...-4","dir":"client","msg":"INFO","dat":"<base64>"}
{"ts":"2026-04-06T16:32:16.730858Z","id":"...-5","dir":"backend","msg":"PUB","dat":"<base64>"}
{"ts":"2026-04-06T16:32:16.772177Z","id":"...-8","dir":"client","msg":"DISCONNECT","dat":"<base64>"}

Protocol operations captured:

The dat field contains the exact bytes on the wire. Decoding the base64 reveals the raw NATS protocol line, e.g., a CONNECT message decodes to:

CONNECT {"verbose":false,"pedantic":false,"name":"NATS CLI","lang":"go","version":"1.48.0","protocol":1,"echo":true,"headers":true,"no_responders":true}

Footer

The last line contains trace statistics:

{ "ts": "2026-04-06T16:32:16.772185Z", "duration": 142991000 }

The duration is in nanoseconds (142ms in this example).

Trace file naming

Files are named with a timestamp, connection UUID, and CID:

20260406-163216_qkI9g1V850igJiW2XeKxbl_2.log