# Provision the streams and buckets

This guide creates the streams and buckets that your config file names, and checks them before you deploy. The gateway never creates, changes or deletes a stream or a bucket. You decide their limits.

Before you start, you need:

* Your config file.
* The `mq-gateway` binary.
* The `nats` CLI, [NACK](https://github.com/nats-io/nack) or the JetStream Terraform provider.
* A NATS user that can create streams in each account that the gateway serves.

## 1. Generate the resources

`generate` reads the config file and prints the streams and buckets in the format of your tool:

```
mq-gateway generate --config gateway.yaml --format nats-cli > provision.sh
```

For NACK manifests, use `--format nack`. For Terraform, use `--format terraform`. Names, subjects and domains come from the config file.

Each stream has deny delete and deny purge set. With these settings, the gateway acknowledges stored deliveries in batches. You cannot clear these settings after you create the stream.

## 2. Fill in the limits

Every limit is a placeholder, with the reason for it beside it. The script stops before it creates anything if a limit has no value.

For the `nats-cli` script, set each limit as an environment variable. This example is for a config file that binds the stream `ORDERS` and names the buckets `MQTT_RETAINED` and `MQTT_SESSIONS`, on one server:

```
export ORDERS_RETENTION=limits ORDERS_REPLICAS=1 ORDERS_MAX_BYTES=1GB ORDERS_MAX_AGE=24h ORDERS_DISCARD=old ORDERS_MAX_CONSUMERS=20000

export KV_MQTT_RETAINED_REPLICAS=1 KV_MQTT_RETAINED_MAX_BYTES=100MB KV_MQTT_RETAINED_TTL=0

export KV_MQTT_SESSIONS_REPLICAS=1 KV_MQTT_SESSIONS_MAX_BYTES=100MB KV_MQTT_SESSIONS_TTL=0
```

For NACK and Terraform, replace each `<REQUIRED: ...>` value in the file.

Use these values:

* **Retention:** `limits`.
* **Replicas:** `1` on one server. `3` in a cluster.
* **Sessions bucket TTL:** `0`. The gateway expires sessions itself.
* **`max_consumers`:** more than the number of MQTT sessions with a QoS 1 or 2 subscription on the stream, plus any other consumers. Each of these sessions holds one consumer. If you do not set a limit, nats-server 2.15 and later allow 1,000 consumers for each stream.
* **AMQP `delivery-counts` bucket TTL:** longer than a message waits in an AMQP queue.

## 3. Apply them

Run the script with a NATS context, or with `NATS_URL` and `NATS_CREDS` set for the account:

```
NATS_URL=nats://admin:secret@nats.example.com:4222 bash provision.sh
```

If the gateway serves several accounts that use the top-level bindings, run the script once for each account, with credentials for that account.

An account with its own bindings under `accounts` has its own section in the script. Set its `<ACCOUNT>_CONTEXT` variable to a `nats` context of a user in that account.

## 4. Check before you deploy

Run `check` with the gateway's config file. It connects with the gateway's credentials:

```
mq-gateway check --config gateway.yaml
```

`check` prints nothing and exits 0 when every check passes. If a check fails, it prints each failure and exits 1.

* `--verbose` prints every check.
* `--strict` also exits 1 on a warning.
* `--json` prints one JSON object for a script.

## Fix what `check` reports

| `check` says                                                                        | Fix                                                                                                                                |
| ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `stream "ORDERS": ... stream not found`                                             | Create the stream in that account and domain, or fix `bindings[].stream`.                                                          |
| `stream "ORDERS" does not capture subject "orders.>"`                               | Add the subject to the stream, or fix `bindings[].subjects`.                                                                       |
| `no JetStream API answers for domain "hub"`                                         | Fix `bindings[].domain`, or connect the domain to the gateway's NATS server.                                                       |
| `nats: bucket not found`                                                            | Create the bucket, or fix the bucket name in the config file.                                                                      |
| `the gateway user cannot watch it, and the gateway needs a watch to index its keys` | Give the gateway user the permissions from [Configure listeners and auth](/mq-gateway/0.4/how-to/configure-listeners-and-auth.md). |
| warning: `... sets no max_consumers` or `allows 1000 consumers`                     | Increase the limit: `nats stream edit ORDERS --max-consumers 20000`.                                                               |

## If the gateway starts first

A gateway can start before its streams and buckets exist, for example when it deploys with the job that creates them. The gateway then refuses that account's clients and logs `account's startup check failed, trying again`. When the streams and buckets exist, it serves the account with no restart.

A stream or bucket that exists but has incorrect settings is different. The gateway refuses that account's clients until you fix it and restart the gateway.

## Change a stream later

You can add subjects and increase limits at any time. Do not add a limit for each subject or allow message TTLs on a stream that has deny delete set. `check` fails such a stream.

## Next

[Configure listeners and auth](/mq-gateway/0.4/how-to/configure-listeners-and-auth.md): the ports that clients connect to, and the NATS permissions for each user.
