Call NATS services from MQTT
This guide sends a request from an MQTT v5 client to a NATS service. The client reads the reply on a topic that no other client can read.
The client asks the gateway for a reply prefix when it connects. The
gateway gives each connection its own prefix, such as
$MQG/reply/K7R2Q.../, and refuses every other client a subscription
under it. Replies under the prefix travel on core NATS, so they arrive at
QoS 0 and only while the client is connected.
Before you start, you need:
- A gateway with an MQTT listener and the client permissions from Configure listeners and auth.
- A
nats-server.confthat you can edit and reload. - The
natsCLI. - Python 3 with
paho-mqtt2.0 or later.
1. Give the service a NATS user
Add a user for the service to the account of your MQTT clients:
{ user: echo, password: "echo-secret", permissions: {
subscribe: ["svc.echo"]
allow_responses: true
} }
svc.echois the subject that the service answers on.allow_responses: truelets the service send one reply to each request it gets, on the reply subject that the request names.
The client users need:
- Publish permission on
svc.echo. Add it when you narrow the client topics. - Subscribe permission on
$MQG.reply.>.$client_permissionsfrommq-gateway generatehas it.
Reload the server:
nats-server --signal reload
2. Start the service
nats --user echo --password echo-secret reply svc.echo 'pong'
The service answers each request on svc.echo with pong.
3. Send the request from MQTT
Save this as request.py. It sends three requests on one connection:
import queue
import threading
import uuid
import paho.mqtt.client as mqtt
from paho.mqtt.packettypes import PacketTypes
from paho.mqtt.properties import Properties
prefix = None
subscribed = threading.Event()
pending = {}
def on_connect(client, userdata, flags, reason_code, properties):
global prefix
if reason_code.is_failure:
return
prefix = properties.ResponseInformation
client.subscribe(prefix + "#")
def on_subscribe(client, userdata, mid, reason_codes, properties):
subscribed.set()
def on_disconnect(client, userdata, flags, reason_code, properties):
subscribed.clear()
def on_message(client, userdata, msg):
reply = pending.pop(msg.topic, None)
if reply is not None:
reply.put(msg.payload)
def request(topic, payload, timeout=5):
if not subscribed.wait(timeout):
raise TimeoutError("not connected")
reply_topic = prefix + uuid.uuid4().hex
reply = pending[reply_topic] = queue.Queue(1)
props = Properties(PacketTypes.PUBLISH)
props.ResponseTopic = reply_topic
client.publish(topic, payload, properties=props)
try:
return reply.get(timeout=timeout)
except queue.Empty:
raise TimeoutError("no reply") from None
finally:
pending.pop(reply_topic, None)
connect = Properties(PacketTypes.CONNECT)
connect.RequestResponseInformation = 1
client = mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2, client_id="requester-1", protocol=mqtt.MQTTv5
)
client.username_pw_set("device", "device-secret")
client.on_connect = on_connect
client.on_subscribe = on_subscribe
client.on_disconnect = on_disconnect
client.on_message = on_message
client.connect("127.0.0.1", 1883, properties=connect)
client.loop_start()
for i in range(3):
print(request("svc/echo", f"ping {i}").decode())
client.disconnect()
client.loop_stop()
RequestResponseInformation = 1asks the gateway for the prefix. The CONNECT reply carries it asResponseInformation.client_idlets paho reconnect. paho resumes the session when it reconnects, and the gateway refuses that with an empty client ID.on_connectsubscribes to every reply under the prefix. It runs again after each reconnect, with the new prefix.subscribedholds back each request until the gateway acknowledges the subscription. A reply sent before that is lost.request()sends one request and waits for its reply. It can run at any time while the client is connected.ResponseTopictells the service where to reply: a new topic under the prefix for each request.pendingmaps each reply topic to the request that waits for it. The script ignores a message on any other topic under the prefix.timeoutlimits the wait.request()raisesTimeoutErrorwhen the client is not connected or no reply arrives in time.
The topic svc/echo is the subject svc.echo, so the request reaches the
service.
Check it works
Run the script:
python3 request.py
The script prints one pong for each request:
pong
pong
pong
Things to know
- The prefix lasts as long as the connection. Subscribe under the new prefix after each connect. A persistent session keeps no subscription under an earlier prefix. A request that waits for its reply when the connection drops gets no reply. Send it again after the reconnect.
- Only the client that got the prefix can read it. The gateway refuses
any other MQTT client a subscription under it, or a subscription with a
wildcard over
$MQG/reply/. A subscription to#or+/...gets no replies, because these filters do not match$topics. The gateway also refuses AMQP receivers on$MQG.reply.>, and a receiver on>gets no replies. - Do not give a service publish permission on
$MQG.reply.>. With it, the service's NATS user can publish under any client's prefix while that client is connected.allow_responseslimits the user to one reply for each request it gets. - NATS clients are not held to this. A NATS user with subscribe
permission on
$MQG.reply.>reads every reply. Give that permission only to the users of gateway clients. - The service must copy
Mqg-Correlation-Idto the reply. The gateway keeps no record of a request. A client that sends Correlation Data gets it back only when the service copies this header from the request to its reply. See Mapping to NATS. - Replies arrive at QoS 0. No binding can store subjects under
$MQG.>. A reply sent while the client is disconnected is lost. - Topics under
_INBOX/are not reply topics. The gateway refuses an MQTT subscription there. Use the prefix instead.