Skip to main content
Version: 0.4

Call NATS services from MQTT

This guide sends a request from an MQTT v5 client to a NATS service. The client reads the reply on a topic that no other client can read.

The client asks the gateway for a reply prefix when it connects. The gateway gives each connection its own prefix, such as $MQG/reply/K7R2Q.../, and refuses every other client a subscription under it. Replies under the prefix travel on core NATS, so they arrive at QoS 0 and only while the client is connected.

Before you start, you need:

  • A gateway with an MQTT listener and the client permissions from Configure listeners and auth.
  • A nats-server.conf that you can edit and reload.
  • The nats CLI.
  • Python 3 with paho-mqtt 2.0 or later.

1. Give the service a NATS user​

Add a user for the service to the account of your MQTT clients:

{ user: echo, password: "echo-secret", permissions: {
subscribe: ["svc.echo"]
allow_responses: true
} }
  • svc.echo is the subject that the service answers on.
  • allow_responses: true lets the service send one reply to each request it gets, on the reply subject that the request names.

The client users need:

  • Publish permission on svc.echo. Add it when you narrow the client topics.
  • Subscribe permission on $MQG.reply.>. $client_permissions from mq-gateway generate has it.

Reload the server:

nats-server --signal reload

2. Start the service​

nats --user echo --password echo-secret reply svc.echo 'pong'

The service answers each request on svc.echo with pong.

3. Send the request from MQTT​

Save this as request.py. It sends three requests on one connection:

import queue
import threading
import uuid

import paho.mqtt.client as mqtt
from paho.mqtt.packettypes import PacketTypes
from paho.mqtt.properties import Properties

prefix = None
subscribed = threading.Event()
pending = {}

def on_connect(client, userdata, flags, reason_code, properties):
global prefix
if reason_code.is_failure:
return
prefix = properties.ResponseInformation
client.subscribe(prefix + "#")

def on_subscribe(client, userdata, mid, reason_codes, properties):
subscribed.set()

def on_disconnect(client, userdata, flags, reason_code, properties):
subscribed.clear()

def on_message(client, userdata, msg):
reply = pending.pop(msg.topic, None)
if reply is not None:
reply.put(msg.payload)

def request(topic, payload, timeout=5):
if not subscribed.wait(timeout):
raise TimeoutError("not connected")
reply_topic = prefix + uuid.uuid4().hex
reply = pending[reply_topic] = queue.Queue(1)
props = Properties(PacketTypes.PUBLISH)
props.ResponseTopic = reply_topic
client.publish(topic, payload, properties=props)
try:
return reply.get(timeout=timeout)
except queue.Empty:
raise TimeoutError("no reply") from None
finally:
pending.pop(reply_topic, None)

connect = Properties(PacketTypes.CONNECT)
connect.RequestResponseInformation = 1
client = mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2, client_id="requester-1", protocol=mqtt.MQTTv5
)
client.username_pw_set("device", "device-secret")
client.on_connect = on_connect
client.on_subscribe = on_subscribe
client.on_disconnect = on_disconnect
client.on_message = on_message
client.connect("127.0.0.1", 1883, properties=connect)
client.loop_start()

for i in range(3):
print(request("svc/echo", f"ping {i}").decode())

client.disconnect()
client.loop_stop()
  • RequestResponseInformation = 1 asks the gateway for the prefix. The CONNECT reply carries it as ResponseInformation.
  • client_id lets paho reconnect. paho resumes the session when it reconnects, and the gateway refuses that with an empty client ID.
  • on_connect subscribes to every reply under the prefix. It runs again after each reconnect, with the new prefix.
  • subscribed holds back each request until the gateway acknowledges the subscription. A reply sent before that is lost.
  • request() sends one request and waits for its reply. It can run at any time while the client is connected.
  • ResponseTopic tells the service where to reply: a new topic under the prefix for each request.
  • pending maps each reply topic to the request that waits for it. The script ignores a message on any other topic under the prefix.
  • timeout limits the wait. request() raises TimeoutError when the client is not connected or no reply arrives in time.

The topic svc/echo is the subject svc.echo, so the request reaches the service.

Check it works​

Run the script:

python3 request.py

The script prints one pong for each request:

pong
pong
pong

Things to know​

  • The prefix lasts as long as the connection. Subscribe under the new prefix after each connect. A persistent session keeps no subscription under an earlier prefix. A request that waits for its reply when the connection drops gets no reply. Send it again after the reconnect.
  • Only the client that got the prefix can read it. The gateway refuses any other MQTT client a subscription under it, or a subscription with a wildcard over $MQG/reply/. A subscription to # or +/... gets no replies, because these filters do not match $ topics. The gateway also refuses AMQP receivers on $MQG.reply.>, and a receiver on > gets no replies.
  • Do not give a service publish permission on $MQG.reply.>. With it, the service's NATS user can publish under any client's prefix while that client is connected. allow_responses limits the user to one reply for each request it gets.
  • NATS clients are not held to this. A NATS user with subscribe permission on $MQG.reply.> reads every reply. Give that permission only to the users of gateway clients.
  • The service must copy Mqg-Correlation-Id to the reply. The gateway keeps no record of a request. A client that sends Correlation Data gets it back only when the service copies this header from the request to its reply. See Mapping to NATS.
  • Replies arrive at QoS 0. No binding can store subjects under $MQG.>. A reply sent while the client is disconnected is lost.
  • Topics under _INBOX/ are not reply topics. The gateway refuses an MQTT subscription there. Use the prefix instead.