An MQTT broker you can reason about.

One thread, memory sized at start, every limit a flag, every refusal a named rule with an exact count, and an authority report the compiler proves: this program cannot touch a file, dial out or call foreign code.

alpha · for a trusted network MQTT 3.1.1, QoS 0, 1, 2 written in cancho EUPL-1.2

Download the binary Try it Source on GitHub

One complete flow

A refusal with the command that fixes it, the broker, a retained reading, a $SYS statistic, and what the compiler proved the program can reach. This is the built program's own output; a check in CI fails if this page drifts from it.

# A bound below its smallest value is refused, with the command that fixes it
$ mqtt serve --port 1883 --max-packet 10
{"type":"error","error":{"code":"INVALID_ARGS","rule":"args.out-of-range","message":"a flag's value is outside the range this broker accepts","hint":"use a value within the range introspect names for the flag","repair":{"kind":"retry","argv":["mqtt","serve","--port","1883","--max-packet","64"]},"detail":{"flag":"max-packet","value":10,"minimum":64,"ceiling":262144}}}
# exit status 2

# Run it; a client keeps a retained reading; another reads it back; a statistic
$ mqtt serve --port 1883 > broker.log &
$ head -1 broker.log
{"type":"listening","port":1883,"max_connections":1024,"max_packet":16384,"memory_bytes":73099264,"t_ms":0}
$ mosquitto_pub -p 1883 -t sensors/door -m open -r -q 1
$ mosquitto_sub -p 1883 -t 'sensors/#' -v -q 1 -C 1
sensors/door open
$ mosquitto_sub -p 1883 -t '$SYS/broker/clients/connected' -v -C 1
$SYS/broker/clients/connected 1
$ kill -TERM %1; jq -c 'select(.type=="end") | {ok, complete, publishes, delivered}' broker.log
{"ok":true,"complete":true,"publishes":1,"delivered":2}

# What the compiler proved this program can reach
$ mqtt introspect | jq -c '.authority | {bounded, effects}'
{"bounded":true,"effects":["args","clock","conn_accept","conn_read","conn_write","err_write","heap","io_read","io_write","net_in","poll","signals","signals_read"]}

What is in it

We try to break it

Claims are measured, and a claim found false is corrected where it was made. Behind this page:

• 184 black-box tests that read only what a client sees, and 31 unit tests of the codec, the topic trie, the flag table and the password hash.
• A differential against Mosquitto 2.0.18. The same scripted and generated scenarios run against both brokers and what each client sees is compared, including QoS 2, MQTT 3.1 clients and the $SYS counters. The differences that remain are written down and asserted, so one nobody wrote down fails.
• Hostile input. Thousands of malformed, split and oversized connections, slowloris, tight bounds; no input reaches a panic.
• Memory flat under churn, asserted, with transparent huge pages off (they make a lazily touched table grow in 2 MiB steps).
• The authority gate must be able to fail. Five mutants (a file read, a foreign call, a ceiling missing a label, one missing the new standard-input label, a stale embedded report) are applied to a copy and each must be refused.
• Real clients: paho-mqtt and the Mosquitto command line tools.

How fast

Supporting evidence, not the point. One core each, one shared 4-core VM, defaults for every broker. Read the caveats first. The QoS 0 figure for this broker is a lower bound: it used about two thirds of its core, so the load generator was the limit. Its QoS 2 figure counts deliveries, and this broker drops the newest message under overload where others slow the publishers, so only about 13% of what was offered arrived.

Deliveries a second at the subscribers, 4 publishers and 100 subscribers going as fast as they can, median of three or five runs:

Latency with one message in flight: p50 42 microseconds (Mosquitto 46; the probe is Python, so a few microseconds either way is not evidence). Memory with 5,000 idle connections: 14 MiB (Mosquitto 38). The conditions, what was not measured, and the results that are not explained are in docs/benchmark.md.

Where it would earn its place

Four scenarios, each run against the built program; scripts/site.py --check fails CI if the output drifts. They are illustrations of the properties above, not reports of deployments: nobody runs this in production yet.

1. Plant-floor gateways and dashboards: last value and presence

Operators want the state of every line (or charger, or pump) at a glance, including the ones that just lost power. Retained messages hold the last reading, a will turns a dead gateway into an offline message, and a dashboard that connects later needs no polling.

# A gateway on the plant network keeps its last reading and its state where anyone can read them
$ mosquitto_pub -p 1883 -t plant/line1/temp -m 71.5 -r -q 1
$ mosquitto_pub -p 1883 -t plant/line1/status -m online -r -q 1

# It registered a will; the power is cut (SIGKILL, no DISCONNECT) and the broker publishes the will
$ mosquitto_sub -p 1883 -i gw-line1 -t gw/in --will-topic plant/line1/status --will-payload offline --will-retain -q 1 &
$ kill -9 $!

# A dashboard that connects afterwards sees the last state of everything
$ mosquitto_sub -p 1883 -t 'plant/#' -v -C 2 | sort
plant/line1/status offline
plant/line1/temp 71.5

2. Test and staging brokers that explain themselves

A broker for integration tests should fail loudly and say why. Every refusal is a named rule with a log record a test can assert on, so "the client was disconnected" becomes "limit.packet-size, client auto-1".

# A test harness sets tight limits and asserts on what the broker refuses, by rule
$ mqtt serve --port 1883 --max-packet 256 > broker.log &
$ mosquitto_pub -p 1883 -t sensors/big -m "$(head -c 1000 /dev/zero | tr '\0' x)"
$ jq -c 'select(.type=="refusal") | {rule, client_id}' broker.log
{"rule":"limit.packet-size","client_id":"auto-1"}

# and what the broker does for that rule is data too
$ mqtt rules | jq -c 'select(.tag=="limit.packet-size")'
{"type":"rule","tag":"limit.packet-size","action":"close"}

3. Small fixed boxes: size it before you deploy it

On an edge gateway or a shared host the question is how much memory the broker will take. The tables are allocated at start from the flags, so the footprint is a number printed on the first log line, not a function of the workload.

# Know the memory before it accepts a connection: it is computed from the flags and printed at start
$ mqtt serve --port 1883 --max-connections 1024 | head -1
{"type":"listening","port":1883,"max_connections":1024,"max_packet":16384,"memory_bytes":73099264,"t_ms":0}
$ mqtt serve --port 1883 --max-connections 8192 --queue-bytes 65536 | head -1
{"type":"listening","port":1883,"max_connections":8192,"max_packet":16384,"memory_bytes":713771008,"t_ms":0}

4. A shared network: only known gateways connect

Authentication is optional and off by default. With --auth stdin the broker reads a table of users once from standard input, so it still reads no file and the authority report gains one label, io_read. A refused login is a named rule and the log never holds a user name or a password. Passwords and keys cross plain TCP, so this is for a network you trust or behind something that does TLS.

# On a shared network only known gateways may connect: a table made by the script, read once from standard input
$ printf 'correct horse' | python3 scripts/passwd.py gateway-1 >> users.txt
$ mqtt serve --port 1883 --auth stdin < users.txt > broker.log &
# no credentials
$ mosquitto_pub -p 1883 -t plant/line1/temp -m 71.5
Connection error: Connection Refused: not authorised.
# the wrong password
$ mosquitto_pub -p 1883 -t plant/line1/temp -m 71.5 -u gateway-1 -P wrong
Connection error: Connection Refused: bad user name or password.
# the right one
$ mosquitto_pub -p 1883 -t plant/line1/temp -m 71.5 -u gateway-1 -P 'correct horse'
accepted

# Each refusal is a named rule, with no user name or password in the log
$ jq -c 'select(.type=="refusal") | {rule}' broker.log
{"rule":"auth.required"}
{"rule":"auth.bad-credentials"}

Where it fits

What it cannot do yet

• No TLS; authentication is optional. With --auth stdin users and keys are checked, but they cross plain TCP, and there is no per-topic authorisation, no reload without a restart and no client certificates. cancho now has a TLS 1.3 server (not independently reviewed); this broker has no transport layer for it yet.
• Nothing survives a restart: sessions, queues and retained messages are in memory.
• A log reader that stops reading stalls the broker. The log is written to standard output with a blocking write. Measured: 13 minutes to fill a 64 KiB pipe idle, 36 seconds under a flood of refusals. Run it under something that always reads. A test pins this.
• $SYS has no load/*, heap/* or store/* topics.
• No soak yet, no run of days.

What each missing piece would cost, in what order, and what it would do to the authority report: docs/later.md.

Download

mqtt-linux-x86_64 checksum, authority report and release notes

The file is the binary the CI gate built and tested on the commit it names, from the latest push to main: the same formatting, unit, conformance, interop and differential checks, and the authority report it was compared with. It is a dynamically linked x86-64 Linux executable built on Ubuntu, so it needs a glibc at least that new; there are no builds for other systems yet. Verify it with sha256sum -c mqtt-linux-x86_64.sha256, then chmod +x mqtt-linux-x86_64 && ./mqtt-linux-x86_64 introspect.

Build it yourself

You need git, Rust, gcc and Python 3; the mosquitto-clients package to talk to it.

git clone https://github.com/alpibrusl/cancho                         # the compiler
git clone https://github.com/alpibrusl/cancho-mqtt && cd cancho-mqtt
REV=$(sed -n 's/^cancho *= *"\(.*\)".*/\1/p' cancho.toml)             # the compiler these sources need
(cd ../cancho && git fetch -q origin && git checkout "$REV" && cargo build --release -p cancho)
export PATH=$PWD/../cancho/target/release:$PATH
cancho build                                                          # builds build/mqtt
build/mqtt serve --port 1883 > broker.log &
mosquitto_sub -p 1883 -t 'sensors/#' -v &
mosquitto_pub -p 1883 -t sensors/door -m open