WebAssembly plugins

A plugin is a WebAssembly component that mercuriod loads from a file and runs in a sandbox. It can authenticate clients, decide topic access, change, drop or publish messages and react to events, without a rebuild of the broker.

What a plugin can do

A plugin implements the hooks it needs. init returns the list of hooks the broker calls; the others are never called.

The interface is wit/plugin.wit, version 0.2. A plugin also imports the host interface: log, a key-value store and publish.

Hooks get the client as a client-info: its client id; its user name, set only once the broker verified it, except in authenticate and auth-step during CONNECT, where it is the name the client claims; its address; and its connection, none where no connection is known, such as for the publisher of a retained message. The connection gives the listener's address or socket path (local for clients inside the broker), the MQTT version (3, 4 or 5), whether it is TLS, the name taken from the client's certificate on a listener that takes user names from certificates, and the listener's mount point. Topics reach plugins with the mount point in front.

How it runs

Plugins join the same chain as plugins written against the server library, after the built-in password and ACL checks and in the order of the [[plugins]] tables. The rules of the chain (the first allow or deny decides, defer asks the next plugin, every plugin sees a message in turn) are described in Embedding the broker.

• Time limits fail closed. A call that runs out of time or traps denies, for authentication and access control, and drops, for messages.
• A plugin runs as several copies, set by instances, so calls run in parallel. A hook is a plain function: it keeps no changing state in statics, because the next call may run in another copy. Settings are the exception: they are the same for every copy and never change, and each copy runs init before any other hook, so a copy may keep them in a static set in init, such as a Rust OnceLock.
• A copy that traps or runs out of time is discarded. A fresh copy takes its place and runs init again.

The key-value store

kv-get, kv-set, kv-delete and kv-list give copies a place for shared state.

• One store is shared by all copies of a plugin. It is in memory and is lost when the broker restarts. It stays across SIGHUP reloads.
• It holds at most 16 MiB. Each entry counts its key, its value and 64 bytes for the entry itself. Past that, kv-set returns an error.
• It is not transactional. A kv-get followed by a kv-set can race with another copy doing the same, so do not rely on a read-modify-write being atomic.

Publishing

publish sends a message with a topic, payload, QoS, retain flag and the MQTT 5 properties on-message sees. The plugin publishes as the client mercurio-plugin-<name>, inside the broker: access control is not asked, and the message is retained and sent on as any client's. The call returns once the message is queued. A message published before the broker serves clients, such as from init, goes out once it does.

• Topics that start with $, such as $SYS/... and $CONTROL/..., are refused, as are topic names that are not valid, such as ones with wildcards.
• A message is at most 64 KiB, counting its topic, payload and properties.
• A plugin publishes at most 100 messages at once, then 100 a second, all its copies together. The limit stays across SIGHUP reloads.
• At most 256 messages wait to go out.

A refused message returns an error and nothing is published; the call itself does not fail. The broker logs the first refusal of a run.

The other plugins' on-message see the message as from mercurio-plugin-<name>; the publishing plugin's own on-message does not, so a plugin that republishes what it sees never feeds itself. Two plugins that republish each other's messages still loop, at the pace their rate limits allow, so give such pairs topics that do not overlap. on-event sees the plugin's client connect, on its first message.

Writing a plugin in Rust

Install the target and make a library crate:

rustup target add wasm32-wasip2
cargo new --lib deny-topic

In Cargo.toml, build a cdylib:

[lib]
crate-type = ["cdylib"]

and add the latest release of mercurio-plugin, which holds the bindings and the Plugin trait:

cargo add mercurio-plugin

mercurio-plugin 0.1.x and 0.2.0 speak interface 0.1; the releases after 0.2.0 speak 0.2, which this page describes. Until a release after 0.2.0 is on crates.io, take the crate from the repository instead:

[dependencies]
mercurio-plugin = { git = "https://github.com/frenzox/mercurio" }

This plugin, examples/plugins/deny-topic, refuses publishing under a topic prefix from its configuration and defers every other decision:

use std::sync::OnceLock;

use mercurio_plugin::{
    export_plugin, Access, ClientInfo, Config, Decision, Hooks, Plugin, Registration,
};

/// Set by `init`; settings never change, so each copy keeps its own.
static PREFIX: OnceLock<String> = OnceLock::new();

struct DenyTopic;

impl Plugin for DenyTopic {
    fn init(config: Config) -> Result<Registration, String> {
        let prefix = config.require("prefix")?;
        if prefix.is_empty() {
            return Err("`prefix` is empty, which would deny every publish".into());
        }
        let _ = PREFIX.set(prefix.to_string());
        Ok(Registration {
            hooks: Hooks::AUTHORIZE,
            auth_methods: Vec::new(),
        })
    }

    fn authorize(_client: ClientInfo, topic: String, access: Access) -> Decision {
        let Some(prefix) = PREFIX.get() else {
            return Decision::Defer;
        };
        if access == Access::Write && topic.starts_with(prefix.as_str()) {
            Decision::Deny
        } else {
            Decision::Defer
        }
    }
}

export_plugin!(DenyTopic);

init gets the plugin's config table as a Config: get returns a setting if it is there, and require returns it or an error that names it. If init returns an error, the plugin does not load and the broker does not start. Combine flags to register more hooks, such as Hooks::AUTHORIZE | Hooks::ON_MESSAGE. The examples/plugins directory also has webhook-audit, which sends an HTTP request, and payload-rewrite, which changes messages.

Build it:

cargo build --release --target wasm32-wasip2

The component is target/wasm32-wasip2/release/deny_topic.wasm. Load it in the broker's configuration:

[[plugins]]
path = "/etc/mercurio/plugins/deny_topic.wasm"
name = "deny"
config = { prefix = "$internal/" }

webhook-audit sends plain HTTP, so its host needs the port in allow_http:

[[plugins]]
path = "/etc/mercurio/plugins/webhook_audit.wasm"
config = { url = "http://audit.internal/events" }
allow_http = ["audit.internal:80"]

A plugin built for mercurio:plugin/guest@0.2.x or @0.1.x loads. A component built for another version is refused, with an error that names the version. listPlugins in the admin API shows the version each plugin was built for. See the configuration reference for every key.

The broker matches a component to the interface by the shape of its types, so adding a hook, a field or a case would break plugins built for the old shape. Such changes come with a new minor version. A 0.2.x release changes only documentation, so a plugin built for any 0.2.x loads in a broker built for any other.

Plugins built for 0.1 keep loading and run as before: they get the client without its connection, have no authorize-control hook, so they defer on $CONTROL, send no data with an authentication success and no reason string with a failed, and cannot publish. To use those, rebuild against 0.2. In the Rust SDK, init takes a Config instead of a list of pairs, and AuthResult::Success takes a Success with the user name and the data.

The sandbox

• A plugin has no files, no environment variables and no standard input. Standard output and error are discarded; use host::log, which keeps the first 4 KiB of a message and escapes control characters such as newlines. A Rust plugin built with mercurio-plugin sends its panic messages to the broker's log, at error level.
• allow_http lists the hosts it may send HTTP requests to, as host or host:port. An entry without a port means port 443. The scheme is not checked, so http://host:443 is allowed too, and a plain-HTTP endpoint on port 80 needs host:80. Other hosts are refused.
• allow_sockets lists the address:port pairs it may connect TCP sockets to and exchange UDP datagrams with; a wildcard address or port 0 is refused at load. A socket may bind only to the wildcard address with port 0, which is what a connect needs, and may never listen or accept connections; a UDP socket drops datagrams from addresses not listed. DNS lookups are off, so use addresses. With an empty list the plugin can't create a socket at all.
• Each copy may hold at most 4096 host resources at once, such as sockets, streams and HTTP requests. A call that tries to hold more traps.
• memory_mb caps the linear memory of each copy, all its memories together. The default is 64.
• timeout_ms bounds the CPU time (counted in epochs) and the wall-clock time of each call and of init. It defaults to [server] plugin_timeout, 5 seconds unless set, the broker's own limit per hook call, and may not be longer.

A plugin that traps, runs out of memory or runs out of time fails that call as described above, and its copy is replaced. The broker logs the cause.

Other languages

Any toolchain that builds a WASI preview 2 component against wit/plugin.wit works. Starting points are wit-bindgen for C and C++, and componentize-py and jco componentize for Python and JavaScript. The project has examples only in Rust.

Reloading

Send SIGHUP to swap a plugin's code. The broker rereads each plugin's file, and replaces the running code once the new file has loaded and run init. Calls already running finish on the old code. If the new file fails to load, the old code keeps running and the broker logs why. The store is kept.

Changes to the [[plugins]] tables, such as config, instances or the allowlists, need a restart. A reload can change the interface version a plugin was built for.

Cost

A call into a component costs about 0.5 microseconds (0.55 in a microbenchmark of authorize). In the broker's benchmark, fan-in with QoS 1 had about 0.7% less throughput with an authorize plugin (deny-topic) on every message, and about 1.6% less with an on-message plugin. Fan-out was within noise. These numbers are approximate and depend on what the plugin does.

With the default wasm feature, the stripped mercuriod binary is about 18 MB. Build with --no-default-features to leave out wasmtime.