Administration

Look at and change a running broker: its clients and sessions, bans, retained messages and subscriptions, its log level and files, without restarting it.

The broker takes JSON commands on the MQTT topic $CONTROL/broker/v1, from any MQTT 5 client that access control allows, and from anyone on its host who may use its admin socket. mercurioctl runs each of these commands from the command line. This page describes the commands, who may run them, and what changes for setups upgrading from earlier versions.

The admin socket

[admin]
socket = "/run/mercurio/admin.sock"   # "" turns it off
group = "mercurio"

mercuriod serves this Unix socket by default. Clients connecting through it skip authentication and access control: they may run every command, dynamic security's included. So the broker checks who connects, from the user and group the kernel reports for the socket, and closes everyone else at once:

• root and the user the broker runs as;
• members of group, and users whose primary group it is.

The socket file belongs to the broker's user and to group, with mode 0660. With group = "", or a group that doesn't exist, it is 0600 and only root and the broker's user get in.

The packages run the broker as the mercurio user, and its systemd unit creates /run/mercurio, so the defaults work as installed. To let a user administer the broker:

sudo usermod -aG mercurio alice
sudo systemctl restart mercuriod

The broker reads the group's members when it starts, so a new member needs a restart of the broker, and a new login before their own processes carry the group. Changes to [admin] also need a restart.

With another socket path, every directory on it must be writable only by root or the broker's user: anyone who can write to one of them could put their own socket in its place. The broker refuses a socket whose directory belongs to another user or is writable by its group or by others; the directories above it are up to you. A broker that can't create its socket, such as one started by hand where /run/mercurio doesn't exist, logs a warning and runs without it. A socket file left by a broker that stopped without removing it is replaced; one where another process still accepts connections, such as a second broker started with the same configuration, stops the new broker with an error instead. For a broker run as yourself:

mkdir -m 700 -p "$XDG_RUNTIME_DIR/mercurio"

[admin]
socket = "/run/user/1000/mercurio/admin.sock"

The admin socket needs Linux; elsewhere [admin] is ignored with a warning.

Over the network

Any MQTT client may send commands once access control allows it. Like every $CONTROL topic, $CONTROL/broker/v1 is refused unless a rule grants it. Each command is also checked on its own, as a publish to $CONTROL/broker/v1/<command>: a client granted $CONTROL/broker/v1/listClients may list clients and nothing else. The client must also be allowed to subscribe to, and receive, the topic its answers come on.

With an ACL file:

# Every command
user ops
topic readwrite $CONTROL/broker/v1/#

# Only listing and showing clients
user monitor
topic write $CONTROL/broker/v1
topic write $CONTROL/broker/v1/listClients
topic write $CONTROL/broker/v1/getClient
topic read $CONTROL/broker/v1/response/#

write alone isn't enough: without read, subscribing to the answers is refused.

With dynamic security, mercuriod dynsec-init gives its admin role publishClientSend, subscribePattern and publishClientReceive on $CONTROL/broker/v1/#; give other roles the same ACLs, or narrower ones. With HTTP authentication, acl_url is asked about $CONTROL/broker/v1 and $CONTROL/broker/v1/<command> with acc 8, and about the answer topic with 4 and 1; a user its superuser_url names a superuser is allowed all of these without asking. A plugin that allows every topic grants none of this: publishing to $CONTROL is a separate question, which only an explicit grant answers yes to.

Bridges and other in-process connections run every command without checks, as the admin socket's clients do.

The protocol

A request is a JSON object with a list of commands, published to $CONTROL/broker/v1. The commands run in order, and the answer holds one response for each, in the same order:

mosquitto_rr -V mqttv5 -u ops -P "$OPS_PASSWORD" \
    -t '$CONTROL/broker/v1' -e '$CONTROL/broker/v1/response/ops' \
    -m '{"commands": [
          {"command": "getLogLevel", "correlationData": "1"},
          {"command": "listBans", "count": 2},
          {"command": "nope"}
        ]}'

{"responses":[{"command":"getLogLevel","correlationData":"1","data":{"level":"info"}},
 {"command":"listBans","data":{"bans":[],"totalCount":0}},
 {"command":"nope","error":"unknown command `nope`","code":"invalidArgument"}]}

• A command that succeeds may return data. One that fails returns error, a message for people, and code, for scripts. A command's correlationData comes back in its response.
• The answer goes only to the client that sent the request: on the request's MQTT 5 Response Topic, with its Correlation Data, or on $CONTROL/broker/v1/response for a request without one, such as an MQTT 3.1.1 client's. Either way the client receives it only while subscribed to that topic, so one client must subscribe and publish. mosquitto_rr does; separate mosquitto_sub and mosquitto_pub commands never see an answer.
• A payload that isn't a list of commands gets one response, with an empty command and invalidArgument. So does a request with more than 100 commands, none of which run.
• The broker logs each command that changes something, such as addBan or reload, at info, with the client id and verified user name of the client that sent it, or the admin socket or in-process client that did.
• An answer larger than the client's Maximum Packet Size is dropped. Page through long lists instead of asking for everything in one message.
• v1 only gains commands, optional arguments and response fields. A change that would break scripts goes to a v2.

Errors

A client that may not publish to $CONTROL/broker/v1 at all gets no answer; at QoS 1, its PUBACK says Not Authorized.

Lists

List commands take offset, 0 by default, and count, 100 by default and at most 1000. Beside the items they return totalCount, the length of the whole list. The rest of a list of 250 clients is {"command": "listClients", "offset": 100}, then "offset": 200.

Commands

Arguments go in the command object beside command; those marked ? are optional, with their default in parentheses. A client is any session, connected or not. Times are Unix seconds, and durations seconds. Dynamic security has its own commands, on $CONTROL/dynamic-security/v1.

Clients and sessions

Each mount point has its own client ids, so one id can name a session on each. getClient, disconnectClient and deleteSession take the client's mountpoint, "" for a listener without one. Without it they act on the one mount point the id is on, and fail with invalidArgument, listing the mount points, when it is on more than one:

{"command": "deleteSession", "clientid": "sensor-7", "mountpoint": "tenant-a/"}

Bans

Retained messages

Subscriptions

The broker

mercuriod serves the following ones; a broker embedded without them answers unavailable.

Bans

A ban refuses clients by client id, user name or address, until it expires or is removed:

{"command": "addBan", "type": "address", "value": "203.0.113.0/24", "duration": 3600, "reason": "flooding"}

• Connected clients that match are disconnected at once with Administrative Action (0x98), as MQTT allows Banned only in CONNACK, without publishing their will. New connections that match are refused at CONNECT with Banned, or Not Authorized for MQTT 3.x clients.
• clientid and username bans match the whole value, or with prefix every value starting with it. A user name ban matches the user the broker verified, so never an anonymous client. A client id ban matches the id the broker assigned to clients that sent none.
• address bans take an IP address or CIDR range and match the client's address, or behind a load balancer the one its PROXY header gives. An IPv6 range never matches IPv4 clients, ::/0 included: ban 0.0.0.0/0 as well to refuse everyone. IPv4 clients on an IPv6 listener are matched as IPv4.
• Adding a ban with the type, value and prefix of an existing one replaces its duration and reason.
• A ban leaves stored sessions alone. A will already waiting for a client that disconnected before the ban, delayed by its Will Delay Interval or restored after a restart, is still published; delete the session with deleteSession to drop it.
• With SQLite or PostgreSQL storage, bans survive restarts, and a standby broker loads them when it takes over. In memory, they last until the broker stops.

Upgrading from mercuriod 0.4.5 or earlier

• Messages of a shared subscription's member go to another member when its session ends. Earlier versions dropped what was still queued for a member, or sent to it at QoS 1 and not acknowledged, with its session. Now those messages go to another member of the group, so a member that processed a QoS 1 message without acknowledging it before leaving makes the application see it twice; see shared_subscription_redelivery in the configuration reference. SQLite and PostgreSQL databases get a column recording the shared subscription of queued and in-flight messages when the broker first opens them; messages stored before have none and stay with their member.
• Each mount point has its own client ids. Earlier versions refused a client the id another listener's client used while that session existed; now the same id on listeners with different mount points makes two independent sessions. SQLite and PostgreSQL databases are migrated when the broker first opens them: each stored session, with its subscriptions, queue, in-flight messages and will, moves under the mount point of the listener its client last connected to, so it resumes as before. The migration rewrites the session tables once, which takes a while on a large database. Scripts that run getClient, disconnectClient or deleteSession, or mercurioctl client show, disconnect or delete-session, must name the mountpoint, or --mount-point, once an id is on more than one mount point; see mount points. Without mount points nothing changes.
• Sessions end a week after their client left, unless configured otherwise. max_session_expiry now defaults to 604800 seconds; earlier versions kept a session as long as the client asked, forever for MQTT 3.x clients without clean session and MQTT 5 clients asking for no expiry. Such sessions, those restored from storage included, now expire a week after their client disconnects, with their subscriptions and queued messages. To keep the old behaviour, set max_session_expiry = "unlimited" in [server], or a longer bound.
• Dynamic security checks roles in mosquitto's order. Earlier versions took the matching ACL of highest priority across all of a client's roles. Now the client's own roles are checked first, then its groups' roles, group by group, each list by its priority, and the first role with a matching ACL decides; a role's ACL priorities order only that role's ACLs. A client whose roles have ACLs on the same topics may get a different answer: for example, a deny in its own role now wins over an allow of higher ACL priority in one of its groups. Existing files load unchanged, every role and group with priority -1. To keep an old decision, move the ACLs that must win into the role checked first, or give roles and groups priorities with mercurioctl dynsec add-client-role, add-group-role and add-group-client, which now take one. See Dynamic security.
• An HTTP ACL endpoint is asked about publishing to $CONTROL/... with acc 8, no longer 2. Answer 2xx to it only for administrators. WebAssembly plugins can no longer grant it, and Rust plugins grant it from Plugin::authorize_control, no longer from authorize.
• A configuration with a key the broker does not know no longer starts. Earlier versions ignored such keys, so a misspelling such as enable = true under [auth] or [server.tls] left authentication or TLS off without a word. The error names the key; fix or remove it. This applies to every table, the files of include_dir included, and to the keys of a [storage] backend other than the one chosen, such as path with backend = "memory".
• password_file, dynamic_security or user_url with enabled = false in [auth] or [listeners.auth] no longer starts. Earlier versions ignored the password file and let every client in, and loaded dynamic security only for access control. Set enabled = true to authenticate clients, or remove the key. acl_file and acl_url alone still work without authentication.
• A password file with a plain-text password no longer loads. Earlier versions compared a hash field that did not start with $ with the password as it was, so a line cut short to alice: let alice in with an empty password. Now every line needs an Argon2 hash, as mercuriod passwd writes, or a $7$ or $6$ hash of mosquitto_passwd; the error names the line. Set the password again with mercuriod passwd -f FILE -u USER, which still reads such a file.

Upgrading from mercuriod 0.4.3 or earlier

• Answers to $CONTROL requests go only to the client that sent the request, for dynamic security and plugins as well. A client subscribed to $CONTROL/dynamic-security/v1/response no longer sees the answers to other clients' commands, so a script that subscribes with one client and publishes with another, such as recado sub beside recado pub, gets nothing back. Install mercurioctl and use it, or one client that subscribes and then publishes, as mosquitto_rr does.
• Dynamic security files created by earlier versions give the administrator role $CONTROL/dynamic-security/# and $SYS/# only. Dynamic security commands keep working from anywhere, and every command works through the admin socket. To run the broker's commands over the network too, add the ACLs mercuriod dynsec-init now grants, for example through the admin socket as root (a user newly added to the socket's group needs a new login and a broker restart first):

  for acltype in publishClientSend subscribePattern publishClientReceive; do
      sudo mercurioctl dynsec add-role-acl admin "$acltype" '$CONTROL/broker/v1/#' allow
  done

• ACL files need topic readwrite $CONTROL/broker/v1/#, or narrower grants, for the users who administer the broker over the network.
• The admin socket is on by default, at /run/mercurio/admin.sock, for root, the broker's user and the mercurio group. socket = "" under [admin] turns it off.
• Logging: with connection_messages = false, a level naming mercurio::connections now turns connection messages on, and an empty level means info. See Logging.