Establishing compatibility

“Compatible with Mumble” is worth exactly what backs it. This runtime shares no code with the Mumble project, so nothing about it is compatible by construction: every message it sends is one it decided to send, and the only proof that a real client accepts it is that a real client was asked.

An implementation also cannot certify itself. A test written from the same understanding that produced a defect agrees with the defect. What follows is what backs the claim here, and the last section states what it does not cover.

The protocol is not guessed

Protocol facts come from the Mumble project’s own sources, pinned at a single commit and kept in the repository next to the extracts they were taken from. The message schema is compiled from those files when the workspace builds, so the wire format is never transcribed by hand and cannot drift from its source through a typing mistake.

Every module asserting a protocol fact names the upstream file it came from. Those references are a convention rather than a machine check, and nothing verifies that one is still accurate. It is the weakest link in what follows.

The encryption layer is the one place upstream truth is executed rather than cited: it is checked against the published test vectors for the cipher it implements.

Real traffic, recorded

Seven sessions between an official Mumble client and a real reference server were captured and kept: a login, channel entry and exit, channel creation and removal, two people talking, a whisper, a refusal, a disconnection.

Replaying them proves three things at once. Every byte of every stream is accounted for, so nothing is quietly skipped as unrecognised. Every decoded message re-encodes to the bytes that were originally captured, so the codec reads real traffic without losing anything. And a capture from an older server, whose voice format is deliberately out of scope, is still required to be rejected rather than misparsed into something plausible.

The corpus grows slowly, because capturing a scenario means driving a graphical client by hand.

Real traffic, live

A proxy sits between a real client and a real reference server, decoding and re-encoding every message in flight and rewriting the key exchange so it holds an independent cipher domain toward each side.

The codec therefore meets live traffic generated by two implementations that know nothing about it. A message that survives the round trip unchanged has been read correctly, and neither endpoint had any way to be lenient about it.

A client that refuses to be lied to

A simulated Mumble client applies every message it receives to a strict model of what a client believes, and rejects any state the official client would reject. It is written against the specification and the upstream sources, never against this server, which is what makes its verdict worth anything.

It refuses a channel whose parent it cannot see, a participant placed in a channel it cannot see, a cycle in the tree, the removal of an occupied channel or of a parent before its children, the removal of the root, any message naming a channel or a session invisible to it, an identifier reused after being retired, and synchronisation arriving before the connection has been introduced to itself.

Each of those is a separate named check rather than one large validation pass, so deleting one is visible instead of being absorbed by another. A table-driven test feeds an invisible identifier into every field of every message that carries one, and requires each to be caught.

Catching the mistake that repairs itself

That same strict client is the right-hand side of an equality checked after every message a shard sends:

  what the client believes  ==  the scoped shared view  +  its private overlay

Two decisions inside that sentence carry the weight.

The model is fed encoded messages rather than the runtime’s internal plan, so the translation between the two is inside the property instead of being assumed correct.

The check runs after every message rather than at the end of a change. This is the point of the whole exercise: an ordering defect produces a state that is briefly wrong and finally right, so comparing only the endpoints sees nothing. A client that flickers, or that momentarily shows a channel it should never have seen, fails here and nowhere else.

The generator interleaves four kinds of change at random, and asserts its own coverage, because a generator that quietly stops exercising one of them is worse than none. Three further tests compose the pipeline wrongly on purpose and require the property to fail; if one ever passes, the generator has gone weak rather than the code correct.

Real sockets, and arbitrary bytes

The end-to-end suite opens real TLS connections and real datagrams against a server on a loopback port. Nothing is mocked and nothing sleeps: a test advances when the server has actually said something. It covers what a client experiences rather than what a function returns, including two clients hearing each other over UDP, a client served through the tunnel because it never opened UDP, a one-way listener, two connections given different trees, and a migration that does not remove a client from its own view.

Underneath, the decoders are required to return on arbitrary bytes, never to panic, and to fail closed rather than accept something they did not understand. Re-encoding a decoded message is required to reproduce the original bytes exactly, which is the property the live proxy depends on.

The judge is not the defendant

A change may not touch a judge and the implementation it judges in the same revision. Loosening a model, editing a captured session or weakening a check is legitimate only as its own reviewed change, never as a side effect of making something else pass.

What none of this proves

The list matters as much as the mechanisms.

Rules a client cannot observe are out of reach. A model built from what arrives on the wire judges only what arrives on the wire.

The strict model knows structure and identity: channels, participants, parents, names, sessions. It does not model permissions, positions, flags, comments or links, so a defect confined to those fields passes it.

Three deliberately wrong pipelines are three known kinds of defect, not a measure of coverage. They show the property is sensitive to those three and say nothing about the rest.

The corpus contains no encryption resynchronisation and no older client, which are the two cases most likely to behave unexpectedly.

Official Mumble clients were used throughout development, and that is a person watching rather than a machine checking. It catches what a person notices.

Running them

# everything below, and the rest
cargo test --workspace

cargo test -p mumble-server-runtime-corpus-decode          # the corpus
cargo test -p mumble-server-runtime-mitm-proxy             # the proxy, offline
cargo test -p mumble-server-runtime-testkit                # the simulated client
cargo test -p mumble-server-runtime-shard --test oracle    # the equality above
cargo test -p mumble-server-runtime-gateway --test gateway # real sockets
ci/fuzz-smoke.sh                                           # every fuzz target

The proxy is also a program. Pointing a real client at it, with a reference server upstream, is the observation the offline tests stand in for:

cargo run -p mumble-server-runtime-mitm-proxy -- \
    --listen 0.0.0.0:64738 --upstream 127.0.0.1:64739 --tls-name localhost