Safe Rust, Corrupt Wire

I have been writing network code for a long time, long enough that none of the ways a connection can betray you are surprising anymore. A timeout that lands mid-write. A peer left parsing a length prefix for bytes that never arrive. A stream that is structurally fine and semantically garbage. That is the job. What caught my attention while building Monocoque was not that any of this can happen. It was the specific shape Rust gives the problem, and what that shape says about where the language's guarantees actually stop.

A word on what Monocoque is, since the rest of this leans on it. It is a pure-Rust messaging runtime that speaks ZeroMQ's wire protocol, ZMTP 3.1. It is a from-scratch, native-Rust implementation of what libzmq does, so a Rust program can talk to ordinary ZeroMQ peers, using the familiar socket patterns, without linking the C library or going through an FFI binding. The source is at github.com/vorjdux/monocoque. It implements the ZeroMQ socket patterns natively, stays wire-compatible with existing libzmq peers, and is checked against real libzmq in its interop tests. The I/O runs on io_uring through compio, which keeps it completion-based and syscall-minimal and deliberately not tied to Tokio. Messages are handled zero-copy, a received frame becoming a reference-counted buffer that can fan out to many subscribers without a copy, and it supports the ZeroMQ security mechanisms, including CURVE. Throughout, every unsafe operation is confined to a single auditable module.

I built it to get rid of a C dependency. Using ZeroMQ from Rust normally means binding libzmq through FFI, which drags a C library, its build, and a foreign unsafe boundary into a system that is otherwise safe and async-native. I wanted that gone: a pure-Rust implementation that is memory-safe by construction, async-native, io_uring-backed, runtime-agnostic, and zero-copy, while staying compatible on the wire with the ZeroMQ peers already in the field, so existing systems keep working and the Rust side stops paying the FFI tax. It is young, a work in progress, tested and fuzzed but not battle-hardened, so read this as a design note from the middle of building it rather than a finished result.

The thread running through the whole project is one idea: Rust's real contribution is correctness by construction, and it applies that idea to exactly one thing.

Safety by construction, for memory

Start with the part Rust does extraordinarily well. io_uring forces an ownership question on you immediately, because when you submit a read the kernel takes your buffer and writes into it later, after the call has returned. If a future owns that buffer and is cancelled, which in async Rust means dropped, a naive design frees the buffer while the kernel still intends to write to it. The borrow checker cannot see that hazard, because the other party holding the memory is the kernel, which has no lifetime to reason about. The right response is not vigilance. It is structure.

So that is what I built, and the structure is the point. Monocoque uses compio, a completion-first runtime where you move a buffer into an operation and get it back on completion rather than lending it out. Every read lands in a pinned slab page, allocated once, never moved, reference counted, reachable only through the single type permitted to write unsafe in the whole codebase. When the read finishes, that slab is frozen, by value, into immutable Bytes sharing the same allocation, so a frame fans out to many subscribers with no copy and nothing can mutate it afterward. All the danger lives in one module. Everything above it is plain safe Rust. The buffer use-after-free is not a bug I have to remember to avoid. It is a program the types will not let me write.

This is Rust's actual idea, stated plainly. Not "be careful with memory," but "arrange things so the careless version does not compile." Correctness by construction, and it is one of the best ideas systems programming has had in decades.

The idea has a boundary, and cancellation sits on it

Here is where the long experience does its work, because what it buys you is the instinct to go looking for the invariants the compiler is not enforcing. In a stateful protocol there is an obvious one: a message is all or nothing on the wire.

A ZeroMQ message is frequently multipart, several frames sent in sequence, which is several awaits in a row. Now bring cancellation back. In most languages an interruption mid-write is something you reason about through control flow, an explicit branch, a cleanup label, a path you can see in the source. Rust's async cancellation is not like that. It is Drop, and it can happen at any await point, structurally and implicitly, with no line of code marking where. That property is a gift for memory, because Drop runs your destructors and the slab discipline holds no matter where you are cut. It is a hazard for protocol state, because Drop unwinds your side of the world and touches nothing on the other side. The frames already sent are gone. The peer is already mid-parse. Dropping the future cannot reach into the kernel's send queue and pull bytes back, and it cannot un-confuse the peer.

So the very mechanism that makes memory safe under cancellation makes protocol integrity fragile under it, and it does so quietly, because from the compiler's and the runtime's point of view nothing is wrong. Every buffer is owned and accounted for. The bug is one layer up, in what the bytes mean, which is exactly the layer Rust makes no promises about. None of this surprised me. I went looking for it, because I have been burned by its cousins for twenty-five years. What was worth thinking about is that Rust had quietly changed how the gap behaves. It had taken a control-flow problem, the kind you guard with explicit cleanup, and turned it into a structural one.

So the fix has to be structural too

A discipline does not survive contact with a real codebase. "Remember to handle cancellation on every multipart write" is the kind of rule that holds until the third contributor and the fortieth write path. If memory safety taught me one portable thing, it is that the durable answer is to make correctness the default and the mistake the thing you have to actively do.

So the guard arms itself. The moment a critical write begins, the connection is marked poisoned, and it stays poisoned unless the write runs all the way to completion and explicitly clears it. If the future is dropped anywhere in the middle, the flag is already set, the connection is condemned, and it gets torn down and reconnected rather than handed the next message on top of a half-sent one. There is no path you can forget, because forgetting is the safe direction. The mechanism is a few lines. The decision behind it is the whole thing: I took the one idea Rust applies to memory and applied it by hand to protocol state, because the language was never going to do it for me.

Two boundaries, and the work between them

That leaves a clean picture. The borrow checker reasons about memory inside the Rust abstract machine and stops at every edge past it: the kernel mid-syscall, hardware acting on a pointer later, another process across a shared mapping. The runtime keeps your buffers alive across a cancellation and says nothing about whether the message they were forming still means anything. Both are honest, and both draw their line exactly where their guarantee ends. The interesting engineering lives in the space past both lines, the region that is fully memory safe and still wrong, and the move that works there is the same one Rust makes for memory: stop relying on care, encode the invariant in something that cannot be skipped.

Where this is going

There is a broader version of this worth watching. Rust's async model, a future droppable at any await, came out of the readiness world of epoll, where cancellation genuinely is as cheap as not asking again. Completion I/O and stateful protocols break that assumption, because the work is already in flight, the peer has already seen things, and the drop rolls none of it back. The ecosystem is still working out what cancellation safety should mean above the memory layer. Owned-buffer APIs settled the memory half. Async Drop and structured cancellation are the open questions for the rest. The honest position is that correctness by construction is solved for memory and an active design problem for everything above it, which is one of the more interesting places to be working right now.

The frame worth keeping

What I would hand to someone starting a serious systems project in Rust is not a warning, it is a frame. Rust's safety is real, it is narrow, and the narrowness is deliberate: it tells you precisely which invariants it will enforce, which is the only way to know precisely which ones are still yours. Memory it handles by construction. Protocol, state, the atomicity of a message on the wire, the ordering your system promises, those it will protect only if you build the same kind of structure for them that the language built for memory. The borrow checker stops at memory. The runtime stops at the byte. Everything those bytes are meant to mean is engineering, and it always was.

Monocoque is early, so there are certainly more of these invariants I have not yet pinned down structurally. That is the actual work, and it is the good part. The tools made memory safe by construction and then handed back the more interesting half of the problem.