Skip to content
Select theme

Speaking the RESP protocol

Step 2 of 10, and it starts from nothing. Step 1 drew the map and committed no code, so the project begins here: cargo init --name mini-redis inside the examples repository, or git checkout mini-ch02 once the tag has been pushed. By the end of this step there is a binary that binds 127.0.0.1:6379, reads one RESP2 frame off the wire, and answers PING. No crates, no runtime, one file.

RESP2 is a text protocol with a length prefix wherever a length matters. A frame opens with one byte that says what it is, and every line ends with CRLF — two bytes, not one, and the decoder has to agree about both.

Five types carry data, and a sixth value means “nothing”:

First byte Frame Wire form
+ simple string +PONG\r\n
- error -ERR unknown command 'foo'\r\n
: integer :100\r\n, signed
$ bulk string $5\r\nhello\r\n
* array *2\r\n then two frames
$-1 / *-1 null a bulk string that is not there

The length prefix is the whole point. A bulk string’s payload can contain CRLF, or NUL, or invalid UTF-8, and the reader still knows where it ends, because the reader counts bytes instead of scanning for a terminator. Arrays nest to whatever depth the client likes.

Every request is an array of bulk strings. Here is what the socket actually receives when the real redis-cli 7.2.4 sends PING, captured off a live connection while this step was written:

*1\r\n$4\r\nping\r\n

Fourteen bytes. *1 is an array of one element, $4 is a bulk string of four bytes, ping is the payload, and the command name arrives lowercase — redis-cli does not bother with capitals, and real Redis does not care. The clone has to agree, which is why command matching later in this file folds case.

Encoding holds no decisions, so it gets no ceremony: a tag byte, the content, CRLF, recursing into array elements. Frame::encode appends onto a Vec<u8> the caller owns, which is what lets a reply be built in place and written with one write_all:

+OK\r\n Frame::Simple("OK")
:-2\r\n Frame::Integer(-2)
$5\r\nhello\r\n Frame::Bulk(b"hello")
$-1\r\n Frame::Null
*2\r\n:1\r\n$3\r\ntwo\r\n Frame::Array([Integer(1), Bulk(b"two")])

Integers are i64, not u64, because TTL answers -1 and -2 at step 5, and a frame type that cannot hold the reply is a frame type you get to rewrite later.

This is the half with the decisions in it. The decoder takes a &[u8], and returns the frame together with the number of bytes it consumed, so the caller can find the next frame in the same buffer. Three helpers do the work: read_line scans to the next CRLF and steps past it, take counts out n bytes, and expect_crlf checks the two bytes that close a bulk string’s payload. Two more small ones — length, which parses a count line and reads a negative as null, and text, which refuses a line that is not UTF-8 — keep the main match arms readable.

The distinction that matters is in the error type. Incomplete means the buffer ended inside the frame: the rest of it may still be on the wire, and nothing is wrong yet. An empty buffer is Incomplete, not an error. Invalid means every byte is there and the bytes are still not RESP. A parser that reports one error for both cannot tell “wait” from “no”, and step 3 is entirely about being able to wait.

Here is the reader half of src/main.rs with a main that decodes the captured bytes above:

wire "*1\r\n$4\r\nping\r\n"
frame Array([Bulk([112, 105, 110, 103])])
used 14 of 14 bytes
short Err(Incomplete)

Two details worth a second look. Bulk holds Vec<u8>, not String, which is why the debug print shows [112, 105, 110, 103] instead of "ping": a bulk string is bytes, and the frame layer has no business deciding it is text. The command layer makes that decision, once, where it reads the command name. And the array arm builds its Vec with push rather than with_capacity, because the count is the client’s claim while the buffer is the truth — a header announcing a million elements costs four bytes and buys nothing. The recursion is bounded the same way: every nesting level consumes at least four bytes of buffer, so a 4 KiB chunk caps the depth at about a thousand.

main binds the port, prints one line, and accepts connections in a loop. Each connection goes to serve, which does the same three things until the client hangs up: read one chunk, decode one frame, write one reply. A read that returns zero bytes is the end of the connection, not an error.

execute is the command table, and at this step it is one command deep. Command names fold to upper case first, because the captured bytes above are lowercase. PING alone answers +PONG; PING with one argument echoes that argument back as a bulk string, which is what the real server does — sent ping hello to a real redis-server 7.2.4 and the socket answered $5\r\nhello\r\n. Two arguments answer the real server’s own wording, captured the same way: -ERR wrong number of arguments for 'ping' command. Anything else is -ERR unknown command '<name>'.

Here is the whole file, byte for byte what the tag mini-ch02 contains:

Seven tests pin the behaviour down. Every frame type round-trips through encode and decode. The exact captured bytes decode to the expected frame. A frame cut in half — including the case where the payload has arrived and the closing CRLF has not — is Incomplete, while a wrong type byte, a non-numeric length, and a payload followed by something other than CRLF are all Invalid. One decode consumes exactly one frame and reports where the next one starts. And one test speaks over a real socket: an ephemeral listener, a server thread, a client that writes *1\r\n$4\r\nping\r\n and half-closes, and an assertion that the bytes coming back are +PONG\r\n.

$ cargo test
Compiling mini-redis v0.1.0
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.33s
Running unittests src/main.rs (/Volumes/RustBuild/tracks/debug/deps/mini_redis-f7b0b7e7128a87e0)
running 7 tests
test tests::bytes_that_are_not_resp_are_invalid ... ok
test tests::a_frame_in_two_pieces_is_incomplete_not_invalid ... ok
test tests::every_frame_type_round_trips ... ok
test tests::ping_with_an_argument_echoes_it ... ok
test tests::one_decode_consumes_exactly_one_frame ... ok
test tests::the_exact_bytes_the_real_redis_cli_sends_for_ping ... ok
test tests::ping_over_a_real_socket ... ok
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The order the tests print in varies between runs; the set does not. The Running line names this machine’s shared build cache; elsewhere it reads target/debug/deps/mini_redis-….

cargo run on this machine stops before it listens, because a real redis-server already owns 127.0.0.1:6379:

$ cargo run
Running `/Volumes/RustBuild/tracks/debug/mini-redis`
Error: Os { code: 48, kind: AddrInUse, message: "Address already in use" }

On a machine where the port is free the server prints listening on 127.0.0.1:6379 and waits. To check the clone against the real client without disturbing the server that owns the port, the same binary was bound to a scratch port and the real redis-cli pointed at it:

$ redis-cli -p 7777 ping
PONG
$ redis-cli -p 7777 ping hello
hello
$ redis-cli -p 7777 foo
ERR unknown command 'foo'
$ redis-cli -p 7777 ping a b
ERR wrong number of arguments for 'ping' command
  • A frame split across two packets ends the connection. serve does one read per frame, so a frame in two pieces decodes as Incomplete and the connection closes with a frame arrived in pieces. Verified by sending *1\r\n$4\r\npi, pausing, then ng\r\n: no reply, and that line in the server’s log.
  • A second frame in the same chunk is dropped. decode reports how many bytes it used and serve ignores the number. Pipelining — two commands written back to back — loses the second one. Both gaps are the same missing buffer, and step 3 is that buffer.
  • Inline commands are not spoken. A bare PING\r\n with no array around it is legal RESP2, and the real server answers it (captured: +PONG). The clone’s parser accepts arrays only, which is the dialect every client library uses.
  • A null bulk inside a request array decodes here. The real server refuses one with -ERR Protocol error: invalid bulk length (captured). Nothing sends that frame yet; GET will send $-1 as a reply at step 4, which is the direction that matters.

Step 3 keeps the bytes that serve throws away: a connection with a buffer, a decoder that survives a packet boundary, and ECHO to prove it. The track overview carries the full list as articles land.