Rate limiting per API key
Step 12 of 15. Step 11 put /links behind an API key, and left it wide open
behind that. A client holding the key can call the service as fast as curl
can spin, and no amount of error polish from step 10 changes what a thousand
requests a second does to a pool of five database connections. This step adds
a token bucket per API key: a burst allowance, a steady refill, and a 429 Too Many Requests with a Retry-After header for everything past it. It
also ends where every in-memory limiter ends, with the question in the brief:
why one instance is not enough.
Start from step 11’s end state: git checkout api-ch11 in your clone of
the examples repository, inside
axum-sqlx-api/, with the Postgres container from step 5 running and
DATABASE_URL exported. No crate joins Cargo.toml this step. The bucket is
a hundred lines of standard library underneath the tower machinery the last
step already put in place.
Where the limiter sits
Section titled “Where the limiter sits”The limiter needs to know which key is asking, and step 11 already answers
that: the auth layer leaves an ApiUser in the request extensions. So the
limiter is a second layer wrapped inside the auth layer. A request meets
auth first, then the limiter, then the handler. Two route_layer calls on
the same routes produce that order with one rule worth memorising: the last
layer added is the first to see the request. So auth goes last:
let protected = Router::new() .route("/links", get(list_links).post(create_link)) .route_layer(RateLimit::new(10.0, 5.0)) .route_layer(RequireApiKey::new(key));(A fragment, marked unverified because it names handlers and layers defined
elsewhere in the file and cannot compile on its own. main appears in full
further down.)
The order is not cosmetic. Add the limiter last instead and every request
reaches it before auth has run, with no ApiUser in its extensions. Rebuild
that way and this service answers 500 to everything, keyed or not:
curl -s -w " [%{http_code}]\n" -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/links{"error":"internal server error"} [500]That failure is deliberate. The limiter has nothing to key on and must not guess, so the error enum gains a variant for exactly that mistake.
The limiter, by hand
Section titled “The limiter, by hand”Everything new or changed in src/main.rs is here: the two error variants,
the bucket, and the layer. The parts step 11 wrote (ValidatedJson, the
handlers, RequireApiKey and check_key) are unchanged and left out:
use axum::{ Json, Router, extract::{Extension, FromRequest, Path, Query, Request}, http::{StatusCode, header, HeaderMap, HeaderValue}, response::{IntoResponse, Response}, routing::get,};use serde::{Deserialize, Serialize, de::DeserializeOwned};use serde_json::json;use sqlx::{PgPool, postgres::PgPoolOptions};use std::{ collections::HashMap, convert::Infallible, future::Future, pin::Pin, sync::{Arc, Mutex}, task::{Context, Poll}, time::Instant,};use thiserror::Error;use tower::{Layer, Service};use validator::{Validate, ValidationErrors};
#[derive(Debug, Error)]enum ApiError { // Extractor rejections carry their own status: 400 for malformed JSON, // 415 for a missing or wrong content-type. #[error("{1}")] BadBody(StatusCode, String), #[error("{0}")] Validation(#[from] ValidationErrors), #[error("{0}")] Unauthorized(&'static str), #[error("not found")] NotFound, #[error("rate limit exceeded")] TooManyRequests(u64), // The rate limiter reads the identity the auth layer left behind. This // variant means it found none - the two layers were wired in the wrong // order. That is a deployment bug, not a client's fault, so it maps to // 500 like the other internal errors. #[error("internal server error")] Miswired, // The sqlx error is kept as the source for logging (step 13) but never // serialised - its message names tables and columns a client has no // business reading. #[error("internal server error")] Database(#[from] sqlx::Error), #[error("internal server error")] Exhausted,}
impl IntoResponse for ApiError { fn into_response(self) -> Response { let status = match &self { ApiError::BadBody(status, _) => *status, ApiError::Validation(_) => StatusCode::UNPROCESSABLE_ENTITY, ApiError::Unauthorized(_) => StatusCode::UNAUTHORIZED, ApiError::NotFound => StatusCode::NOT_FOUND, ApiError::TooManyRequests(_) => StatusCode::TOO_MANY_REQUESTS, ApiError::Miswired | ApiError::Database(_) | ApiError::Exhausted => { StatusCode::INTERNAL_SERVER_ERROR } }; let mut res = (status, Json(json!({ "error": self.to_string() }))).into_response(); if let ApiError::TooManyRequests(retry) = &self { // RFC 9110: Retry-After tells the client when to try again. The // bucket refills continuously, so this is a whole-second // estimate, rounded up. res.headers_mut().insert( header::RETRY_AFTER, HeaderValue::from_str(&retry.to_string()).expect("a u64 is a valid header value"), ); } res }}
// The identity the auth layer proved. Handlers never re-check the header,// and the rate limiter below keys its buckets on it.#[derive(Clone)]struct ApiUser { key: Arc<str>,}
// One bucket per key. `capacity` tokens are available immediately - that is// the burst a client may spend at once - and `refill_per_sec` drips back in// continuously. A client that waits between requests never sees the limit;// one that hammers is cut to the refill rate once the burst is spent.struct TokenBucket { tokens: f64, updated: Instant,}
// Cloned services share one Limiter through the Arc, so every connection// agrees on how many tokens a key has left.struct Limiter { buckets: Mutex<HashMap<Arc<str>, TokenBucket>>, capacity: f64, refill_per_sec: f64,}
impl Limiter { fn new(capacity: f64, refill_per_sec: f64) -> Self { Self { buckets: Mutex::new(HashMap::new()), capacity, refill_per_sec, } }
// One token per request. Ok(()) admits it; Err(retry) is how many // seconds until a token is back, rounded up to a whole second because // Retry-After takes integer seconds. fn check(&self, key: Arc<str>, now: Instant) -> Result<(), u64> { let mut buckets = self.buckets.lock().expect("bucket map poisoned"); let bucket = buckets.entry(key).or_insert(TokenBucket { tokens: self.capacity, updated: now, }); let elapsed = now.duration_since(bucket.updated).as_secs_f64(); bucket.tokens = (bucket.tokens + elapsed * self.refill_per_sec).min(self.capacity); bucket.updated = now; if bucket.tokens >= 1.0 { bucket.tokens -= 1.0; Ok(()) } else { let secs = ((1.0 - bucket.tokens) / self.refill_per_sec).ceil(); Err(secs.max(1.0) as u64) } }}
#[derive(Clone)]struct RateLimit { limiter: Arc<Limiter>,}
impl RateLimit { fn new(capacity: f64, refill_per_sec: f64) -> Self { Self { limiter: Arc::new(Limiter::new(capacity, refill_per_sec)), } }}
impl<S> Layer<S> for RateLimit { type Service = RateLimitService<S>;
fn layer(&self, inner: S) -> Self::Service { RateLimitService { inner, limiter: self.limiter.clone(), } }}
#[derive(Clone)]struct RateLimitService<S> { inner: S, limiter: Arc<Limiter>,}
impl<S> Service<Request> for RateLimitService<S>where S: Service<Request, Response = Response, Error = Infallible> + Send + 'static, S::Future: Send + 'static,{ type Response = Response; type Error = Infallible; type Future = Pin<Box<dyn Future<Output = Result<Response, Infallible>> + Send>>;
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Infallible>> { self.inner.poll_ready(cx) }
fn call(&mut self, req: Request) -> Self::Future { // The auth layer runs first and left the identity behind. Its // absence here means the layers were wired in the wrong order - the // limiter has nothing to key on and must not guess. let key = match req.extensions().get::<ApiUser>() { Some(user) => user.key.clone(), None => { let res = ApiError::Miswired.into_response(); return Box::pin(async move { Ok(res) }); } }; match self.limiter.check(key, Instant::now()) { Err(retry) => { let res = ApiError::TooManyRequests(retry).into_response(); Box::pin(async move { Ok(res) }) } Ok(()) => Box::pin(self.inner.call(req)), } }}Three pieces, in the order they appear.
TokenBucket is the whole idea. It holds a fractional token count plus the
instant it was last touched. check tops the bucket up with however much
time has dripped in since, capped at capacity, then either spends one token
or reports the wait until one is back. Refill-by-elapsed-time means there is
no timer to wake and nothing to tick: a bucket that nobody touches costs
nothing. The wait is ceil-ed to a whole second because that is what
Retry-After takes. now is a parameter, not a call to Instant::now(),
for one reason: tests can pass any instant they like, and no test sleeps.
The Mutex is a standard-library lock inside async code, which usually
deserves suspicion. Here the critical section is a hashmap update measured in
nanoseconds and nothing awaits while it is held, so there is nothing for the
runtime to block on. One honest limit sits next to it: the map keeps one
entry per key it has ever seen, forever. With a single key from an
environment variable that is one entry. If keys ever come from a large pool,
the map needs eviction before it becomes a leak.
RateLimit and RateLimitService are step 11’s shape again: a Layer
factory, a Service that sees requests, Error = Infallible because axum
demands it, and rejections built by err.into_response() on an ApiError.
The limiter itself lives behind an Arc because tower clones the service per
connection; every clone must see the same token counts, which is what the
shared Mutex inside Limiter is for.
Two variants joined the enum. TooManyRequests carries the retry seconds,
and IntoResponse turns it into a 429 plus the header, the same
one-place-builds-the-body pattern the 401 uses:
let mut res = (status, Json(json!({ "error": self.to_string() }))).into_response(); if let ApiError::TooManyRequests(retry) = &self { // RFC 9110: Retry-After tells the client when to try again. The // bucket refills continuously, so this is a whole-second // estimate, rounded up. res.headers_mut().insert( header::RETRY_AFTER, HeaderValue::from_str(&retry.to_string()).expect("a u64 is a valid header value"), ); } res(An excerpt of the IntoResponse impl shown complete in the block above,
marked unverified because a fragment of it cannot compile on its own.)
Miswired maps to 500. A missing identity is a wiring bug in the deployment,
not a client’s fault, and it should look like every other internal error.
The warning, answered
Section titled “The warning, answered”Step 11 ended with the compiler asking who reads ApiUser.key. This step is
the answer. Rebuild and cargo build -q prints nothing: the dead-code
warning is gone because the limiter reads the field, and the comment on
ApiUser now describes what is true instead of what is coming.
Wiring it in
Section titled “Wiring it in”main changes by two lines and one comment:
#[tokio::main]async fn main() { let url = std::env::var("DATABASE_URL").expect("DATABASE_URL is not set"); let pool = PgPoolOptions::new() .max_connections(5) .acquire_timeout(std::time::Duration::from_secs(3)) .connect(&url) .await .expect("failed to connect to DATABASE_URL");
sqlx::migrate!().run(&pool).await.expect("failed to run migrations");
// The key lives in the environment, not in the source. The service // refuses to start without one - a missing key means every /links // request is rejected, which is better discovered at boot. let key = std::env::var("SHORTLINK_API_KEY").expect("SHORTLINK_API_KEY is not set");
// Two routers, because the routes have different audiences: browsers // hitting /{code} send no headers, and a load balancer polling /health // has none to offer. Only /links - the routes that change or read data - // sit behind the layers. route_layer, not layer: it runs only once a // route has matched, so a typo like /foo/bar stays a 404 instead of // being told to authenticate against a path that does not exist. // Both route_layers wrap the same routes, and the last one added is the // outermost: a request meets auth before the limiter, because the // limiter needs the identity auth leaves in the extensions. let protected = Router::new() .route("/links", get(list_links).post(create_link)) .route_layer(RateLimit::new(10.0, 5.0)) .route_layer(RequireApiKey::new(key));
let app = Router::new() .route("/health", get(health)) .route("/{code}", get(resolve_link)) .merge(protected) .layer(Extension(pool));
let listener = tokio::net::TcpListener::bind("0.0.0.0:8000").await.unwrap(); axum::serve(listener, app).await.unwrap();}(The block is unverified for the same reason as step 11’s: sqlx::migrate!
reads migrations/ at compile time, which the verifier’s standalone sandbox
does not have. The file is in the tag and cargo build proves it.)
RateLimit::new(10.0, 5.0) reads as: ten tokens of burst, five back per
second. A client can spend ten requests at once, then sustain five a second
forever. A client that spaces its requests out never hits the wall at all,
which is the point: the bucket punishes bursts, not use.
The whole flow, offline
Section titled “The whole flow, offline”No server needed to watch the bucket work. The demo drives the same Limiter
the layer uses, with a capacity of three and a refill of one per second, at
explicit instants so the output is deterministic:
use std::{ collections::HashMap, sync::{Arc, Mutex}, time::{Duration, Instant},};
struct TokenBucket { tokens: f64, updated: Instant,}
struct Limiter { buckets: Mutex<HashMap<Arc<str>, TokenBucket>>, capacity: f64, refill_per_sec: f64,}
impl Limiter { fn new(capacity: f64, refill_per_sec: f64) -> Self { Self { buckets: Mutex::new(HashMap::new()), capacity, refill_per_sec, } }
fn check(&self, key: Arc<str>, now: Instant) -> Result<(), u64> { let mut buckets = self.buckets.lock().expect("bucket map poisoned"); let bucket = buckets.entry(key).or_insert(TokenBucket { tokens: self.capacity, updated: now, }); let elapsed = now.duration_since(bucket.updated).as_secs_f64(); bucket.tokens = (bucket.tokens + elapsed * self.refill_per_sec).min(self.capacity); bucket.updated = now; if bucket.tokens >= 1.0 { bucket.tokens -= 1.0; Ok(()) } else { let secs = ((1.0 - bucket.tokens) / self.refill_per_sec).ceil(); Err(secs.max(1.0) as u64) } }}
fn main() { let limiter = Limiter::new(3.0, 1.0); let key: Arc<str> = "dev-key".into(); let mut now = Instant::now(); for i in 1..=4 { now += Duration::from_millis(100); match limiter.check(key.clone(), now) { Ok(()) => println!("request {i}: allowed"), Err(retry) => println!("request {i}: rejected, retry after {retry}s"), } } now += Duration::from_secs(2); for i in 5..=7 { now += Duration::from_millis(100); match limiter.check(key.clone(), now) { Ok(()) => println!("request {i}: allowed"), Err(retry) => println!("request {i}: rejected, retry after {retry}s"), } } println!();
let now = Instant::now(); let single = Limiter::new(3.0, 1.0); let allowed = (0..10).filter(|_| single.check(key.clone(), now).is_ok()).count(); println!("one instance: {allowed} of 10 requests allowed"); let replicas = [Limiter::new(3.0, 1.0), Limiter::new(3.0, 1.0)]; let allowed = (0..10) .filter(|i| replicas[i % 2].check(key.clone(), now).is_ok()) .count(); println!("two replicas: {allowed} of 10 requests allowed"); println!();
let limiter = Limiter::new(3.0, 1.0); let alpha: Arc<str> = "alpha".into(); let beta: Arc<str> = "beta".into(); for _ in 0..4 { limiter.check(alpha.clone(), now).ok(); } match limiter.check(beta, now) { Ok(()) => println!("alpha is spent; beta's first request: allowed"), Err(_) => println!("alpha is spent; beta's first request: rejected"), }}request 1: allowedrequest 2: allowedrequest 3: allowedrequest 4: rejected, retry after 1srequest 5: allowedrequest 6: allowedrequest 7: rejected, retry after 1sThe first three requests spend the burst inside 300 milliseconds. The fourth finds 0.3 tokens and waits. Two seconds later the bucket has refilled past two tokens, admits two more, and runs dry again. The last two lines are the part to remember from this step: with two replicas round-robining the same ten requests, six get through, and one key’s spend never touches another key’s bucket. Both of those facts return in “Why one instance is not enough” below.
See it run
Section titled “See it run”Boot with a key, as in step 11:
DATABASE_URL='postgres://postgres:dev@localhost:5432/shortlink' \SHORTLINK_API_KEY=dev-key cargo runFire twelve requests at once:
seq 1 12 | xargs -P12 -I{} curl -s -o /dev/null -w "%{http_code}\n" \ -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/links | sort | uniq -c 10 200 2 429Ten land inside the burst allowance; two are told to back off. The boundary holds because the volley arrives within tens of milliseconds, less than the refill can matter. Spread the same twelve requests over two seconds and every one of them passes, two seconds being worth ten tokens at five per second.
One rejected request in full:
curl -si -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/linksHTTP/1.1 429 Too Many Requestscontent-type: application/jsonretry-after: 1content-length: 31date: Sun, 18 Oct 2026 16:24:00 GMT
{"error":"rate limit exceeded"}retry-after: 1 is the bucket’s arithmetic: a fraction of a second until the
next token, rounded up to the whole second the header wants. A client that
honours it retries in one second and finds roughly five tokens waiting.
Two seconds of quiet later, the bucket is full again:
sleep 2curl -s -o /dev/null -w "%{http_code}\n" -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/linkscurl -s -o /dev/null -w "%{http_code}\n" -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/linkscurl -s -w " [%{http_code}]\n" -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/links200200{"page":1,"limit":20,"links":[]} [200]And the ordering proof. Spend the bucket, then ask once without a key and once with it:
seq 1 12 | xargs -P12 -I{} curl -s -o /dev/null -w "%{http_code}\n" \ -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/links | sort | uniq -ccurl -s -o /dev/null -w "no key: %{http_code}\n" -X POST http://127.0.0.1:8000/links \ -H 'content-type: application/json' -d '{"url":"https://example.com/x"}'curl -s -o /dev/null -w "with key: %{http_code}\n" \ -H 'Authorization: Bearer dev-key' http://127.0.0.1:8000/links 10 200 2 429no key: 401with key: 429The bucket is empty and the unauthenticated request still gets its 401, not a 429 it has not earned and not the 500 a mis-wired stack would give. Auth runs first, exactly as wired.
Tests that do not sleep
Section titled “Tests that do not sleep”Three tests join the existing three in src/main.rs:
#[cfg(test)]mod tests { use super::*; use std::time::Duration;
#[test] fn a_fresh_bucket_bursts_to_capacity_then_rejects() { let limiter = Limiter::new(3.0, 1.0); let key: Arc<str> = "dev-key".into(); let start = Instant::now(); for _ in 0..3 { assert!(limiter.check(key.clone(), start).is_ok()); } // 0.7 seconds until the next token, rounded up to a whole second. assert_eq!(limiter.check(key, start), Err(1)); }
#[test] fn elapsed_time_refills_the_bucket() { let limiter = Limiter::new(3.0, 1.0); let key: Arc<str> = "dev-key".into(); let start = Instant::now(); for _ in 0..3 { limiter.check(key.clone(), start).unwrap(); } // Half a second back is half a token - not enough to admit. let half = start + Duration::from_millis(500); assert!(limiter.check(key.clone(), half).is_err()); // Half a second more completes the token. let full = half + Duration::from_millis(500); assert!(limiter.check(key, full).is_ok()); }
#[test] fn keys_do_not_share_a_bucket() { let limiter = Limiter::new(2.0, 1.0); let alpha: Arc<str> = "alpha".into(); let beta: Arc<str> = "beta".into(); let start = Instant::now(); for _ in 0..2 { limiter.check(alpha.clone(), start).unwrap(); } assert!(limiter.check(alpha, start).is_err()); assert!(limiter.check(beta, start).is_ok()); }}They pass explicit Instants, so none of them sleeps and none of them is
flaky on a loaded machine. In the project, cargo test runs all six:
running 6 teststest tests::a_fresh_bucket_bursts_to_capacity_then_rejects ... oktest tests::keys_do_not_share_a_bucket ... oktest tests::encodes_the_boundaries ... oktest tests::elapsed_time_refills_the_bucket ... oktest tests::the_sequence_keeps_moving ... oktest tests::codes_are_seven_characters_from_the_alphabet ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s(Your line order may differ; the tests run in parallel.)
Why one instance is not enough
Section titled “Why one instance is not enough”The limiter is correct and complete for one process, and that is the most it
can ever be. Run two replicas of this service behind a round-robin load
balancer and each keeps its own HashMap: the offline demo’s six-of-ten is
the general rule, and N replicas hand a client N times the burst. Deploy and
every bucket resets, so every restart hands abusers a fresh allowance, and a
crash loop hands them one every few seconds.
The fix is to move the counter somewhere all replicas look: Redis is the
usual choice. One INCR plus an EXPIRE per request, or a small Lua script
where the check and the increment must be one atomic step, turns N private
buckets into one shared one. The cost is real: a network round trip on every
request’s hot path, and a new decision to make, because when Redis is down
the service must choose to fail open (no limits) or closed (no traffic).
What survives the migration is the seam. check takes a key and an instant
and returns admit-or-wait; nothing else in the service knows the buckets live
in a HashMap. Swap the map for a Redis client behind that signature and the
layer, the error variants, and the 429s do not change. What does not survive
is the pretence that this step is production-complete: it is the right
mechanism with a one-process horizon, and step 15’s deployment story is where
the replica question stops being hypothetical.
Where this leaves the service
Section titled “Where this leaves the service”The writes and reads that step 11 gated are now metered per key: a burst, a sustained rate, and a polite 429 with a retry time. The error format did not fork, the public routes stayed public, and the compiler’s question from step 11 has an answer in the code.
The end state is tagged api-ch12 in
the examples repository —
git checkout api-ch12 inside axum-sqlx-api/. cargo test runs the six
tests offline; the curl sequence above needs the container up.
Next: structured logging and request tracing. The Database variant has kept
the sqlx error as its source since step 10 for exactly that step, and the 429s
this one added are far more useful in a log line than in a test.
The track overview keeps the full list current as
articles land.