Benchmarking WebAssembly honestly
Step 6 of 8, and it is the measurement step. Step 4 profiled instead of guessing, and step 5 moved the 100,000-item dataset into wasm memory. Both quoted times on the way past. This step builds the benchmark those numbers deserve — one cold run reported as a cold run, two warm-ups, seven timed runs, the median quoted — and collects a lesson most wasm benchmark posts skip: the first attempt at timing from inside Rust does not merely mislead, it traps.
Start from the step 5 end state — git checkout wasm-ch05 in your clone of the
examples repository — or from that article’s own
steps. Versions used here: Rust 1.99.0, wasm-pack 0.15.0, wasm-bindgen 0.2.129, web-time 1.1.0,
Node v22.22.2, on an arm64 Mac.
What one run is worth
Section titled “What one run is worth”A benchmark that runs the thing once and prints the number is reporting the least representative run it could have picked. The first time a JavaScript engine executes a function, the function is interpreter bytecode with cold inline caches; by the tenth call the optimising compiler has rebuilt it. Neither run is “the” speed. Quoting the first as if it were the steady state inflates every wasm-versus-JavaScript comparison in the same direction — towards whichever side warmed up worse.
So the harness, bench.mjs, is built around the distinction. It times three variants of the same
search over the same 100,000 items: original is step 1’s all-JavaScript search exactly as the app
had it, lowered is step 4’s folding fix in pure JavaScript, and wasm is step 5’s port. Each
variant gets one timed cold run, two warm-ups that are never reported, and seven timed runs whose
median is the number this article quotes. The dataset is the same deterministic generator as every
step before, so the best hits are identical on every machine; the times are not, which is why they
are reported with their spread.
// bench.mjs — the honest version of the benchmark step 1 promised.//// node bench.mjs//// Three search variants over the same 100,000-item dataset:// original — step 1's all-JavaScript search, unchanged// lowered — the pre-folded JavaScript from step 4's profile// wasm — the step 5 wasm search, dataset loaded once// Each variant gets one cold, timed run — what a naive benchmark reports —// then two warm-up runs, then seven timed runs. The median of the seven is// the number this article quotes. The dataset is generated deterministically,// so the results are the same on every machine; the times are not, which is// exactly why they are reported the way they are.
import { readFile } from 'node:fs/promises';
const wasm = await import('./pkg/fuzzy_search.js');wasm.initSync({ module: await readFile('./pkg/fuzzy_search_bg.wasm') });
// The dataset: 100,000 catalogue items, exactly as step 1 generated them.const MATERIALS = ['steel', 'brass', 'carbon', 'nylon', 'ceramic', 'alloy'];const KINDS = ['bracket', 'spacer', 'gear', 'clamp', 'hinge', 'pulley', 'bearing', 'fastener'];const USES = ['damping', 'mounting', 'rotation', 'sealing', 'support', 'tension'];
const items = [];for (let i = 0; i < 100_000; i++) { const material = MATERIALS[i % 6]; const kind = KINDS[Math.floor(i / 6) % 8]; const use = USES[Math.floor(i / 48) % 6]; items.push({ title: `${material} ${kind} #${i}`, description: `${material} ${kind} for ${use}, rated ${100 + (i % 40)} kg, batch ${9000 + (i % 900)}`, });}
const corpus = items.map((item) => `${item.title}\t${item.description}`).join('\n');wasm.load_dataset(corpus);
// Fuzzy subsequence score of query against text. -1 means no match. Step 1's// function, unchanged.function fuzzyScore(query, text) { const q = query.toLowerCase(); const t = text.toLowerCase(); let score = 0; let qi = 0; let streak = 0; for (let ti = 0; ti < t.length && qi < q.length; ti++) { if (t[ti] === q[qi]) { qi++; streak++; score += 2 + streak; // consecutive matches are worth more } else { streak = 0; } } if (qi < q.length) return -1; // ran out of text before running out of query return score;}
function searchJS(query) { const hits = []; for (const item of items) { const score = Math.max(fuzzyScore(query, item.title), fuzzyScore(query, item.description)); if (score >= 0) hits.push({ title: item.title, score }); } hits.sort((a, b) => b.score - a.score); return hits.slice(0, 20);}The second half of the file: the lowered variant, the wasm variant, and the timing loop. The wasm variant unpacks the same twenty hits the real app displays — benchmarking less work than the app does is its own kind of lie.
// The pre-folded variant from step 4: the dataset's case folding happened// once, above, and the query folds once per search.const lowered = items.map((item) => ({ title: item.title.toLowerCase(), description: item.description.toLowerCase(),}));
function scoreLower(q, text) { let score = 0; let qi = 0; let streak = 0; for (let ti = 0; ti < text.length && qi < q.length; ti++) { if (text[ti] === q[qi]) { qi++; streak++; score += 2 + streak; } else { streak = 0; } } if (qi < q.length) return -1; return score;}
function searchLowered(query) { const q = query.toLowerCase(); const hits = []; for (const item of lowered) { const score = Math.max(scoreLower(q, item.title), scoreLower(q, item.description)); if (score >= 0) hits.push({ title: item.title, score }); } hits.sort((a, b) => b.score - a.score); return hits.slice(0, 20);}
// The wasm search, the call the real app makes: packed pairs cross once,// then the twenty hits are unpacked for display.function searchWasm(query) { const flat = wasm.search(query, 20); const hits = []; for (let i = 0; i < flat.length; i += 2) { hits.push({ title: items[flat[i]].title, score: flat[i + 1] }); } return hits;}
const VARIANTS = [ ['original JS', searchJS], ['lowered JS', searchLowered], ['wasm', searchWasm],];
const QUERIES = ['bracket', 'steel bracket', 'nylon pulley for tension'];
function timeOne(fn) { const t0 = performance.now(); fn(); return performance.now() - t0;}
for (const q of QUERIES) { console.log(`query "${q}"`); for (const [label, fn] of VARIANTS) { const call = () => fn(q); const cold = timeOne(call); // timed, and reported as what it is timeOne(call); // warm-up: the JIT sees the function, optimizes, tiers up timeOne(call); const times = []; for (let i = 0; i < 7; i++) times.push(timeOne(call)); times.sort((a, b) => a - b); console.log( ` ${label}: cold ${cold.toFixed(1)} ms, median ${times[3].toFixed(1)} ms of 7, min ${times[0].toFixed(1)} ms`, ); if (q === 'bracket') { console.log(` all seven: ${times.map((t) => t.toFixed(1)).join(', ')}`); } } // The scan alone, timed inside the module: the query crosses once, then // bench_search repeats the exact call a keystroke makes seven times with // one Instant around the loop. One warm call first — wasm tiers up too, // just less dramatically than JavaScript. wasm.bench_search(q, 1); const inside = wasm.bench_search(q, 7) / 7; console.log(` wasm scan only, inside the module: ${inside.toFixed(2)} ms per call`);}Why a median and not a mean, given seven samples of the same work? Because one scheduler hiccup moves a mean and barely moves a median. With the samples below — plausible timings for the same search, one of them hit by a hiccup — the mean lands above every typical sample while the median stays where six of the seven runs actually were:
// Seven timings of the same search, as a harness collects them. The 31.7// is a scheduler hiccup — the same work ran every time.fn median(times: &mut [f64]) -> f64 { times.sort_by(|a, b| a.total_cmp(b)); times[times.len() / 2]}
fn main() { let mut samples = [29.3, 29.4, 31.7, 29.5, 29.8, 29.9, 30.2]; let mean = samples.iter().sum::<f64>() / samples.len() as f64; println!("mean {:.1} ms", mean); println!("median {:.1} ms", median(&mut samples));}mean 30.0 msmedian 29.8 msThe mean is higher than five of the seven samples. The median is one of them. That is the whole argument for sorting instead of averaging.
The last two lines of the loop call bench_search, an export that does not exist yet — the module
timing its own scan. That is where this step goes sideways once, on purpose.
Timing from inside the module
Section titled “Timing from inside the module”The JavaScript-timed wasm number bundles two things: the scan and the road to it — the glue, the
query string copied in, the packed pairs copied out. To see the scan alone, time it in Rust, around
the same call a keystroke makes. The obvious import for that is std::time::Instant, and the first
draft of bench_search used exactly that. It compiles. Here is node bench.mjs with that draft
built in:
node bench.mjsquery "bracket" original JS: cold 32.6 ms, median 29.8 ms of 7, min 29.3 ms all seven: 29.3, 29.5, 29.8, 29.8, 30.0, 31.1, 36.4 lowered JS: cold 26.8 ms, median 25.3 ms of 7, min 25.0 ms all seven: 25.0, 25.1, 25.1, 25.3, 25.5, 25.5, 25.5 wasm: cold 25.5 ms, median 22.9 ms of 7, min 22.6 ms all seven: 22.6, 22.7, 22.7, 22.9, 23.1, 23.2, 23.5The three JavaScript variants of the first query print, and then the run dies:
wasm://wasm/0002eaa2:1
RuntimeError: unreachable at wasm://wasm/0002eaa2:wasm-function[24]:0x5bce at wasm://wasm/0002eaa2:wasm-function[41]:0x6d35 at wasm://wasm/0002eaa2:wasm-function[28]:0x5e52 at Module.bench_search (file:///Users/rango/www/hackingwithrust/code/react-wasm/pkg/fuzzy_search.js:21:22) at file:///Users/rango/www/hackingwithrust/code/react-wasm/bench.mjs:151:8
Node.js v22.22.2Exit code 1, queries two and three never run, and the panic message — if there is one — is gone.
The wasm addresses are this build’s and the path is this machine’s checkout; the shape is what
matters. It is step 5’s trap again, RuntimeError: unreachable, but this time the code did nothing
wrong at run time. The problem is the import.
Instant has no working implementation on wasm32-unknown-unknown. The pinned toolchain’s std
source says so outright — for this target, Instant::now is a stub that panics:
impl Instant { pub fn now() -> Instant { panic!("time not implemented on this platform") }Rust has no clock to ask for on a platform whose only clock lives in JavaScript, so the call panics,
the panic aborts the module, and without a panic hook the trap reaches JavaScript as the same
messageless RuntimeError: unreachable step 5 diagnosed. bench_search is not unusual in hitting
this — anything that wants to time itself inside wasm on this target hits it.
The fix, and the code this step adds
Section titled “The fix, and the code this step adds”The crate for the job is web-time: a drop-in replacement that
re-exports std::time on every native target and implements Instant with the browser’s
Performance.now() on wasm32-unknown-unknown. One dependency, one import change:
[package]name = "fuzzy-search"version = "0.1.0"edition = "2024"
[lib]crate-type = ["cdylib"]
[dependencies]wasm-bindgen = "0.2"web-time = "1.1.0"The new part of src/lib.rs — bench_search and the pieces it calls. load_dataset, which sits
between the static and search in the file, is left out here because the listing never runs; in
your lib.rs everything below the use lines is already there from step 5 except bench_search
itself:
use std::hint::black_box;use std::sync::OnceLock;use wasm_bindgen::prelude::*;use web_time::Instant;
// The dataset once it has crossed: the corpus keeps the bytes exactly as they// arrived — one string, one item per line, title and description split by a// tab — and ranges records where each item's two fields sit inside it.static DATASET: OnceLock<(String, Vec<[u32; 4]>)> = OnceLock::new();
// The walk both scorers share: the query arrives already folded, the text// folds here. fuzzy_score and search call this, which is what keeps their// scores identical to each other and to step 1's JavaScript.fn score_folded(q: &[char], text: &str) -> i32 { let t = text.to_lowercase(); let mut score = 0; let mut qi = 0; let mut streak = 0; for tc in t.chars() { if qi == q.len() { break; } if tc == q[qi] { qi += 1; streak += 1; score += 2 + streak as i32; } else { streak = 0; } } if qi < q.len() { return -1; } score}
// One crossing per keystroke: the query in, packed index/score pairs out.// The titles never cross back — JavaScript kept its own copy for display.#[wasm_bindgen]pub fn search(query: &str, limit: usize) -> Vec<u32> { let Some((corpus, ranges)) = DATASET.get() else { panic!("search called before load_dataset"); }; let q: Vec<char> = query.to_lowercase().chars().collect();
let mut hits: Vec<(i32, u32)> = Vec::new(); // (score, item index) for (index, &[ts, te, ds, de]) in ranges.iter().enumerate() { let score = score_folded(&q, &corpus[ts as usize..te as usize]) .max(score_folded(&q, &corpus[ds as usize..de as usize])); if score >= 0 { hits.push((score, index as u32)); } } // Stable sort, biggest score first: ties stay in index order, the same // order JavaScript's Array#sort keeps. hits.sort_by(|a, b| b.0.cmp(&a.0)); hits.truncate(limit);
let mut out = Vec::with_capacity(hits.len() * 2); for (score, index) in hits { out.push(index); out.push(score as u32); } out}
// The scan timed from inside the module: the query crosses once, then the// same call a keystroke makes repeats iters times with one Instant around// the loop. Total milliseconds back; no boundary crossing inside the timed// region, so this measures the scan, not the road to it.#[wasm_bindgen]pub fn bench_search(query: &str, iters: u32) -> f64 { let start = Instant::now(); let mut sink = 0u32; for _ in 0..iters { let hits = search(query, 20); sink = sink.wrapping_add(hits.first().copied().unwrap_or(0)); } black_box(sink); // keep the optimizer from deleting the loop start.elapsed().as_secs_f64() * 1000.0}Three details are doing work in bench_search.
web_time::Instant, not std::time::Instant. Same API, working clock — on native targets
web-time is literally std::time re-exported, so the type is not a wasm special case anywhere
else.
black_box(sink). Every timed call’s result flows into sink, and sink flows into
black_box. A release-build optimizer is allowed to delete a computation whose result it can prove
is never observed — a benchmark whose loop vanishes into dead-code elimination reports the speed of
nothing. black_box marks the value as observable, and the loop survives.
The result is returned as total milliseconds. The caller divides by the iteration count; one
Instant around the whole loop keeps the clock calls out of the measurement.
Rebuild, and this time nothing traps:
wasm-pack build --target web[INFO]: 🎯 Checking for the Wasm target...[INFO]: 🌀 Compiling to Wasm... Compiling fuzzy-search v0.1.0 (/Users/rango/www/hackingwithrust/code/react-wasm) Finished `release` profile [optimized] target(s) in 0.23s[INFO]: ⬇️ Installing wasm-bindgen...[INFO]: Optimizing wasm binaries with `wasm-opt`...[INFO]: Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended[INFO]: ✨ Done in 0.54s[INFO]: 📦 Your wasm pkg is ready to publish at /Users/rango/www/hackingwithrust/code/react-wasm/pkg.The path is this machine’s checkout; yours will show your own. The module grew from 47,539 to
53,421 bytes — the new export, its glue, and the clock call web-time compiles in.
The numbers
Section titled “The numbers”node bench.mjsquery "bracket" original JS: cold 32.4 ms, median 29.5 ms of 7, min 29.3 ms all seven: 29.3, 29.3, 29.3, 29.5, 29.8, 30.1, 30.7 lowered JS: cold 26.6 ms, median 24.5 ms of 7, min 24.0 ms all seven: 24.0, 24.4, 24.4, 24.5, 24.5, 24.6, 25.1 wasm: cold 24.6 ms, median 22.8 ms of 7, min 22.6 ms all seven: 22.6, 22.7, 22.7, 22.8, 23.1, 23.1, 23.6 wasm scan only, inside the module: 23.13 ms per callquery "steel bracket" original JS: cold 30.7 ms, median 30.7 ms of 7, min 29.9 ms lowered JS: cold 25.6 ms, median 25.5 ms of 7, min 25.0 ms wasm: cold 22.7 ms, median 22.9 ms of 7, min 22.6 ms wasm scan only, inside the module: 22.75 ms per callquery "nylon pulley for tension" original JS: cold 30.6 ms, median 30.9 ms of 7, min 30.5 ms lowered JS: cold 24.7 ms, median 25.1 ms of 7, min 24.7 ms wasm: cold 24.2 ms, median 24.5 ms of 7, min 23.9 ms wasm scan only, inside the module: 24.02 ms per callRead it in four groups.
Cold against median. The original’s cold run is 9% above its median, wasm’s is 8% above. That
gap is the price of quoting the first run: a benchmark that stops after one call publishes a number
nobody will ever see again in steady use. Note also "steel bracket", where the cold wasm run
(22.7) came in below the median (22.9) — single runs do not even lie consistently.
The spread. The all-seven line for "bracket" stays inside a 1.4 ms window on every variant.
That is what makes the median worth quoting: it is a stable number, and a second full run of this
harness landed within a millisecond of every median printed here. The times are this machine’s; the
shape of the comparison is not.
The honest score so far. wasm runs the original search in about four fifths of the time — 21%
to 25% faster across the three queries. But the lowered variant, which is pure JavaScript and is
nothing more than step 4’s folding fix, already takes 17% to 19% off the original. What is left for
wasm over JavaScript that has stopped folding 200,000 strings per keystroke is 2% to 10%. The
biggest win in this track so far came from an algorithmic change in JavaScript, not from the port.
The port still wins — but by a margin you could talk yourself out of, and step 8 will come back to
exactly that.
Where the time actually goes. The in-module scan (23.13, 22.75, 24.02 ms) lands within half a millisecond of the JavaScript-timed wasm median in every query. The boundary crossing step 5 worked so hard to slim down costs nothing measurable at this input size: the scan itself is the workload. That is worth knowing before anyone starts hand-optimising glue.
Why most published numbers are wrong
Section titled “Why most published numbers are wrong”Every failure mode this harness guards against shows up in real benchmark posts, and each one has a concrete fix:
- The cold run quoted as steady state. First-run numbers carry compilation, cache misses, and tier-up — 8–10% of inflation here. Fix: warm up before timing, or time the cold run and label it.
- One run, no median. A single 31.7 among 29.5s drags a mean past every typical sample, as the
mean/medianblock above showed. Fix: several runs, sorted, middle one quoted. - The result never observed. An optimizer may delete a loop whose result is dead — the
benchmark then measures the empty loop. Fix: feed every result somewhere (
black_boxin Rust; in JavaScript, sum the hits and print the sum). - Two sides, two environments. A wasm build on one machine against a JavaScript baseline on another compares computers, not languages. Fix: one machine, one engine, and the versions in the article header — which exist so a reader can judge how far this number travels.
None of this is wasm-specific, which is the point: the same harness discipline you would apply to any timing claim applies here, plus one wasm-only trap — the clock itself is not where you expect it.
Where the code stands
Section titled “Where the code stands”This step’s end state is tagged wasm-ch06 in
the examples repository. bench.mjs is new;
Cargo.toml gains web-time = "1.1.0"; src/lib.rs gains bench_search and the web_time::Instant
import. check.mjs and check-search.mjs are exactly as step 5 left them, and still pass — the
scores did not move while nobody was measuring. Checking out the tag puts your tree where this
article ends:
git clone https://github.com/hackingwithrust/examplescd examples/react-wasmgit checkout wasm-ch06wasm-pack build --target webnode check.mjsnode check-search.mjsnode bench.mjsStep 7, Wiring WebAssembly into a React app, takes the search into a real interface: the module loaded through Vite, the search running off the main thread, and the numbers from this page as the budget the UI has to hit. The track overview keeps the full list current as articles land.