Skip to content
Hacking with RustHacking with Rust
Select theme

How to handle UTF-8 text

Short answer: A String or &str in Rust can only ever hold valid UTF-8: literals are checked at compile time, and the methods that add text take text that is already valid. Handling UTF-8 text usually means doing nothing special — chars() when you mean characters, as_bytes() when you mean bytes, and no decoding step at any boundary.

From a fresh project — cargo new utf8-demo, then replace src/main.rs with each block below and run it with cargo run. The project name only shows up in cargo’s own output; nothing else on this page depends on it.

The compiler checks the literals

Section titled “The compiler checks the literals”

Write a byte outside ASCII into a string literal and the program does not build:

fn main() {
    let text = "\xFF";
    println!("{text}");
}
Run this in the Rust Playground
error: out of range hex escape
 --> src/main.rs:2:17
  |
2 |     let text = "\xFF";
  |                 ^^^^ must be a character in the range [\x00-\x7f]
  |
  = help: if you want to write a byte literal, use `b'\xFF'`
  = help: if you want to write a Unicode character, use `'\u{FF}'`

No binary is produced, so there is no runtime left to surprise anyone. The two help lines name the types that do hold that byte: b'\xFF' is a u8, and '\u{FF}' is a char. A string literal is checked as it is written, which is why a &str that isn’t valid UTF-8 cannot exist.

Every way in takes valid text

Section titled “Every way in takes valid text”

The check does not stop at literals. The methods that put text into a String take a &str or a char:

fn main() {
    let mut text = String::from("caf");
    text.push('é');
    text.push_str(" au lait");
    println!("{text} is {} chars and {} bytes", text.chars().count(), text.len());
}
Run this in the Rust Playground
café au lait is 12 chars and 13 bytes

push takes a char — one Unicode scalar value, valid UTF-8 by definition even when it takes two or four bytes in the buffer. push_str takes a &str, text that something has already checked; String::from, format! and the + operator take text the same way. Ask for the byte entry point and the compiler says there isn’t one:

fn main() {
    let mut text = String::from("caf");
    text.push(0xC3);
    println!("{text}");
}
Run this in the Rust Playground
error[E0308]: mismatched types
 --> src/main.rs:3:15
  |
3 |     text.push(0xC3);
  |          ---- ^^^^ expected `char`, found `u8`
  |          |
  |          arguments to this method are incorrect
  |
note: method defined here
 --> /rustc/b940084d7eb6a299eb4bfeb8e34901bc051e7ac4/library/alloc/src/string.rs:1432:11


For more information about this error, try `rustc --explain E0308`.

0xC3 is a u8; push wants a char. String::from_utf8 does take bytes, and it hands back a Result because checking is its job — that conversion has its own article later in this chapter.

What the guarantee buys

Section titled “What the guarantee buys”

A function that takes a &str starts work immediately: no decode step, nothing to unwrap, no encoding to name. The operations are Unicode-wide, because the text is known to be valid:

fn main() {
    for text in ["straße", "¡hola!", "καλημέρα"] {
        println!("{text} -> {}", text.to_uppercase());
    }
}
Run this in the Rust Playground
straße -> STRASSE
¡hola! -> ¡HOLA!
καλημέρα -> ΚΑΛΗΜΈΡΑ

ß becomes SS: full Unicode case mapping, not the ASCII rules, and the Greek line goes through the same call. Nothing here configured an encoding or converted anything. The same guarantee is what makes iterating over the characters of a string total, why slices may only cut at char boundaries, and why s[0] is refused. When the consumer wants bytes, as_bytes() hands over the UTF-8 encoding directly, with no conversion step.

Text that isn’t UTF-8

Section titled “Text that isn’t UTF-8”

Some bytes were never text: a file in another encoding, a network payload, an image. Those are Vec<u8> or &[u8], and they stay that way until something checks them. Filenames are the other case: a path is an OsStr, not a String, because a filename can hold bytes that aren’t UTF-8, and Path::to_str() returns None when it can’t be text. The types keep the guarantee by refusing the data that would break it.

Section titled “Related”