Allocation

from_utf8_lossy doesn’t always allocate. It hands back a Cow<str> that borrows your bytes when they’re already valid UTF-8 — you only pay for a String when there’s an invalid byte to replace.

The assumption that costs allocations

It’s easy to read this and assume every call builds a fresh String:

1

let text = String::from_utf8_lossy(bytes);

It doesn’t. The return type is Cow<'_, str> — clone-on-write. If bytes is valid UTF-8 (the common case for most files, headers, and protocol fields), you get back a Cow::Borrowed that points straight at your slice. No copy, no heap allocation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

use std::borrow::Cow;

fn read_name(bytes: &[u8]) -> Cow<'_, str> {
    String::from_utf8_lossy(bytes)
}

let valid = b"hello world";
let s = read_name(valid);
assert!(matches!(s, Cow::Borrowed(_))); // borrowed — nothing allocated
assert_eq!(s, "hello world");

You only pay on the rare path

The allocation happens only when there’s an invalid byte to swap for the replacement character U+FFFD. Then — and only then — it builds an owned String:

1
2
3
4

let invalid = &[b'c', b'a', b'f', 0xFF];
let s2 = read_name(invalid);
assert!(matches!(s2, Cow::Owned(_))); // owned — had to fix a bad byte
assert_eq!(s2, "caf\u{fffd}");

So the cost scales with how messy your input is, not with how often you call it.

Don’t undo it with a reflexive .to_string()

The anti-pattern is forcing an allocation right back on:

1

let owned = String::from_utf8_lossy(bytes).to_string(); // ⚠️ always allocates

Keep the Cow for as long as you’re only reading. If a caller genuinely needs ownership, into_owned() allocates on the borrowed path but reuses the buffer on the owned path — no double allocation:

1
2

let owned: String = read_name(b"abc").into_owned();
assert_eq!(owned, "abc");

When you’re decoding bytes you’ll mostly just inspect, let from_utf8_lossy stay a Cow. Valid input — the usual case — flows through without touching the heap.

Every collect::<Vec<_>>() in the middle of a pipeline is a heap allocation and a full pass over your data. Adapters like map and filter are lazy and fuse together — chain them and the whole transformation runs in one pass with zero temporary Vecs.

A Vec between every step

It’s tempting to do one transformation at a time, binding each result to a variable. Every collect() allocates a throwaway Vec and walks the entire sequence before the next step even starts:

1
2
3
4
5
6
7

let nums = [1, 2, 3, 4, 5, 6];

let doubled: Vec<i32> = nums.iter().map(|&n| n * 2).collect();
let big: Vec<i32> = doubled.into_iter().filter(|&n| n % 4 == 0).collect();
let sum: i32 = big.iter().sum();

assert_eq!(sum, 24);

Two intermediate Vecs, two extra allocations, three separate passes — all to compute a single number.

One chain, one pass, no temporaries

The adapters compose directly. Nothing is materialized until the final consumer (sum) pulls values through, so there are no intermediate collections at all:

1
2
3
4
5
6
7
8
9

let nums = [1, 2, 3, 4, 5, 6];

let sum: i32 = nums
    .iter()
    .map(|&n| n * 2)        // 2, 4, 6, 8, 10, 12
    .filter(|&n| n % 4 == 0) // 4, 8, 12
    .sum();                  // 24

assert_eq!(sum, 24);

Each element flows through map then filter then into the sum, one at a time. No buffer is ever allocated.

Laziness means short-circuiting works

Because nothing runs until pulled, a chain only does the work it needs. Add a take(2) and the pipeline stops after producing two results — the elements past that point are never touched:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

use std::cell::Cell;

let visited = Cell::new(0);
let nums = [1, 2, 3, 4, 5, 6, 7, 8];

let first_two: Vec<i32> = nums
    .iter()
    .inspect(|_| visited.set(visited.get() + 1))
    .map(|&n| n * 10)
    .filter(|&n| n > 20)
    .take(2)
    .collect();

assert_eq!(first_two, [30, 40]);
assert_eq!(visited.get(), 4); // stopped early — never looked at 5..8

The intermediate-collect version can’t do this: collect() always drains the whole iterator, so it would have visited all eight elements before take ever saw one.

When you genuinely do need a Vec

The point isn’t “never collect” — it’s “don’t collect between steps.” Collect once, at the end, when you actually need an owned, reusable collection:

1
2
3
4
5
6
7
8
9

let words = ["fast", "lazy", "fused", "iter"];

let shouted: Vec<String> = words
    .iter()
    .filter(|w| w.len() == 4)
    .map(|w| w.to_uppercase())
    .collect();

assert_eq!(shouted, ["FAST", "LAZY", "ITER"]);

One collect, at the end, when the Vec is the actual result. Everything before it stays lazy and allocation-free.

with_capacity (bite 193) buys a buffer once instead of growing it repeatedly. But if you allocate a fresh String or Vec inside a loop, you throw that buffer away every iteration. .clear() resets the length to zero while keeping the capacity — so one allocation serves the whole loop.

A fresh allocation every iteration

It’s easy to declare the working buffer inside the loop. Each pass allocates a new heap buffer and drops it at the end of the iteration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

let lines = ["alpha", "beta", "gamma"];
let mut out = Vec::new();

for line in lines {
    let mut buf = String::new();   // new heap allocation, every iteration
    buf.push_str(line);
    buf.make_ascii_uppercase();
    out.push(buf.clone());
}

assert_eq!(out, ["ALPHA", "BETA", "GAMMA"]);

Three iterations, three allocate-then-free cycles for the scratch buffer. Scale that to a million lines and it’s a million wasted allocations.

.clear() keeps the capacity

Hoist the buffer out of the loop and clear() it at the top of each pass. clear() sets the length to 0 but leaves the allocated capacity in place, so after the first iteration the buffer is already big enough and never reallocates:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

let lines = ["alpha", "beta", "gamma"];
let mut out = Vec::new();
let mut buf = String::new();       // allocated once

for line in lines {
    buf.clear();                   // len -> 0, capacity untouched
    buf.push_str(line);
    buf.make_ascii_uppercase();
    out.push(buf.clone());
}

assert_eq!(out, ["ALPHA", "BETA", "GAMMA"]);

The contract is the whole point — clear drops the contents but not the buffer:

1
2
3
4
5
6
7

let mut s = String::with_capacity(64);
s.push_str("hello");
let cap = s.capacity();

s.clear();
assert_eq!(s.len(), 0);            // empty again
assert_eq!(s.capacity(), cap);     // ...but the buffer is still there

The read-into-a-reused-buffer pattern

This shows up constantly when reading input. BufRead::read_line appends to the buffer you give it, so the idiomatic loop clears one String each pass instead of allocating a new one per line:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

use std::io::BufRead;

let input = "12\n34\n56\n";
let mut reader = std::io::BufReader::new(input.as_bytes());

let mut line = String::new();      // one buffer for every line
let mut sum = 0i64;

loop {
    line.clear();                  // required — read_line appends
    let n = reader.read_line(&mut line).unwrap();
    if n == 0 {
        break;                     // 0 bytes read == EOF
    }
    sum += line.trim().parse::<i64>().unwrap();
}

assert_eq!(sum, 102);

The same trick works for any scratch Vec — clear() it at the top of the loop and reuse the capacity for the next batch:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

let mut scratch: Vec<u8> = Vec::new();
let mut total = 0;

for chunk in [&[1u8, 2, 3][..], &[4, 5], &[6]] {
    scratch.clear();
    scratch.extend_from_slice(chunk);
    total += scratch.iter().map(|&b| b as u32).sum::<u32>();
}

assert_eq!(total, 21);

Reach for a fresh Vec/String only when you actually need to keep each result. When the buffer is just scratch space, allocate it once, clear() it, and let the loop run free.

A Vec you push into one element at a time doesn’t grow one element at a time — it doubles, copying every existing item to a fresh allocation each time it outgrows its buffer. If you already know how many items are coming, Vec::with_capacity buys the whole buffer once.

The hidden cost of push

Vec::new() starts with zero capacity. As you push, it reallocates geometrically — roughly doubling each time — and every reallocation copies all existing elements into the new, larger buffer. Fill a vector with 1000 items and you pay for that copying around ten times over:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

let mut v = Vec::new();
let mut reallocs = 0;
let mut last_cap = v.capacity();

for i in 0..1000 {
    v.push(i);
    if v.capacity() != last_cap {
        reallocs += 1;        // the buffer just moved
        last_cap = v.capacity();
    }
}
// ~10 reallocations, each copying everything built so far
assert!(reallocs >= 8);

In a hot loop, that churn is pure waste: allocate, copy, free, allocate bigger, copy again.

Reserve the space once

If you know the final size, hand it to Vec::with_capacity. The buffer is allocated a single time, and push never has to move it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

let mut v = Vec::with_capacity(1000);
let mut reallocs = 0;
let mut last_cap = v.capacity();

for i in 0..1000 {
    v.push(i);
    if v.capacity() != last_cap {
        reallocs += 1;
        last_cap = v.capacity();
    }
}
assert_eq!(reallocs, 0);       // zero — the buffer never moved

Capacity is not length: with_capacity(1000) gives you room for 1000 items but the vector is still empty (len() == 0) until you push.

Already have a Vec? Use reserve

When the vector exists and you’re about to add a known number of elements, reserve grows the buffer ahead of time without touching the contents:

1
2
3
4
5

let mut v = vec![1, 2, 3];
v.reserve(100);               // ensure room for 100 *more*

assert!(v.capacity() >= 103);
assert_eq!(v.len(), 3);       // still 3 elements — only capacity changed

Use reserve before a batch of pushes; use reserve_exact when you want the buffer sized precisely, with no geometric slack.

collect often does this for you

Iterators expose a size_hint, so collecting from a sized iterator already reserves the right capacity — no manual call needed:

1
2

let squares: Vec<i32> = (0..1000).map(|x| x * x).collect();
assert_eq!(squares.len(), 1000);

The win is biggest exactly where it matters: tight loops building large vectors. If you can name the size, name it once and let push run free.