106. Iterator::by_ref — Take Part of an Iterator and Keep the Rest

Called iter.take(n).collect() and watched the borrow checker swallow the whole iterator? by_ref lets you consume a chunk and keep iterating from where you stopped.

The trap: adapters consume their iterator

Most iterator adapters take self, not &mut self. The moment you write iter.take(2), you have moved iter into the new Take adapter. The original binding is gone:

1
2
3
4
5
6

let v = vec![1, 2, 3, 4, 5];
let mut iter = v.into_iter();

let _first_two: Vec<i32> = iter.take(2).collect();
// let rest: Vec<i32> = iter.collect();
// ^^ error: use of moved value: `iter`

So even though there are clearly elements left, the variable that points at them is no longer usable.

The fix: iter.by_ref()

Iterator::by_ref returns a &mut Self — and &mut I itself implements Iterator whenever I: Iterator. Adapters chained off by_ref() consume from the underlying iterator without moving it:

1
2
3
4
5
6
7
8

let v = vec![1, 2, 3, 4, 5];
let mut iter = v.into_iter();

let first_two: Vec<i32> = iter.by_ref().take(2).collect();
let rest: Vec<i32> = iter.collect();

assert_eq!(first_two, [1, 2]);
assert_eq!(rest, [3, 4, 5]);

Same iterator, two phases, no clones, no indexing.

A real use: parse a header, then a body

The pattern shines when the front of a stream uses different rules than the rest. Pull lines until you hit a blank one, then hand the same iterator to the body parser:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

let input = "Subject: hi\nFrom: a@b\n\nbody line 1\nbody line 2";
let mut lines = input.lines();

let headers: Vec<&str> = lines
    .by_ref()
    .take_while(|l| !l.is_empty())
    .collect();

let body: Vec<&str> = lines.collect();

assert_eq!(headers, ["Subject: hi", "From: a@b"]);
assert_eq!(body, ["body line 1", "body line 2"]);

Without by_ref, take_while would have consumed lines whole and the body would be unreachable.

Gotcha: take_while peeks one past

take_while reads the first non-matching element to know when to stop — and that element is gone from the underlying iterator. With by_ref you’ll see this directly:

1
2
3
4
5
6
7

let mut nums = (1..).into_iter();

let small: Vec<i32> = nums.by_ref().take_while(|&n| n < 3).collect();
let next = nums.next();

assert_eq!(small, [1, 2]);
assert_eq!(next, Some(4)); // 3 was eaten by take_while

If you need to keep the boundary element, reach for Peekable instead.

When to use it

Any time you want to apply an adapter to part of an iterator and continue afterwards: chunked parsing, header/body splits, “take the first N then process the rest”, consuming until a sentinel. by_ref() turns “I moved my iterator and now I can’t use it” into a single extra method call.