Articles / Rust

727 words

  • rust
  • tokio
  • async
  • concurrency
  • scheduling
  • debugging

Why Your Async Task Never Yields and Starves Its Neighbours

An async task does not get preempted. If it never reaches an .await that returns Pending, it keeps its thread until it finishes, and every other task queued on that thread just waits. No error, no warning, only a timer that fires seconds late.

This is cooperative scheduling doing exactly what it says. The surprise is how easy it is to write a task that never cooperates, including tasks full of .await calls.

A ticker that stops ticking

Here is the smallest reproduction I can think of. One runtime thread, one task printing a tick every 100ms, and one task doing plain arithmetic.

use std::hint::black_box;
use std::time::Duration;

fn main() {
    let rt = tokio::runtime::Builder::new_current_thread()
        .enable_time()
        .build()
        .unwrap();

    rt.block_on(async {
        tokio::spawn(async {
            loop {
                tokio::time::sleep(Duration::from_millis(100)).await;
                println!("tick");
            }
        });

        let hog = tokio::spawn(async {
            let mut n: u64 = 0;
            for i in 0..4_000_000_000u64 {
                n = n.wrapping_add(black_box(i));
            }
            n
        });

        println!("result: {}", hog.await.unwrap());
    });
}

Run it in release mode and you get silence for a few seconds, then result: .... The ticker only gets a turn once the hog returns. (black_box stops the optimiser folding the loop into a constant.)

Why the scheduler can't step in

A future is a state machine, and the runtime moves it forward by calling poll. Control only returns to the runtime when poll returns, and an async block returns early only at an await point whose inner future says Pending.

The for loop above has no await at all, so one call to poll runs the whole thing. The runtime is not ignoring the ticker; it is simply not running, because your code is on the stack.

Quick detour: awaits that don't yield

Hang on, this gets stranger. Awaiting something does not mean you yielded. If the future is already ready, the await completes immediately and the task carries straight on.

Picture a task draining a channel that a fast producer keeps full. Every recv().await finds a message waiting, so none of them returns Pending, and the task could spin forever.

Tokio guards against this for its own resources. Each task gets a budget of operations per poll (128 at the time of writing, but treat that as an implementation detail), and tokio channels, sockets and timers deliberately report Pending once it runs out. That forces a trip back to the scheduler.

The catch: the budget only counts tokio's own operations. Pure computation, a std channel, a ready future from another library, or a hand-rolled poll never touches it.

Three fixes, in order of preference

  1. Move CPU-heavy work off the async threads with tokio::task::spawn_blocking (or a rayon pool), then await the handle.
  2. If the work must stay async, break it into slices and call tokio::task::yield_now().await between them.
  3. For loops that already use tokio resources, check that the budget is doing its job before adding anything by hand.

The first is almost always right. Here is the hog rewritten:

let hog = tokio::task::spawn_blocking(|| {
    let mut n: u64 = 0;
    for i in 0..4_000_000_000u64 {
        n = n.wrapping_add(std::hint::black_box(i));
    }
    n
});
println!("result: {}", hog.await.unwrap());

Now the ticker keeps its rhythm, because the blocking pool is a separate set of threads that the async scheduler does not depend on.

When the work really must interleave, slice it:

let mut n: u64 = 0;
for i in 0..4_000_000_000u64 {
    n = n.wrapping_add(std::hint::black_box(i));
    if i % 1_000_000 == 0 {
        tokio::task::yield_now().await;
    }
}

The interval is a trade-off. Yield too rarely and latency suffers; too often and you pay a trip through the scheduler for nothing. Pick it by measuring how long a slice takes, aiming for well under a millisecond or so of work between yields.

The multi-threaded runtime only hides it

With the default multi-threaded runtime, one hog occupies one worker and the others carry on, so the symptom is mild: some requests are slow, most are fine. That makes it harder to spot, not safer.

Put as many hogs in flight as you have workers, say one expensive request per core during a traffic spike, and the whole runtime stalls exactly like the single-threaded demo. Load tests that use cheap requests will never show it.

Finding the culprit

The tell-tale signs are timers firing late, health checks timing out while CPU sits at 100% on one core, and latency that scales with the size of one particular input. Tokio's tokio-console reports tasks with long poll times, which points straight at the offender.

A cheap manual check is to time each poll-heavy section with Instant::now() and log anything over a few milliseconds. If a task's poll takes longer than your timer resolution, every neighbour feels it.

For a related symptom with a different cause, a blocking call rather than a long loop, see the earlier post on blocking calls in async Rust tasks. The cure is the same family: do not hold the thread.

← All articles