Articles / Rust

1005 words

  • rust
  • tokio
  • async
  • concurrency
  • spawn-blocking
  • debugging

Why a Blocking Call in an async Rust Task Stalls Other Requests

You add one innocent line to a Tokio handler, say a synchronous file read or a CPU-heavy hash, and suddenly requests that have nothing to do with it start timing out. Nothing panics and nothing logs an error. The server just goes quiet for a while and then carries on.

The cause is simple: an async task only gives up its thread when it hits an .await. If it never does, nothing else on that thread runs. Let's reproduce it properly, because the symptoms differ depending on the runtime flavour.

Reproducing it on a single thread

The cleanest demo uses Tokio's current-thread runtime, where there is exactly one thread and no escape. A ticker task prints every 100ms, and a second task does a blocking sleep partway through.

use std::time::{Duration, Instant};

#[tokio::main(flavor = "current_thread")]
async fn main() {
    let start = Instant::now();

    let ticker = tokio::spawn(async move {
        for _ in 0..5 {
            tokio::time::sleep(Duration::from_millis(100)).await;
            println!("tick at {:?}", start.elapsed());
        }
    });

    let hog = tokio::spawn(async {
        tokio::time::sleep(Duration::from_millis(250)).await;
        // Blocks the whole thread, not just this task.
        std::thread::sleep(Duration::from_secs(2));
    });

    let _ = tokio::join!(ticker, hog);
}

The first two ticks arrive at roughly 100ms and 200ms. Then the hog wakes at 250ms, parks the only thread for two seconds, and the third tick, due at 300ms, turns up at about 2.25s. The remaining ticks follow at their normal spacing, as if nothing happened.

Why the runtime cannot just interrupt it

Quick detour, because this is the bit that makes it click. An async function compiles to a state machine, and the executor drives it by calling poll. That call is an ordinary function call: the executor is on the stack beneath your code, and it only gets control back when poll returns.

Returning happens at an .await that is not ready. There is no timer interrupt and no preemption, as an OS scheduler would use for threads. Async Rust is cooperative scheduling, and std::thread::sleep is a task refusing to cooperate.

(Tokio has a per-task "budget" that forces a yield from busy Tokio resources such as sockets and channels. It only triggers at await points inside Tokio's own types, so it cannot help with a loop that never awaits.)

What changes on the multi-threaded runtime

The default #[tokio::main] runtime is multi-threaded, with one worker per CPU core by default. One blocked worker is survivable: other workers keep running tasks, and idle ones can steal queued work. So a single slow request often looks fine in testing.

Production is different. The failure mode looks like this:

  • One blocking request ties up one worker, and throughput drops by a core's worth.
  • Several arrive together and each takes a worker.
  • Once as many are blocked as there are workers, nothing runs. Health checks fail, timers do not fire and the load balancer pulls the instance.

That is why it feels like unrelated requests "randomly" stall. They are not unrelated; they share a small pool of threads with the culprit.

What counts as blocking

It is wider than sleep. Anything that holds the thread without awaiting qualifies:

  • Synchronous std::fs calls, std::net sockets, and blocking database drivers.
  • Waiting on a std::sync::Mutex that another thread holds for a long time.
  • CPU-bound work: password hashing, compression, big JSON parses, regex over huge inputs.
  • std::sync::mpsc::Receiver::recv or JoinHandle::join.

A rough rule from the Tokio maintainers is that code should not run for more than something like 10 to 100 microseconds between awaits. That is guidance, not a hard limit, but it tells you a bcrypt-style hash at tens of milliseconds is miles over it. Alice Ryhl's post Async: What is blocking? is the clearest write-up I know.

The fixes, in order of preference

  1. Use the async version if one exists: tokio::fs, tokio::net, tokio::time::sleep, tokio::sync::Mutex when the guard must be held across an await.
  2. Move it to spawn_blocking when the work is blocking I/O or short CPU work.
  3. Use a dedicated pool such as rayon, or your own thread, for sustained CPU-bound work.

Here is the earlier hog fixed with spawn_blocking:

use std::time::Duration;

async fn hash_password(pw: String) -> Result<String, tokio::task::JoinError> {
    tokio::task::spawn_blocking(move || {
        // Stand-in for a deliberately slow, synchronous hash.
        std::thread::sleep(Duration::from_millis(80));
        format!("hashed:{}", pw.len())
    })
    .await
}

The closure runs on a separate pool of threads reserved for blocking work, so the async workers stay free to poll everything else. The calling task awaits the JoinHandle, which is a perfectly good await point.

Gotchas with the fixes

spawn_blocking is not a magic bin. The pool is large (its default cap is in the hundreds of threads) but finite, and a closure that has started cannot be cancelled. Dropping the future or hitting a timeout abandons the result, but the thread keeps going until the closure returns. At runtime shutdown Tokio waits for these too, unless you use shutdown_timeout.

For heavy CPU work, a bounded rayon pool plus a tokio::sync::oneshot channel to carry the result back gives you a hard limit on concurrency. Otherwise a burst of hashing requests spawns hundreds of CPU-bound threads that fight each other.

Then there is tokio::task::block_in_place. It tells the multi-threaded runtime "this worker is about to block, hand its queue to someone else". It avoids moving data into a 'static closure, which is handy. It panics on the current-thread runtime, though, and it still pins one worker for the duration, so it is a convenience rather than a cure.

Finding the culprit in a running service

If you inherit a service with this problem, look for symptoms before reading code:

  • Latency spikes that affect every endpoint at once, not just one.
  • Low CPU on an apparently stuck process (blocked on I/O or a lock) or exactly one core pegged (a CPU-bound hog).
  • Timers firing late: add a task that sleeps 10ms in a loop and logs how late it woke. Drift is a direct measure of how long your workers are held.

tokio-console goes further and flags tasks with long poll times, though it needs the tokio_unstable cfg and the tracing instrumentation turned on. The lateness loop costs nothing, and I would add it to anything async that matters.

One last point worth keeping: the compiler cannot warn you here. A blocking call inside an async fn type-checks, runs and passes tests that only ever send one request at a time. The only safe habit is to ask, for every non-await call in a handler, how long it can hold the thread.

← All articles