How can I avoid running some tests in parallel?

Question

I have a collection of tests. There are a few tests that need to access a shared resource (external library/API/hardware device). If any of these tests run in parallel, they

Answer 1

Use the serial_test crate. With this crate added, you put in your code:

#[serial]

in front of any test you want run in sequentially.

Answer 2

You can always provide your own test harness. You can do that by adding a [[test]] entry to Cargo.toml:

[[test]]
name = "my_test"
# If your test file is not `tests/my_test.rs`, add this key:
#path = "path/to/my_test.rs" 
harness = false

In that case, cargo test will compile my_test.rs as a normal executable file. That means you have to provide a main function and add all the "run tests" logic yourself. Yes, this is some work, but at least you can decide everything about running tests yourself.

---

You can also create two test files:

tests/
  - sequential.rs
  - parallel.rs

You then would need to run cargo test --test sequential -- --test-threads=1 and cargo test --test parallel. So it doesn't work with a single cargo test, but you don't need to write your own test harness logic.

Answer 3

As mcarton mentions in the comments, you can use a Mutex to prevent multiple pieces of code from running at the same time:

use once_cell::sync::Lazy; // 1.4.0
use std::{sync::Mutex, thread::sleep, time::Duration};

static THE_RESOURCE: Lazy<Mutex<()>> = Lazy::new(Mutex::default);

type TestResult<T = (), E = Box<dyn std::error::Error>> = std::result::Result<T, E>;

#[test]
fn one() -> TestResult {
    let _shared = THE_RESOURCE.lock()?;
    eprintln!("Starting test one");
    sleep(Duration::from_secs(1));
    eprintln!("Finishing test one");
    Ok(())
}

#[test]
fn two() -> TestResult {
    let _shared = THE_RESOURCE.lock()?;
    eprintln!("Starting test two");
    sleep(Duration::from_secs(1));
    eprintln!("Finishing test two");
    Ok(())
}

If you run with cargo test -- --nocapture, you can see the difference in behavior:

No lock

running 2 tests
Starting test one
Starting test two
Finishing test two
Finishing test one
test one ... ok
test two ... ok

With lock

running 2 tests
Starting test one
Finishing test one
Starting test two
test one ... ok
Finishing test two
test two ... ok

Ideally, you'd put the external resource itself in the Mutex to make the code represent the fact that it's a singleton and remove the need to remember to lock the otherwise-unused Mutex.

This does have the massive downside that a panic in a test (a.k.a an assert! failure) will cause the Mutex to become poisoned. This will then cause subsequent tests to fail to acquire the lock. If you need to avoid that and you know the locked resource is in a good state (and () should be fine...) you can handle the poisoning:

let _shared = THE_RESOURCE.lock().unwrap_or_else(|e| e.into_inner());

If you need the ability to run a limited set of threads in parallel, you can use a semaphore. Here, I've built a poor one using Condvar with a Mutex:

use std::{
    sync::{Condvar, Mutex},
    thread::sleep,
    time::Duration,
};

#[derive(Debug)]
struct Semaphore {
    mutex: Mutex<usize>,
    condvar: Condvar,
}

impl Semaphore {
    fn new(count: usize) -> Self {
        Semaphore {
            mutex: Mutex::new(count),
            condvar: Condvar::new(),
        }
    }

    fn wait(&self) -> TestResult {
        let mut count = self.mutex.lock().map_err(|_| "unable to lock")?;
        while *count == 0 {
            count = self.condvar.wait(count).map_err(|_| "unable to lock")?;
        }
        *count -= 1;
        Ok(())
    }

    fn signal(&self) -> TestResult {
        let mut count = self.mutex.lock().map_err(|_| "unable to lock")?;
        *count += 1;
        self.condvar.notify_one();
        Ok(())
    }

    fn guarded(&self, f: impl FnOnce() -> TestResult) -> TestResult {
        // Not panic-safe!
        self.wait()?;
        let x = f();
        self.signal()?;
        x
    }
}

lazy_static! {
    static ref THE_COUNT: Semaphore = Semaphore::new(4);
}

THE_COUNT.guarded(|| {
    eprintln!("Starting test {}", id);
    sleep(Duration::from_secs(1));
    eprintln!("Finishing test {}", id);
    Ok(())
})

See also:

• How to limit the number of test threads in Cargo.toml?