How can I create parameterized tests in Rust?

Question

I want to write test cases that depend on parameters. My test case should be executed for each parameter and I want to see whether it succeeds or fails for each parameter.

Answer 1

It's possible to construct tests based on arbitrarily complex parameters and any information known at build time (including anything you can load from a file) with a build script.

We tell Cargo where the build script is:

Cargo.toml

[package]
name = "test"
version = "0.1.0"
build = "build.rs"

In the build script, we generate our test logic and place it in a file using the environment variable OUT_DIR:

build.rs

fn main() {
    let out_dir = std::env::var("OUT_DIR").unwrap();
    let destination = std::path::Path::new(&out_dir).join("test.rs");
    let mut f = std::fs::File::create(&destination).unwrap();

    let params = &["abc", "fooboo"];
    for p in params {
        use std::io::Write;
        write!(
            f,
            "
#[test]
fn {name}() {{
    assert!(true);
}}",
            name = p
        ).unwrap();
    }
}

Finally, we create a file in our tests directory that includes the code of the generated file.

tests/generated_test.rs

include!(concat!(env!("OUT_DIR"), "/test.rs"));

That's it. Let's verify that the tests are run:

$ cargo test
   Compiling test v0.1.0 (...)
    Finished debug [unoptimized + debuginfo] target(s) in 0.26 secs
     Running target/debug/deps/generated_test-ce82d068f4ceb10d

running 2 tests
test abc ... ok
test fooboo ... ok

Answer 2

The built-in test framework does not support this; the most common approach used is to generate a test for each case using macros, like this:

macro_rules! fib_tests {
    ($($name:ident: $value:expr,)*) => {
    $(
        #[test]
        fn $name() {
            let (input, expected) = $value;
            assert_eq!(expected, fib(input));
        }
    )*
    }
}

fib_tests! {
    fib_0: (0, 0),
    fib_1: (1, 1),
    fib_2: (2, 1),
    fib_3: (3, 2),
    fib_4: (4, 3),
    fib_5: (5, 5),
    fib_6: (6, 8),
}

This produces individual tests with names fib_0, fib_1, &c.

Answer 3

My rstest crate mimics pytest syntax and provides a lot of flexibility. A Fibonacci example can be very neat:

#[cfg(test)]
mod test {
    use super::*;

    use rstest::rstest;

    #[rstest(input, expected,
    case(0, 0),
    case(1, 1),
    case(2, 1),
    case(3, 2),
    case(4, 3),
    case(5, 5),
    case(6, 8)
    )]
    fn fibonacci_test(input: u32, expected: u32) {
        assert_eq!(expected, fibonacci(input))
    }
}

pub fn fibonacci(input: u32) -> u32 {
    match input {
        0 => 0,
        1 => 1,
        n => fibonacci(n - 2) + fibonacci(n - 1)
    }
}

Output:

/home/michele/.cargo/bin/cargo test
   Compiling fib_test v0.1.0 (file:///home/michele/learning/rust/fib_test)
    Finished dev [unoptimized + debuginfo] target(s) in 0.92s
     Running target/debug/deps/fib_test-56ca7b46190fda35

running 7 tests
test test::fibonacci_test::case_1 ... ok
test test::fibonacci_test::case_2 ... ok
test test::fibonacci_test::case_3 ... ok
test test::fibonacci_test::case_5 ... ok
test test::fibonacci_test_case_6 ... ok
test test::fibonacci_test::case_4 ... ok
test test::fibonacci_test::case_7 ... ok

test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Every case is run as a single test case.

The syntax is simple and neat and, if you need, you can use any Rust expression as the value in the case argument.

rstest also supports generics and pytest-like fixtures.

---

Don't forget to add rstest to dev-dependencies in Cargo.toml.

Answer 4

Use https://github.com/frondeus/test-case crate.

Example:

#[test_case("some")]
#[test_case("other")]
fn works_correctly(arg: &str) {
    assert!(arg.len() > 0)
}

Answer 5

Probably not quite what you've asked for, but by using TestResult::discard with quickcheck you can test a function with a subset of a randomly generated input.

extern crate quickcheck;

use quickcheck::{TestResult, quickcheck};

fn fib(n: u32) -> u32 {
    match n {
        0 => 0,
        1 => 1,
        _ => fib(n - 1) + fib(n - 2),
    }
}

fn main() {
    fn prop(n: u32) -> TestResult {
        if n > 6 {
            TestResult::discard()
        } else {
            let x = fib(n);
            let y = fib(n + 1);
            let z = fib(n + 2);
            let ow_is_ow = n != 0 || x == 0;
            let one_is_one = n != 1 || x == 1;
            TestResult::from_bool(x + y == z && ow_is_ow && one_is_one)
        }
    }
    quickcheck(prop as fn(u32) -> TestResult);
}

I took the Fibonacci test from this Quickcheck tutorial.

---

P.S. And of course, even without macros and quickcheck you still can include the parameters in the test. "Keep it simple".

#[test]
fn test_fib() {
    for &(x, y) in [(0, 0), (1, 1), (2, 1), (3, 2), (4, 3), (5, 5), (6, 8)].iter() {
        assert_eq!(fib(x), y);
    }
}

Answer 6

EDIT: This is now on crates.io as parameterized_test::create!{...} - Add parameterized_test = "0.1.0" to your Cargo.toml file.

---

Building off Chris Morgan’s answer, here's a recursive macro to create parameterized tests (playground):

macro_rules! parameterized_test {
    ($name:ident, $args:pat, $body:tt) => {
        with_dollar_sign! {
            ($d:tt) => {
                macro_rules! $name {
                    ($d($d pname:ident: $d values:expr,)*) => {
                        mod $name {
                            use super::*;
                            $d(
                                #[test]
                                fn $d pname() {
                                    let $args = $d values;
                                    $body
                                }
                            )*
                        }}}}}}}

You can use it like so:

parameterized_test!{ even, n, { assert_eq!(n % 2, 0); } }
even! {
    one: 1,
    two: 2,
}

parameterized_test! defines a new macro (even!) that will create parameterized tests taking one argument (n) and invoking assert_eq!(n % 2, 0);.

even! then works essentially like Chris' fib_tests!, though it groups the tests into a module so they can share a prefix (suggested here). This example results in two tests functions, even::one and even::two.

This same syntax works for multiple parameters:

parameterized_test!{equal, (actual, expected), {
    assert_eq!(actual, expected); 
}}
equal! {
    same: (1, 1),
    different: (2, 3),
}

The with_dollar_sign! macro used above to essentially escape the dollar-signs in the inner macro comes from @durka:

macro_rules! with_dollar_sign {
    ($($body:tt)*) => {
        macro_rules! __with_dollar_sign { $($body)* }
        __with_dollar_sign!($);
    }
}

I've not written many Rust macros before, so feedback and suggestions are very welcome.