cargo bench will execute each target with the --bench flag. cargo test, however, does not pass that flag. The builtin libtest crate takes advantage of that by only running each benchmark exactly once and not gathering statistics if --bench isn't passed. That means cargo test --all-targets can be used to ensure that the benchmarks run successfully, and it's much faster than doing cargo bench.