Alternative to full-string matching

[MauriceKayser]

The Rust compiler can emit error messages that include file paths as well as line and column numbers (e.g. of the standard library). Because these details can vary across environments and compiler versions, tests that rely on trybuild's current full-string matching become fragile and difficult to maintain. It would be very helpful if trybuild provided a more flexible way to handle such volatile output - for instance, by supporting substring matching like should_panic for normal Rust unit tests.

Example Rust Playground:

fn main() {
    trait A { fn a(); }
    impl<T> A for T where T: std::fmt::Display { fn a() {} }

    struct X;
    X::a();
}

Volatile part of the error message:

note: the trait `std::fmt::Display` must be implemented
    --> /playground/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:1006:1
     |
1006 | pub trait Display: PointeeSized {
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For this code snippet I have observed many different paths like:

C:\Users\Maurice.Kayser\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib/rustlib/src/rust\library\core\src\fmt\mod.rs:984:1
         /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:1186:1
          /playground/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:1006:1
            /playground/.rustup/toolchains/beta-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:1007:1
                                     /builddir/build/BUILD/rust-1.91.0-build/rustc-1.91.0-src/library/core/src/fmt/mod.rs:1006:1
                                              /rustc/1159e78c4747b02ef996e55082b704c09b970588/library/core/src/fmt/mod.rs:1006:1

Implementing an alternative to full-string matching might also help for #317.

[dtolnay]

As far as I can tell, this is effectively the same feature request as #317.

[peter-lyons-kehl]

How about a method similar to t.compile_fail(file_path), one that would not compare/store the compiler error output, but instead it would panic! with that output as the panic message? Then in our tests we could apply should_panic with its substring matching, please.

[peter-lyons-kehl]

And, (ideally in addition to the above error-message-as-panic, please): If the source path "generalization"/"portability" is difficult, could we at least have (an option of) no-path-normalization (possibly behind a feature flag)?

As a workaround to the full paths stored in .stderr files, I've hoped to be able to

1. create a symlink (for example: /tmp/my-crate) to link to the crate being tested.
2. cd /tmp/my-crate
3. TRYBUILD=overwrite cargo test
4. have GitHub Actions create the same symlink /tmp/my-crate and cd /tmp/mycrate before cargo test.

While that doesn't suit all use cases (Windows/systems with no symlinks etc.), it would suit the most. Unfortunately, the symlink gets resolved by trybuild. Could we have an option

• not to resolve symlinks, please? Or
• to provide a "base path" string (maybe via an environment variable) and to remove that string from error messages.

[dtolnay]

I would prefer not to support this in this crate. But it would be reasonable for someone else to maintain a more fully featured ui testing library that allows for more sophisticated comparison logic.