ava@v5.3.1 not working on Node v21.x

[alfonsograziano]

Hi @sindresorhus, @novemberborn I noticed you're listed as maintainer for ava so I'd like to bring to your attention that ava@v5.3.1 tests have been failing across different systems on v21.x. Feel free to loop in more contributors that you think may help.

Here a failed run for reference:
https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/3318/testReport/

As you can see on this run jose is failing on two different nodes, but at the same time I see that a lot of nodes are skipped (["14", "ppc", "win32", "x86", "rhel", "aix", "ia32"]), so probably the problem can have a bigger scope. Can you please help us to debug the issue? I'm not sure if this is related to the new Node version or something inside the package.

In the last days we had timeout issues which has been fixed by #1003, but now this new issue looks unrelated to the timeout

[novemberborn]

@alfonsograziano the AVA tests that are failing have to do with deterministic snapshot generation. It seems in the commit-under-test, the snapshots are 8 bytes larger. This is unexpected. I'm not sure how to run AVA against this specific commit to figure out why though.

Scanning through the Jose failures it also looks like some binary mismatch. So this could be a genuine issue with this Node.js build.

[alfonsograziano]

Based on the Jose issue (#1011) the maintainer changed the package to make some tests non-deterministic. I'm not sure if this is the case.

You can get some other insights on Jose here: nodejs/node#50138

Can you please try to build Node.js locally and verify if you're able to reproduce the issue? Would be really helpful to understand if this is a problem with the package or a bug in the Node.js build

[SimenB]

Installing the nightly is probably easier than building from source, FWIW 🙂

[novemberborn]

I'll need some pointers on how to access the nightly, ideally using Volta.

[SimenB]

Never heard of volta, but: https://nodejs.org/download/nightly/

[novemberborn]

The citgm report is now under security embargo.

I ran AVA's tests using https://nodejs.org/download/nightly/v21.0.0-nightly20231009387e2929fe/ on macOS and they passed.

One thing that comes to mind is that the binary data that AVA's tests are failing for is gzip compressed so perhaps there's a change there?

[novemberborn]

This is the same issue as discussed in nodejs/node#50138. It'll be fixed in AVA 6 but tests will fail until that is out, and I cannot provide a timeline on that.

See also avajs/ava#3251.

[alfonsograziano]

Thanks a lot for the update :)
Will keep this issue open until AVA 6 gets released