CI: Nix store corruption causing failures on aarch64-linux and darwin runners #2530
Description
Summary
Multiple CI jobs are failing with error: path '/nix/store/...' is not valid during Nix evaluation. This affects 6 jobs across aarch64-linux and both darwin platforms.
Run: https://github.com/cachix/devenv/actions/runs/22491993839
Affected Jobs
| Job | Platform | Missing store path |
|---|---|---|
| rabbitmq | aarch64-linux | meson-add-options-for-tests-installation-dirs.patch |
| phoenix | aarch64-linux | locale-binary-path.patch |
| terraform | aarch64-linux | setup.sh |
| scala | aarch64-linux | fix-library-path-jdk11.patch |
| extended-tests | aarch64-darwin | cryptonite-remove-argon2.patch |
| extended-tests | x86_64-darwin | devenv binary itself missing from store |
Error Pattern
All failures follow the same pattern: Nix evaluation fails because a store path referenced during derivation evaluation no longer exists. Example:
error: path '/nix/store/c3xkmvw895lg046smm76zc1gxx771dsj-setup.sh' is not valid
Some also show linking errors:
linking '/nix/store/.../SECURITY.md' to '/nix/store/.links/...': error: path '/nix/store/...' is not valid
Likely Cause
Nix store garbage collection running between build and test phases, or binary cache misses causing invalid store references. The aarch64-linux runner seems particularly affected (4 out of 6 jobs).
Suggested Investigation
- Check if GC is running on CI runners between job steps
- Verify binary cache availability for affected paths
- Consider pinning GC roots for intermediate build outputs