Create a working installer script for each commit/PR

[domenkozar]

Using https://github.com/actions/upload-release-asset we could create a prerelease release for each master commit
and upload all artifacts needed for the installer script to work.

The downside is that this wouldn't work for PRs (not that big of a deal) and that release page would quickly get polluted while there's no way to filter out prereleases.

[zimbatm]

One possible approach would be to host nar-serve so that both the install script and the release tarball can be pulled directly from the cache.

[domenkozar]

That or we setup Cachix and get it for free :)

[edolstra]

Given that the Hydra nix/master jobset already does this I'm not sure it makes sense to put a lot of effort into this.

[zimbatm]

Good point. What is the endpoint that can be used on Hydra?

[domenkozar]

I've tried that first, but the installer has wrong urls for tarballs and there's no way to get hydra build url as an input.

[zimbatm]

It would be possible if there was a Hydra end-point that used the derivation hash as a parameter. Then it would be possible to generate a second install script automatically.

[garbas]

Another option could be to extend the install script to accept additional argument/option that could provide custom download url (with a hash in the url, eg. file:///home/user/result/nix.tar.bz2#), then testing would be much easier and could even work for PRs.

[domenkozar]

The installer script needs a custom url with any of the solutions.

But you can't pass anything useful with hydra since you don't know the build url.

Using Cachix we have all needed parameters

Using GitHub releases you get the url as a step output.

[zimbatm]

The problem with Hydra is that each derivation gets a new build ID so there is no unique prefix URL that can be used for all the architectures. Unless there is a different endpoint that I don't know about.

[domenkozar]

Yes, there would need to be a new endpoint. That's why I suggested GitHub releases since I know that will be easier to settle on than to start using Cachix.

[zimbatm]

I created https://github.com/numtide/nix-flakes-installer a while back because it's easier than to have an argument. So another route is to automate the process over there and basically poll for new releases.

[edolstra]

Maybe I'm missing some context here but why do we need github releases for unreleased Nix versions?

[zimbatm]

That's an implementation detail. The goal is to allow to install arbitrary versions of Nix without having to wait for a release.

[abathur]

Something like this (but with PR support--it would be OK if this was conditionally gated by a maintainer-applied tag or something) would have been a big help with the installer update I'm working on in #3996.

There are 2 specific things I like here:

1. This seems like a first-step to having CI install tests running against master (whether per-commit, or just on a daily/hourly schedule). When I cloned Nix to make these fixes, I had to go through multiple rounds of fixes for macOS install breaks that had been introduced by other recent (IIRC at least 3?) merges/PRs. It would be great to catch those pre-merge, but even if they aren't blocking, this is still good information; a simple timeline showing when macOS installs started breaking would have saved me a lot of time figuring out that a merge to master had black-holed some fixes.
2. Having easy access to securely-hosted multi-arch installers that are correctly-generated against PRs lowers the bar for making well-tested fixes. This was exacerbated by work towards flakes shifting things around, but I wasted a lot of time fumbling/asking around and reading code just to figure out how to generate the single-arch tarball I needed to test my fixes, patch up an installer script for it, and now to privately/personally host those files so that others can test my changes (and I still don't know for sure my PR doesn't break Linux installs).

These are probably synergistic. An install/uninstall/reinstall test suite covering all ~official instructions and all of the weird install/uninstall problems people turn up in IRC/discourse/issues with is probably low leverage in isolation, but I imagine it becomes very high leverage once it starts catching breaks and validating fixes during PR review. As a bonus, the steps, conditions, and test cases it would collect with some bearing on reliably uninstalling Nix might simplify creating and testing an uninstall tool.

Edit: I guess https://github.com/nixbuild/nix-quick-install-action/blob/master/flake.nix and https://github.com/numtide/nix-flakes-installer/blob/master/update.rb are both taking bites out of this for different reasons.