libfetchers/tarball: Lock on effectiveUrl

[dramforever]

Basically, if a tarball URL is used as a flake input, and the URL leads
to a redirect, the final redirect destination would be recorded as the
locked URL.

This allows tarballs under https://nixos.org/channels to be used as
flake inputs. If we, as before, lock on to the original URL it would
break every time the channel updates.

[dramforever]

This feature is intended to support the use of good old https://nixos.org/channels with flakes, by using the HTTP redirect already present. A major question would be whether this redirection is actually part of the intended interface of channels.nixos.org.

See Discourse: https://discourse.nixos.org/t/future-of-channels-and-channels-nixos-org-in-a-flakes-world/11563, relevant parts copied here for convenience:

I feel that less reliance on GitHub for releases might be a good idea. If we decide to leave GitHub some day for some reason, it would be nice if we don't suddenly leave countless mentions of github:NixOS/nixpkgs lying around.

So maybe we should add a way to refer to good old channels in a Flake URL? Here's a possible way that I thought of:

1. When a user specifies an http/https URL (type tarball), and it leads to (possibly several) redirects, we instead record the final redirection destination in flake.lock.
2. When a flake.lock is consulted to download the tarball, by default the URL in flake.lock is used. (This seems to be already the case)

This way specifying:

inputs.nixpkgs.url = "https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz";

Would actually just work, as instead of failing with an invalid hash whenever nixos-unstable updates, it saves the redirected URL which points to a stable version. flake.nix would look something like this:

{
  "nodes": {
    "nixpkgs": {
      "locked": {
        "narHash": "sha256-N1qI50AkeTSBp1ffUCHrcK2re52wrn6euFFGGvFa2iw=",
        "type": "tarball",
        "url": "https://releases.nixos.org/nixos/unstable/nixos-21.05pre269929.ff96a0fa563/nixexprs.tar.xz"
      },
      "original": {
        "type": "tarball",
        "url": "https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz"
      }
    },
    // ...
}

This way, older versions of Nix seeing this new lock file would just behave as if someone used --override-input, and newer versions of Nix seeing the old lock file with the pre-redirection URL would simply migrate it over when nix flake update --update-input is used.

[edolstra]

Thanks, merged it. However, I don't think we should fetch the nixpkgs flake via channels.nixos.org, since it doesn't provide the Git revision that the git and github fetchers provide, or the ability to do nix flake clone to start hacking on the nixpkgs flake.

If we decide to leave GitHub some day for some reason, it would be nice if we don't suddenly leave countless mentions of github:NixOS/nixpkgs lying around.

This can be avoided by relying on the nixpkgs registry entry. Also, the registry allows rewriting flakerefs like github:NixOS/nixpkgs to arbitrary other locations, so we could do that if we ever switch.

[dramforever]

Yeah, it does seem to work fine, and given that a (non-indirect) flake url represents where the flake is, github: seems okay.

[johnae]

@edolstra @dramforever I think this broke downloading github releases as they redirect to signed url:s which expire at some point.

[dramforever]

Ah shoot... I'd be fine if this were reverted