Relative git flake inputs `git+file:relative/path`

[roberth]

Is your feature request related to a problem?

• • Nix currently resolves git+file:relative/path according to the process working directory instead of the flake.nix base directory. This is unreliable. Evaluation should be invariant under the process working directory, or it would violate the ability to be used as a flake input, among other things.
  • fetchTree does not have a base directory argument (and no implicit argument either, because that'd also be bad)
• Therefore fetchTree does not have the information to resolve relative paths.

Solution for Flakes users

Users will be able to use a relative path reference since Nix 2.26 and they can enable submodules declaratively, planned for release in 2.27

• Improve support for relative path inputs #10089
• Add inputs.self.submodules flake attribute #12421

{
  # For Nix 2.27:
  inputs.self.submodules = true;
  # instead of: inputs.foo.url = "git+file://./my-submodule";
  inputs.foo.url = ./my-submodule;
}

For Nix 2.26, instead of inputs.self.submodules, use:

nix build .?submodules=true#my-package

Proposed solution (for fetchTree)

• Disallow git relative path inputs in fetchTree
• Either
  • Resolve git relative path inputs in call-flake.nix (with a primop that does it? - unclear if that's future proof enough for reproducibility)
  • add a baseDirectory argument to fetchTree

Alternative solution

Forbid altogether. This blocks some users:

• Flake input git+file:./${submodule} no longer works #9708

Additional context

• fetchers/git: make relative path absolute for local repo #12107
• Warn against the use of relative 'git+file:' flake inputs #12277

---

Add 👍 to issues you find important.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-01-22-nix-team-meeting-minutes-210/59239/1

[alxistr]

I saw warnings like warning: Fetching Git repository 'file:../vim', which uses a path relative to the current directory. This is not supported and will stop working in a future release. See https://github.com/NixOS/nix/issues/12281 for details. but I don't get what I have to do about it. What's call-flake.nix, what's fetchTree, and I guess there are nothing comprehensible in the manual(s) as usual? Where comes this necessity to constantly broke things?

[roberth]

This use case was never intentionally supported, and we are working towards a situation where you don't have to specify such inputs at all, and you can just refer to them by relative path value, e.g. inputs.foo.url = ../vim;.

Thanks to @edolstra's implementation work on #10089 and #12421, you will be able to simplify your flake.

Flakes are an experimental feature, because it was originally developed by @edolstra with a limited budget that did not cover the project's completion. Its experimental release caused a lot of user interest, but only limited investment since its initial creation. (Hence why some users, who unfortunately feel powerless, have resorted to complaining instead of helping out an open source project in need.)

It is experimental exactly because we discover situations like this, where users find workarounds that are broken, and which are not feasible to support in a form that resembles the workaround.

call-flake.nix, fetchTree

These are implementation details of flakes, where fetchTree is also an independent feature that can be used in the Nix language independently of flakes. Since we are dealing with the complexity of a domain-specific programming language, it was not feasible for us to hide these, and only inform you about the upcoming change in terms of directly-provided user input.

You may ignore this warning.
I suggest we demote it to a mere notice, because it is not immediately actionable.

We appreciate your patience and contribution while we work towards flakes features that we can support sustainably into the future, as that is an important promise of the (stable parts of the) Nix project.

[roberth]

I've updated the issue description to add a solution specifically for flakes users. Feel free to ask me if anything is still unclear.

[alxistr]

Thank you very much for clarification. Indeed, I recall there was a problem with relative paths, which is why I ended up with git+file://. I was just afraid that relative paths would be completely removed, and it would complicate the workflow. Glad that's not going to happen.

[donn]

I'm having trouble with the solution for 2.27-- because now any derivations would form a dependency on the flake's copy in the store. Which means any modification to the flake recompiles all derivations written for the submodules. This intuitively makes sense, but was not an issue when using url = "git+file:<path to submodule>?submodules=1#";

[eclairevoyant]

Which means any modification to the flake recompiles all derivations written for the submodules.

That moreso sounds like you need to fix the src for said packages.
See https://nix.dev/tutorials/working-with-local-files

That's also entirely unrelated to this issue.