Add a virtual package to indicate Relocatability

[dra27]

This PR proposes to introduce a single virtual package relocatable.packages. The idea behind this package is that any package which is not relocatable (that is, cannot be copied to another switch without patching) can conflict this package. The PR adds this conflict to all compilers < 5.5 and all versions of ocamlfind and ocamlbuild.

This does not affect availability on OCaml 5.5+ - the idea is that a user can choose to create a switch which requires relocatable packages by including the relocatable package in the switch's invariant. In particular, I anticipate Dune package management (which requires packages to be relocatable) to use this "under-the-hood" to exclude non-relocatable versions of packages. For example, this still works:

$ opam switch create success --repos=dra27-relocatable-packages ocaml.5.5.0 ocamlbuild

<><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml" {= "5.5.0"} "ocamlbuild"]

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>

but this does not:

$ opam switch create failure ocaml.5.5.0 relocatable ocamlbuild

<><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml" {= "5.5.0"} "relocatable" "ocamlbuild"]
[ERROR] Could not determine which packages to install for this switch:
  * Incompatible packages:
    - (invariant) → ocamlbuild
    - (invariant) → relocatable
    You can temporarily relax the switch invariant with `--update-invariant'

[dra27]

(I've cancelled the Windows CI run as it will just be an enormous waste of time!)

[avsm]

Should this package be ocaml-relocatable to fit in with ocaml-variants etc, or does it cover all issues of relocatability beyond just compilation (such as runtime relocatability)?

[dra27]

It's intended to be all issues, indeed. Although can I double-check that we're meaning the same thing with runtime relocatability? The meaning I have here is that any package whose artefacts rely on the switch location (i.e. the prefix) should conflict the "relocatable" package, indicating that the artefacts cannot be copied or moved to another location (i.e. cloned to a different switch)

[dra27]

(in passing, opam-ci will have some useful results: the lint check matters; the build failures in compilers should be down to build errors, and not a failure to try to install the package; the revdeps testing is irrelevant)

[avsm]

I haven't thought about what I mean by relocatability at all, but I think you have :-) It might be worth refining the definition in the opam file description to be clearer:

This virtual package is used to specify switch and package requirements for
Relocatable packages. A package is Relocatable if its binaries may be copied to
another switch, rather than rebuilding the package for that different switch.
A package which is not Relocatable should include "relocatable" in its conflicts field.

Specifically, do "binaries" also mean any binary artefacts (libraries? cmx/t/etc), and also config files (findlib.conf) that may encode paths.

It's probably worth considering if this package should be one specialised to the compiler (ocaml-relocatable) and another for general relocatability.

[dra27]

It's probably worth considering if this package should be one specialised to the compiler (ocaml-relocatable) and another for general relocatability.

I'm not sure that having two of them would add anything from a users' perspective - the definition in https://github.com/ocaml/RFCs/blob/master/rfcs/relocatable.md#what-is-relocatable applies to any package modulo alpha-substituting "installed package" for "compiler distribution" and "installed artefacts" for "binaries". There's nothing special about the compiler packages from a relocatable perspective.

[avsm]

I think we should get this in sooner rather than later, given it's a giant diff. Do you want to convert this to a PR @dra27 ?

[hannesm]

While I understand that this may be useful, I'm curious whether the "dune package management" developers actually have this package in their roadmap, or have another way to figure this out?

Another question is that with this PR you're adding a trillion conflicts to a lot of packages -- will this lead to rebuilds all over the opam switches? And why put the conflicts into the ocaml compiler packages, instead of specifying your "relocatibility" package to be in conflict with ocaml < 5.5 (and ocamlbuild/ocamlfind < whatever version)?

[dra27]

I don’t know with regards to Dune p.m. - it’s an example of it. It’s a bit of a chicken-and-egg problem, right?

The conflict won’t cause rebuilds (I can never remember if it’s opam 2.1 or 2.2 which changed that).

Putting the conflict in the relocatable package I think is the wrong way round because that prevents anything else back-porting it or even pinning an alternate version.

[dra27]

Do you want to convert this to a PR @dra27 ?

I don’t follow, it is a PR?

[hannesm]

It is unclear how this is related to the new "compiler-cloning" package (merged with recent 5.5alpha).

putting conflicts in the eelocatable package would prevent backporting

Since one who "backports" or "pins" things can as well opam edit the proposed relocatable package, they can modify the conflicts.

I lack the technical details (a writeup) what it means for opam or dune package management, so how they'd like to deal with relocatable OCaml compilers and what their needs are.

I won't stand in the way and let the opam-repository maintainers maybe with input from opam and dune package teams decide what to do with this PR.

[avsm]

@dra27 wrote:

Do you want to convert this to a PR @dra27 ?
I don’t follow, it is a PR?

Sorry, I thought it was a draft PR before, but it's a full one!

[avsm]

@hannesm wrote:

It is unclear how this is related to the new "compiler-cloning" package (merged with recent 5.5alpha).

That seems to be answered in the compiler-cloning package description, specifically:

A package which is not Relocatable should include "relocatable" in its conflicts field.

Which also answers 'why conflicts in the compiler packages', so I'm happy with this. I'd like one other approval before merging this though...

[avsm]

@mseri are you ok with this one?