Use `patch` instead of `git apply`

[dinosaure]

See dinosaure/esperanto#52 for more context.

It appears that git apply makes quite a few assumptions about how the project is organised and whether or not it is a Git repository. During my research, I found that quite a few people were having problems with git apply for obscure reasons related to folders, whether or not they were in a Git repository, and possible correlations between .git/index (which tracks all files in the Git repository) and patches.
In short, it is quite "easy" to encounter a situation where git apply does not want to apply the patches for a multitude of reasons other than the fact that the patch cannot be applied.

For all these reasons (and the time spent by myself and @shym on this specific problem), it may be more interesting to use patch rather than git apply when applying patches to OCaml. Currently, patches are generated by git diff and are Git patches that are incompatible with patch, but it should not be difficult to modify the patches accordingly so that they are understood by patch.

The advantage is that patch is a little more predictable than git apply and does not require obscure information such as the .git/index or the fact that we are in a Git repository.

I have already encountered some problems in modifying and improving ocaml-solo5 and its workflow, specifically in this area. I didn't have time to pinpoint the issue at the time, but now I have.

[shym]

Thank you for the exploration and the explanations!
As far as I understood from discussions for opam, relying on patch opens its own can of worms (GNU vs non-GNU?). So I wonder if we should go with https://github.com/hannesm/patch 😅

[dinosaure]

Yes, it could nice to use https://github.com/hannesm/patch indeed.

[shym]

For the record, a possible alternate solution would be to create a variant of ocaml-src, say mirage-ocaml-src, to let opam handle the patches internally. As all our patches are now in trunk but for the default set to maximum 1 domain (we could patch this with a simple cp, say), we could have only 1 or 2 versions of mirage-ocaml-src to release.

[dinosaure]

I prefer using patches because it forces us to follow OCaml's development. Having a mirage-ocaml-src package gives us complete freedom to modify the compiler for better or for worse.

The advantage of patches is that we are always involved in the development of OCaml and can add changes that should become more subtle over time (and thanks to your work) in order to use $(uname -m)-solo5-none-static-{cc,ld}.

I think the right solution, as you mentioned, would be to use hannesm/patch, which we know behaves the same on all platforms (GNU and BSD) — and that's what OPAM currently uses.

[shym]

Glad you like the real patches! :-)
Have you (or someone else) started to try and use hannesm/patch?
(Otherwise I might have a go later this week if I can make some time for it.)

[dinosaure]

Have you (or someone else) started to try and use hannesm/patch?

I can not work this week on that but I can take a look next week. But if you have time to do that, feel free to do so 👍 .

[shym]

I finally stumbled on that bug and understood what is going on. The fix is just git --work-tree=. apply ... (at least, I hope)!
So now I wonder whether:

• to just fix the git call; pro: we can keep the git patches as they are, ie binary diffs for configure, which are a lot more compact (the size of the whole patch series goes from 150KB with binary diffs to 250KB without them)
• to go with a hannesm/patch-based solution; pro: using binary diffs is maybe not the best idea anyway; cons: it’s a lot less battle-tested, I guess.

[hannesm]

To me, it depends on your available time:

• patch is indeed used in opam since 2.4.0 -- and of course there it is only exposed to a certain kind of patches -- nevertheless I'd claim it will work fine, especially for git-generated patches
• using the line above, if this works well for @dinosaure as well, is imho fine
• with the just released 5.4.0, I remember you mentioned there's no more need to add patches - so we're fine without any patch (and can use some sed at the one place where it is necessary)

[dinosaure]

I think I still prefer the patch solution. I spent too much time to see an issue about git apply and I wouldn't like to spend more time on that. patch has the advantage to clarify what we would like to do in a more straightforward way. git apply and Git patches give to us the only advantage to reduce the size of the distribution but I'm not convinced that it's really important (a Git bundle of ocaml-solo5 is about 1M where a bundle of miou for instance is about ~4M — and I suspect that Git is much more smart to compress simple diffs than a binary diffs).

[shym]

• with the just released 5.4.0, I remember you mentioned there's no more need to add patches - so we're fine without any patch (and can use some sed at the one place where it is necessary)

There will no need in 5.5 but for the limit on the number of domains. There are still a couple of patches for 5.4.

[shym]

Related: the message of commit mirage/ocaml-unikraft@be383d5.
I’m in favour of switching to opatch, or whatever the name, but I think the workaround is good enough not to block on opatch being properly (tested and) released.

[hannesm]

addressed in #154