trivial Java build fails on NixOS with rules_java 9.0.0+

[lucasbergman]

Describe the bug
A minimal Java build fails on NixOS x86-64 with rules_nixpkgs toolchain setup with the latest rules_java.

To Reproduce
See lucasbergman/rules-java-bug.

$ git checkout main
# This build (with patches; see archive_override in MODULE.bazel) succeeds
$ nix develop --command bash -c 'bazel clean --expunge && bazel build --config=nix //:hello_world_jar'
$ git checkout main~1
# This build (without patches) fails
$ nix develop --command bash -c 'bazel clean --expunge && bazel build --config=nix //:hello_world_jar'
ERROR: /home/lucas/.cache/bazel/_bazel_lucas/0e86df1e1cb94f04b9b2f0aabb03542e/external/rules_java+/toolchains/BUILD:344:14: Extracting interfaces from external/rules_java+/toolchains/platformclasspath_unstripped.jar failed: (Exit 127): ijar failed: error executing Ijar command (from target @@rules_java+//toolchains:platformclasspath) external/rules_java++toolchains+remote_java_tools_linux/java_tools/ijar/ijar bazel-out/k8-fastbuild/bin/external/rules_java+/toolchains/platformclasspath_unstripped.jar ... (remaining 3 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Could not start dynamically linked executable: external/rules_java++toolchains+remote_java_tools_linux/java_tools/ijar/ijar
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld

Analysis

The failure is that the bootclasspath rule in rules_java started using ijar in version 9.0.0 to save space and build time. The ijar program is pointed at @rules_java//toolchains:ijar which resolves to a pre-built binary on Linux x86-64; this can't be configured. My patch short-circuits that to build ijar from source.

There is a similar failure with the singlejar binary used from the java_single_jar rule. In rules_java there are three toolchains: bootstrap, compilation, and runtime. The provider associated with the compilation toolchain already has a singlejar executable hanging off of it, and it's correctly configured by rules_nixpkgs_java. So to fix singlejar, I just plugged the compilation toolchain into the java_single_jar rule, and things worked as expected.

Questions

1. Surely this must happen all the time: If @rules_nixpkgs_core//platforms:host is Linux, then different rule sets will try to use Linux binaries. What do you folks usually do in that situation? Upstream fixes? Should the host platform be marked as not-Linux somehow? Is there some config I'm holding wrong?
2. I have only a passing familiarity with Bazel toolchains (in spite of being a Google employee). Does the fix for singlejar look reasonable? I could probably do a similar thing with ijar, but it would be more invasive since the bootstrap toolchain would need a new attr on its provider. Is there some better or easier fix I haven't thought of?

[lucasbergman]

Hi, friends, is anyone able to weigh in?

[lucasbergman]

Randomly pinging @z8v who has been committing a lot to this repo lately. (Sorry.) Any thoughts?

[avdv]

Hi @lucasbergman,

I'm not an expert on this, but somehow forcing the tool to be built from source seems a viable option. The best thing wold be if upstream would use static binaries but... 🤷

I think what most of us do is to use an FHS env which makes dynamic binaries work, e.g. nix run github:malt3/bazel-env#bazel-env provides a dev shell with non-nix Bazel wrapped inside such an environment.

[lucasbergman]

Hi @avdv thanks for getting back to me. It sounds like running builds purely on NixOS (with no artificial FHS) is a quixotic thing to do? If so, fair enough, I won't worry about it too much. I'll see if the rules_java folks accept the singlejar fix upstream, since that one seems like an obvious bug - why not use the tool from a toolchain tool if it already has one? - but that's probably about it for now.