Leverage Docker for building ES source artifacts

[dliappis]

Problem statement

Currently we rely on the local machine having the right build JDK for Rally to build Elasticsearch from a source revision.

This complicates benchmark environments as it implies that target nodes need to meet Elasticsearch build requirements namely having various JDK versions installed (and keeping them up to date as build JDK requirements for Elasticsearch change over time).

Proposal

Instead of requiring a proper Java environment for Rally to build ES artifacts from source, build requirements could be reduced down to only one: just ensuring Docker is available.

At a high level:

• Rally will use the eclipse-temurin Docker image¹. The image tag needs to match the major version of bundled_jdk.
• The command invocations to build the ES artifacts will remain the same (see clean_command and system.build.command in rally-teams).
• To copy artifacts built inside the container² under the usual place that Rally stores source builds³, we should leverage bind mounts; @inqueue has provided an example in this gist which assumes Elasticseach has been checked out locally (not inside the container).
• It's an open question whether the Elasticsearch repo should be checked out in the host that Rally runs -- as we do now --, or inside the container. The former might speed up subsequent builds as git doesn't have to download afresh the repo in a new ephmeral container; the latter provides isolation and disk space savings for one-off benchmarks.
• For the new cluster management commands we need a few more things:
  • introduce a new build subcommand: currently we have download which doesn't support source builds and install which does both building and installation.
  • both the build and install subcommands should allow specifying the CPU architecture too, as typically for large cluster setups building happens on the loaddriver and artifacts get copied to the target nodes (which can a high number of nodes on different machines and of varying CPU architectures e.g. linux x86_64 / aarch64).

Implementation details/hints

• Most of the action happens in supplier.py, specifically Builder.run() which is called by ElasticsearchSourceSupplier.prepare() in supplier.py#create.

Footnotes

1. this is the JDK that is bundled with most published ES Linux artifacts nowadays e.g. the 8.1.0 Linux artifacts ↩

2. Elasticsearch creates a new directory for tgz artifacts so e.g. for Linux x86_64 the new artifact can be found under <es_checkout>/elasticsearch/distribution/archives/linux-tar/build/distributions/elasticsearch-8.2.0-SNAPSHOT-linux-x86_64.tar.gz ↩

3. see the property src.root.dir under [node] in ~/.rally/rally.ini; by default it is: ~/.rally/benchmarks/distributions/src) ↩