Retrieve serverless build hash from nodes info API

[gbanasiak]

This PR adds retrieval of build hash for serverless clusters from nodes info API (_nodes?filter_path=**.build_hash).

The PR introduces 2 configuration settings under driver section:

• serverless.mode - equals true if Rally targets serverless cluster
• serverless.operator - equals true if Elasticsearch user has operator privileges

[inqueue]

I left one suggestion. Otherwise, another review round is not needed. LGTM

[ebadyano]

Thank you for the change! Does it make sense to add a test in driver_test.py?

[gbanasiak]

@ebadyano

Does it make sense to add a test in driver_test.py?

Sure, added in b9a2ab8. Please take a look.

[dliappis]

Thank you. This seems to work but I have some questions around populating the retrieved field in the remote metric store.

[dliappis]

What about the stats in the metric store?

There is some code in

rally/esrally/telemetry.py

Lines 1778 to 1781 in c3b04f4

	revision = client_info["version"].get("build_hash", distribution_flavor)
	# build version does not exist for serverless
	distribution_version = client_info["version"].get("number", distribution_flavor)
	self.metrics_store.add_meta_info(metrics.MetaInfoScope.cluster, None, "source_revision", revision)

and in my tests it results in

(correctly)

but

whereas I see in the logs Retrieved actual build hash [ad5c80e1b49f10b42d438253d7cc0b9753c156b9] from serverless cluster.

Additionally, shouldn't it also be collected by the node stats telemetry device?

[gbanasiak]

Telemetry device modifications are scoped separately in ES-6459, but after discussion with @dliappis we concluded the necessary changes to override build hash (revision) in ClusterEnvironmentInfo are small, so I went ahead and added da34a7d. PTAL.

Edit: Additional changes to telemetry devices will go into separate PRs.

[dliappis]

Thanks for iterating. LGTM.