[Feature Request] [S3 Repository Plugin]: Use S3CrtClient for higher throughput while uploading files to S3

[pranikum]

Is your feature request related to a problem? Please describe

We're currently using NettyNioAsyncHttpClient as part fo S3AsyncService to upload the translog/segments (RemoteStore) to S3 which is working fine. However we already have AwsCrtAsyncHttpClient available which improves the throughput (RemoteStore) during the uploads.

https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/http-configuration-crt.html

Describe the solution you'd like

We would like to explore and use AwsCrtAsyncHttpClient client for the s3 segments upload instead of NettyNioAsyncHttpClient

Related component

Plugins

Describe alternatives you've considered

No response

Additional context

No response

[pranikum]

During the perf testing we have seen an increase of 5-7% for indexing throughput improvement with CRT client. (Cluster with Remote store)
Tested under similar environment setting and can see the below gains for throughput

P50: Throughput

Netty: 205,273.8
CRT: 220,188.6

P100 Throughput

Netty: 215,113.7
CRT: 230,803.2

Based on the above result we are making CRT as default. Netty can be used by overriding the setting.