Changes to build star tree in off heap

[bharath-techie]

Description

Sort and aggregation of segment documents when building star tree takes up lots of resources if they are stored in in-memory structures like Lists. So this issue tracks the off heap implementation where we store and retrieve the star tree documents in off heap method using temporary files.

• We use intro sorter to sort the segment documents off heap and we benchmarked it against quick sorter, external sorter etc and found to be equally resource efficient and faster
• We use temp directory to store the segment documents in the file initially and sort the documents off heap with an array of offsets getting swapped during the sort
• Then we build the star tree documents off heap as well by writing the star tree documents in file. There is complexity here as sometimes we need to read the previous docs created in the star tree file, so we close the file and create a new file and keep track of the docs in each of the file.

Related Issues

Resolves #14815

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for c459d9a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 551e941: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 853be87: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for fed49c9:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 0d753f2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 0a10ea9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 8d50c39: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.