Use cluster default remote store path type during snapshot restore

[ashking94]

Description

This PR is a sub task of feature request #12567 and being tracked in meta issue - #12589.

In the prior PR #12607, we have introduced remote store path type in the customData field in IndexMetadata. We associate the path during index creation. However, there are certain cases where the association needs to be handled manually. They are as follows -

1. Snapshot restore - For indexes that were backed up in earlier versions (2.12 or earlier), the custom data would be absent. In such cases, we need to associate the newly restored indexes with the cluster default cluster.remote_store.index.path.prefix.type to benefit from the optimised prefix. In absence of this PR, we will continue to use non-optimised prefix.
2. Migration of indexes from Document replication type to Segment backed by remote store.

In this PR, we are handling 1st case.

Related Issues

Resolves #12730

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• [ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• [ ] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Compatibility status:

Checks if related components are compatible with change 45aaf5b

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git]

[github-actions]

❕ Gradle check result for 3f2262e: UNSTABLE

• TEST FAILURES:

      2 org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.classMethod
      1 org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.testResizeQueueDown

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Attention: Patch coverage is 91.66667% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 71.35%. Comparing base (b15cb0c) to head (45aaf5b).
Report is 57 commits behind head on main.

Files	Patch %	Lines
...h/cluster/metadata/MetadataCreateIndexService.java	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12753      +/-   ##
============================================
- Coverage     71.42%   71.35%   -0.07%     
- Complexity    59978    60153     +175     
============================================
  Files          4985     4999      +14     
  Lines        282275   283045     +770     
  Branches      40946    41030      +84     
============================================
+ Hits         201603   201958     +355     
- Misses        63999    64377     +378     
- Partials      16673    16710      +37     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ashking94]

Ran the MetadataCreateIndexServiceTests.testRemoteCustomData for around 1K iterations after the change and they are successful.

[github-actions]

✅ Gradle check result for 08cb85c: SUCCESS

[github-actions]

❕ Gradle check result for f8f6806: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationAllocationIT.testSingleIndexShardAllocation

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[ashking94]

❕ Gradle check result for f8f6806: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationAllocationIT.testSingleIndexShardAllocation

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Flaky test - #11299

[github-actions]

✅ Gradle check result for 45aaf5b: SUCCESS

[ashking94]

tagging @gbbafna, @linuxpi, @sachinpkale for review.