[Remote Store] Update remote Store flows to support any path type with backward compatibility #12790
Description
Is your feature request related to a problem? Please describe
With optimised prefix pattern for remote store path (as mentioned in #12567), we need to ensure that we are able to allow new indexes to use the optimised remote store path type and at the same time be able to resolve the existing prefix pattern (Fixed remote store path type).
Describe the solution you'd like
As part of this task, we intend to make the necessary code changes to allow new indexes to use any remote store path type based on the cluster setting cluster.remote_store.index.path.prefix.type. We have added a remote store custom data in index metadata that is used to associate the path type for an index. When the custom data is absent, we would treat the path type to be Fixed.
Related component
Storage:Remote
Describe alternatives you've considered
No response
Additional context
No response