[Feature Request] Refactor Engine internals to decouple operation/result types and introduce pluggable planning strategies #20583
Description
Is your feature request related to a problem? Please describe
The current Engine design embeds operation planning logic directly within InternalEngine, particularly through methods such as:
planIndexingAsPrimary / planIndexingAsNonPrimary
planDeletionAsPrimary / planDeletionAsNonPrimary
These methods assume a single execution model and tightly couple planning behavior with a specific engine implementation and data flow. As a result, planning logic is not easily reusable or replaceable across alternative Engine implementations.
Introducing new engine implementations or execution semantics therefore requires duplicating or re-implementing substantial portions of planning logic within InternalEngine, increasing maintenance overhead and limiting extensibility.
This design constrains the ability to support multiple execution strategies while sharing common planning abstractions across different engine types.
Describe the solution you'd like
- Introduce Strategy Planner Interfaces
Refactor indexing and deletion planning logic into dedicated planner interfaces responsible for primary and non-primary execution. Default planner implementations would encapsulate the existing InternalEngine behavior, while enabling alternative engines to provide custom planners or reuse shared implementations.
- Delegate Planning via Composition
Update Engine implementations to delegate planning decisions to injected planner instances. The default behavior for InternalEngine should remain unchanged, ensuring backward compatibility.
Related component
Indexing
Describe alternatives you've considered
No response
Additional context
OpenSearch is actively working toward supporting extensible execution engines for diverse use cases. This refactoring would both simplify the existing Engine and InternalEngine implementations and enable reuse of these abstractions across alternative Engine implementations, as proposed in [RFC] OpenSearch Execution Engine.