[Meta] Validate plugins compatibility with segment replication #8211
Description
Summary
With 2.9.0 release, there are lot of enhancements going in for segment replication[1][2] feature (went GA in 2.7.0), we need to ensure different plugins are compatible with current state of this feature. Previously, we ran tests on plugin repos to verify this compatibility but want plugin owners to be aware of these changes so that required updates (if any) can be made. With 2.10.0 release, remote store feature is going GA which internally uses SEGMENT replication strategy only i.e. it enforces all indices to use SEGMENT replication strategy. So, it is important to validate plugins are compatible with segment replication feature.
What changed
1. Refresh policy behavior
- RefreshPolicy.IMMEDIATE will only refresh primary shards but not replica shards immediately. Instead post refresh, primary will start a round of segment replication to update the replica shard copies leading to eventual consistency.
- RefreshPolicy.WAIT_UNTIL ensures the indexing operation is searchable in your cluster i.e. RAW (Read after write guarantee). With segment replication, this guarantee is not promised due to delay in replica shared updates from asynchronous background refreshes.
2. Refresh lag on replicas
With segment replication, there is inherent delay in documents to be searchable on replica shard copies. This is due to the fact that replica shard copies over data (segment) files from primary. Thus, compared to document replication, there will be on average increase in amount of time the replica shards are consistent with primaries.
3. System/hidden indices support
With #8200, system and hidden indices are now supported with SEGMENT replication strategy. We need to ensure there are no bottlenecks which prevents system/hidden indices with segment replication.
Next steps
With segment replication strong reads are not guaranteed. Thus, if the plugin needs strong reads guarantees specially as alternative to change in behavior of refresh policy and lag on replicas (point 1 and 2 above), we need to update search requests to target primary shard only. With #7375, core now supports primary shards only based search. Please follow documentation for examples and details
Open questions
In case of any questions or issues, please post your question on core
Reference
[1] Design
[2] Documentation
Opensearch Plugins
- Compatibility with segment replication OpenSearch-Dashboards#4444
- Compatibility with segment replication alerting#974
- Compatibility with segment replication anomaly-detection#936
- Compatibility with segment replication asynchronous-search#303
- Compatibility with segment replication common-utils#469
- Compatibility with segment replication reporting#719
- Compatibility with segment replication index-management#833
- Compatibility with segment replication job-scheduler#407
- Compatibility with segment replication k-NN#948
- Compatibility with segment replication performance-analyzer#491
- Compatibility with segment replication performance-analyzer-rca#433
- Compatibility with segment replication security#2916
- Compatibility with segment replication sql#1801
- Compatibility with segment replication observability#1562
- Compatibility with segment replication cross-cluster-replication#1029
- Compatibility with segment replication ml-commons#1023
- Compatibility with segment replication geospatial#346
- Compatibility with segment replication notifications#697
- Compatibility with segment replication neural-search#206
- Compatibility with segment replication security-analytics#473
- Compatibility with segment replication opensearch-oci-object-storage#52
OpenSearch-dashboard plugins
- Compatibility with segment replication dashboards-query-workbench#88
- Compatibility with segment replication alerting-dashboards-plugin#587
- Compatibility with segment replication dashboards-notifications#64
- Compatibility with segment replication security-dashboards-plugin#1494
- Compatibility with segment replication index-management-dashboards-plugin#798
- Compatibility with segment replication anomaly-detection-dashboards-plugin#520
- Compatibility with segment replication dashboards-reporting#132
- Compatibility with segment replication dashboards-observability#586
- Compatibility with segment replication dashboards-visualizations#196
- Compatibility with segment replication dashboards-search-relevance#228
- Compatibility with segment replication dashboards-maps#430
- Compatibility with segment replication security-analytics-dashboards-plugin#639
- Compatibility with segment replication ml-commons-dashboards#226