-
Notifications
You must be signed in to change notification settings - Fork 2.5k
[META] Remote Store Issue List #2992
Copy link
Copy link
Open
Labels
Roadmap:Cost/Performance/ScaleProject-wide roadmap labelProject-wide roadmap labelStorage:RemoteenhancementEnhancement or improvement to existing feature or requestEnhancement or improvement to existing feature or request
Description
Is your feature request related to a problem? Please describe.
- Issues listed under this meta issue are related to Remote Segment Store.
Describe the solution you'd like
- Feature Proposal - [Feature Proposal] Add Remote Storage Options for Improved Durability #1968
- Design Proposal - [Design Proposal] Add Remote Storage Options for Improved Durability #2700
Tasks
OpenSearch 2.3 Release
- [Remote Segment Store] Define RemoteDirectory interface #2993
- [Remote Store] Add index level setting to enable/disable remote store #3141
- [Remote Segment Store] Add another instance of Store to InternalEngine that uses RemoteDirectory #2994
- [Remote Segment Store] Upload new segment files to remote store as a part of refresh flow #2995
- [Remote Store] API support to restore data from remote store #3145
- [Remote Store] Add a check to enable remote store only with Segment Replication #3147
- [Remote Store] Performance test setup for 2.3 release #3148
- [Remote Store] Make repository name configurable #3286
- [Remote Translog] Add extension points to support remote translog stores #3242
- [Remote Segment Store] Upload segments_N file after each refresh #3398
- [Remote Store] Move repository fetch logic from IndexService to RemoteDirectoryFactory #3547
- [Remote Store] Inject the RemoteStoreRefreshListener in the indexShard constructor #3548
- [Remote Store] Support nested index level settings for remote store #4059
- [Remote Store] Fix message on enabling remote store without segment replication #4201
- [BUG] [Remote Store] AssertionError while creating an index with
index.remote_store.enabled=true#4233 - [BUG][Remote Store] Repository base_path is ignored while uploading segments to remote segment store #4283
- [BUG][Remote Segment Store] On failover, new primary uploads all the segments to remote store, it should be incremental #4398
- [Remote Segment Store] Delete merged away segment files from remote segment store #3142
- [BUG][Remote Store] Repository base_path is ignored while uploading segments to remote segment store #4283
Backlog
- [Remote Segment Store] Validate checksum of uploaded and downloaded segment files #3143
- [Remote Store] Add support to auto restore data from remote store #3144
- [Remote Store] Add API to check status of remote store sync #3146
- [Discuss] [Remote Store] Supporting multiple remote stores per index #4437
- [Remote Translog] Integrate remote translog store with recovery #3255
- [Remote Store] Handle transient and permanent un-availability of the remote store #3397
- [Remote Store] Upload index metadata to remote store #3457
- [Remote Store] Delete data from remote store (translog and segments) on index delete #3511
- [Remote Store] Make remote segment store and remote translog path consistent #3589
- [Remote Store] Add wrapper for RestHighLevelClient that provides methods for accessing the Remote Store API. #3602
- [FEATURE] Add java client support for remote store APIs opensearch-java#168
- Add OpenSearch documentation for remotestore APIs and mark the APIs stable #3667
- [Remote Store] Create abstraction CompositeStore which will encapsulate local and remote store #3719
- [Pluggable Translog] Decouple Translog from Engine #3241
- [META] Provide extension points for Remote Translog #3600
- Remote Translog plugin implementation for S3 #3601
- [Discuss] Writes on NRT Replica with Remote Translog #3706
- Move Translog dependency from core to a module #3599
- [Remote Store] Integration of remote translog and remote segment store #3754
- [Remote Store] Integration test scenarios #3759
- [Remote Store] Restore API to support shard level restore #3768
- [Remote Store] Experimental Release Tracking #3974
- [Remote Store] Make uploadMetadata private and call it internally from RemoteSegmentStoreDirectory #4171
- [Remote Store] Create abstraction for RemoteSegmentStoreDirectory.init() method #4410
- [Remote Store] Add support for default repository name for remote store #4409
- [Remote Store] Fail index creation with error message if repository does not exist #4403
- [Remote Store] Support bulk upload and bulk delete while interacting with remote segment store #4317
- [Remote Store] Inject individual dependencies rather than injecting IndexShard into RemoteStoreRefreshListener #4316
- [Remote Store] Delete stale commits in async flow #4315
- [Remote Store] Track segment uploads as a part of an upload listener. #4198
- [Remote Store] Stop creating translog generation and checkpoint files on replicas where translog is stored remotely #4232
- [Remote Store] Introduce segment replication as prefinal step during recovery of replicas #4231
- [Remote Store] Skip translog replay in Recovery code for the replica's recovery when translogs stored remotely. #4230
- [Remote Store] With no-op replication, remove the payload to the in-sync calls to replicas #4229
- [Remote Store] Keep metadata files for last N refreshes #5455
- [Remote Store] Add upload timestamp to metada file to be used in conflict resolution #5475
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Roadmap:Cost/Performance/ScaleProject-wide roadmap labelProject-wide roadmap labelStorage:RemoteenhancementEnhancement or improvement to existing feature or requestEnhancement or improvement to existing feature or request
Type
Projects
Status
🆕 New
Status
New