-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Closed
Labels
Indexing & Searchdistributed frameworkenhancementEnhancement or improvement to existing feature or requestEnhancement or improvement to existing feature or request
Description
Referring to #4964,
Cache Disk reservation logic will impact all of:
NodesStatsas fs stats need to take into consideration the cache reserved sizeDiskThresholdDecideronce monitor service having the reservation logic injected it should work out of the boxMonitorServiceneed to take into consideration the cache reserved sizeorg.opensearch.cluster.ClusterInfo::reservedSpaceorg.opensearch.cluster.InternalClusterInfoService::buildShardLevelInfoorg.opensearch.index.store.StoreStats##reservedSize
so that
org.opensearch.cluster.routing.allocation.DiskThresholdMonitor
and
org.opensearch.cluster.routing.allocation.decider.DiskThresholdDecider
take into consideration the new disk reservation logic
Reservation Logic
When creating the cache, a validations are needed to make sure:
- (cache defined reserved size - current cache phantom files + lucene indices file size) <=
cluster.routing.allocation.disk.watermark.highandcluster.routing.allocation.disk.watermark.lowand so on - When cache is initiated on node start procedure, it's needed to walk through cache dir files and count their sizes
- For now we don't need to handle logic if cache data path has changed and old path has ghost files
Outcome of this
add fields reservedCacheSize and currentCacheSize to org.opensearch.env.NodeEnvironment.NodePath
and make these fields reflected correctly in org.opensearch.monitor.fs.FsProbe#getFSInfo to have semantic of
given Cache max reserved size, current cache occupied size, Implement size reservation logic to be reflected on NodesStats::FileSystemStats, Deciders, Monitoring Service
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Indexing & Searchdistributed frameworkenhancementEnhancement or improvement to existing feature or requestEnhancement or improvement to existing feature or request
Type
Projects
Status
Done