Skip to content

[Searchable Snapshot] Implement disk reservation logic to be used for local file system cache #5508

@aabukhalil

Description

@aabukhalil

Referring to #4964,

Cache Disk reservation logic will impact all of:

  • NodesStats as fs stats need to take into consideration the cache reserved size
  • DiskThresholdDecider once monitor service having the reservation logic injected it should work out of the box
  • MonitorService need to take into consideration the cache reserved size
  • org.opensearch.cluster.ClusterInfo::reservedSpace
  • org.opensearch.cluster.InternalClusterInfoService::buildShardLevelInfo
  • org.opensearch.index.store.StoreStats##reservedSize
    so that
    org.opensearch.cluster.routing.allocation.DiskThresholdMonitor
    and
    org.opensearch.cluster.routing.allocation.decider.DiskThresholdDecider
    take into consideration the new disk reservation logic

Reservation Logic

When creating the cache, a validations are needed to make sure:

  • (cache defined reserved size - current cache phantom files + lucene indices file size) <= cluster.routing.allocation.disk.watermark.high and cluster.routing.allocation.disk.watermark.low and so on
  • When cache is initiated on node start procedure, it's needed to walk through cache dir files and count their sizes
  • For now we don't need to handle logic if cache data path has changed and old path has ghost files

Outcome of this

add fields reservedCacheSize and currentCacheSize to org.opensearch.env.NodeEnvironment.NodePath
and make these fields reflected correctly in org.opensearch.monitor.fs.FsProbe#getFSInfo to have semantic of
given Cache max reserved size, current cache occupied size, Implement size reservation logic to be reflected on NodesStats::FileSystemStats, Deciders, Monitoring Service

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions