Skip to content

[Timestamp Pinning in Remote Store] Scheduler to fetch pinned timestamp list from remote store #15062

@sachinpkale

Description

@sachinpkale

Is your feature request related to a problem? Please describe

Given that remote store garbage collectors are aware of pinned timestamps, it becomes important to communicate any change to pinned timestamp to all the nodes in cluster within pre-defined SLA.

Describe the solution you'd like

  • To make sure each node gets the updated pinned timestamp details within X mins, in this solution, we run a scheduler on each node with period of X/2 mins (this is to make sure we run the scheduler at least twice).
  • This scheduler will fetch the list of pinned timestamps and store it along with timestamp of the successful fetch.
  • The timestamp of successful is required for garbage collector to identify if it is accessing stale pinned timestamp list and in such cases, garbage collector needs to be skipped.

Related component

Storage:Remote

Metadata

Metadata

Assignees

Labels

Storage:RemoteenhancementEnhancement or improvement to existing feature or request

Type

No type

Projects

Status

✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions