Table level user-defined GC #18117
Description
Description
Is your feature request related to a problem? Please describe:
GC process has 3 steps:
- resolve locks for all data
- cleanup ranges covered by truncated/dropped table/index
- cleanup old versions
From GC worker's aspect, all data is the same, let's name it as Global GC. There are some issues with Global GC:
- Must resolve locks for all regions at first
- Some tables are inserted only, GC these tables will waste some resources
- Some tables are updated frequently, these tables need to GC ASAP, but we can't archive it if the cluster is huge.
- Only one global safepoint, if we want to keep the operating history of a specified important table for a long time, let's say 7 days, this means we should set the global safepoint as 7 days, and this will affect all tables.
Describe the feature you'd like:
Split Global GC into table-level
- User can set an individual safepoint for a specified table. For example, we can set a longer gc_life_time for tidb_statistics table, we can use the historical statistics to diagnose suboptimal plan issues.
- User can set the GC interval for a specified table
- User can set lazy GC strategy for these cold tables/data or insert only tables
- User can trigger force GC for a specified table manually
Describe alternatives you've considered:
Adaptive GC:
- Recognize the operating patterns for each table
- Apply different GC strategies for each table
Teachability, Documentation, Adoption, Migration Strategy:
Category
Feature、Reliability
Value
- Cost-effective: flexible data versioning and space management
- Backtrack: keep history versions for some important table, for example, some metadata for TiDB
Workload Estimation
300