-
Notifications
You must be signed in to change notification settings - Fork 174
Description
Problem Statement
When a CDC mirror is created with a group of tables, all operations (running,pause, delete, edit) apply only at the mirror level, affecting all tables together. There is no way to manage individual tables within a mirror group independently. This becomes a critical operational gap in production environments where mirrors contain many tables.
Current Behavior
We dont whether table status, whether its running fine or not? so need Running state at every table level inside the mirror group.
Pausing a mirror pauses all tables in the group — no way to pause just one
Deleting a mirror deletes all tables — no way to remove a single failing or unwanted table
Editing a mirror (e.g. fixing a partition key on one table) is not possible without deleting and recreating the entire mirror
If one table fails during snapshot, the entire mirror is blocked — there is no way to skip, isolate, or fix just that table
Already-completed table snapshots are lost when the mirror is deleted and recreated, wasting time and resources
There is no per-table status visibility — you cannot tell which tables are healthy vs stuck without querying internal PostgreSQL metadata tables directly
Expected / Requested Behavior
PeerDB should support table-level operations within a mirror group, including:
OperationDescriptionPause (per table)Pause CDC/snapshot for a specific table without affecting other tables in the mirrorResume (per table)Resume a previously paused or failed table independentlyDelete (per table)Remove a single table from the mirror group without deleting the whole mirrorEdit (per table)Modify table-specific config — partition key, watermark column, column mappings — without recreating the mirrorSkip (per table)During snapshot phase, skip a failing table and continue with remaining tablesPer-table statusShow individual table status in the UI: Snapshotting / Completed / Failed / Paused / Running
Real World Scenario (our case)
We created a mirror with 6 tables from MariaDB → ClickHouse. During initial snapshot:
✅ Table 1 (test.communs) — completed in ~6.5 mins
✅ Table 2 (test.mer_communs_pref) — completed in ~7 mins
❌ Table 3 (test.mer_profile) — failed due to misconfigured partition key, stuck in infinite retry loop
⏳ Tables 4, 5, 6 — never started, blocked by Table 3
Proposed UI Changes
In the Mirror detail page, each table row should have an action menu (⋮) with options:
[ Pause ] [ Resume ] [ Edit ] [ Delete ] [ Skip ] [Running]
And a status badge per table:
test.communs ✅ Running
test.mer_communs_pref ✅ Running
test.mer_profile ❌ Failed [ Edit ] [ Retry ]
test.user_profile ⏳ Pending
test.ph_numbers ⏳ Pending
test.user_action_items ⏳ Pending
test.user_activities ⏳ Pending
Why This Matters
Production mirrors commonly have 10–50+ tables — mirror-level-only operations don't scale
A single misconfigured table should never block an entire mirror group
Recreating a mirror to fix one table wastes snapshot time on all other tables
Without these controls, operators are forced to directly manipulate internal metadata tables which is risky and undocumented
Environment
PeerDB Version: 0.36.7
Source: MariaDB
Destination: ClickHouse
Deployment: [Docker Compose]