export part - skip or throw on pending mutations and patch parts

[arthurpassos]

Changelog category (leave one):

• Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add export_merge_tree_part_throw_on_pending_mutations and export_merge_tree_part_throw_on_pending_patch_parts to control the behavior of pending mutations / patch parts on the export part feature. By default it will throw. It is important to note that when exporting an outdated it will never throw.

Also add export_merge_tree_part_allow_outdated_parts that allows outdated parts to be exported. Defaults to false.

Documentation entry for user-facing changes

Add export_merge_tree_part_throw_on_pending_mutations and export_merge_tree_part_throw_on_pending_patch_parts to control the behavior of pending mutations / patch parts on the export part feature. By default it will throw. It is important to note that when exporting an outdated it will never throw.

Also add export_merge_tree_part_allow_outdated_parts that allows outdated parts to be exported. Defaults to false.

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All with Aarch64
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• Tiered Storage (2h)

[arthurpassos]

One sec and I will adapt it for export partition

[github-actions]

Workflow [PR], commit [4c59e82]

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 28e124685d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[arthurpassos]

Either getMutationsSnapshot or getAlterConversionsForPart grabs a lock. Putting this in a scope to avoid dead locks

[arthurpassos]

QA team has requested a report of why I think this is ready to be reviewed / merged.

I have written several tests and performed some manual tests that cover the following cases for both part and partition export:

1. Mutations and patch parts created before the export. Skip and throw should behave as expected.
2. Mutations and patch parts created after the export has been issued, but not processed yet. Tests were implemented with iptable rules to simulate s3 outage. Should not throw or apply patches, should simply skip.

I have also tested that IN PARTITION works for replicated engines, they don't for plain merge tree tho.

I have also manually tested that outdated parts can be exported if the setting is set to true. It does not work for partition export, and it is expected. -- Honestly, maybe this setting should not even exist.