Skip to content

c/topic_table: use chunked_hash_map for lifecycle markers#29269

Merged
piyushredpanda merged 1 commit intoredpanda-data:devfrom
bashtanov:lifecycles-markers-oversized
Jan 16, 2026
Merged

c/topic_table: use chunked_hash_map for lifecycle markers#29269
piyushredpanda merged 1 commit intoredpanda-data:devfrom
bashtanov:lifecycles-markers-oversized

Conversation

@bashtanov
Copy link
Copy Markdown
Contributor

@bashtanov bashtanov commented Jan 15, 2026

to avoid oversized allocations

Fixes https://redpandadata.atlassian.net/browse/CORE-15289

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v25.3.x
  • v25.2.x
  • v25.1.x

Release Notes

  • none

Copilot AI review requested due to automatic review settings January 15, 2026 12:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces absl::node_hash_map with chunked_hash_map for lifecycle markers storage to prevent oversized memory allocations. The change affects both the runtime topic_table and the snapshot data structure topics_t.

Changes:

  • Updated lifecycle markers container type from absl::node_hash_map to chunked_hash_map
  • Modified snapshot application logic to use replace() method instead of direct assignment
  • Applied the change consistently across both the topic table and controller snapshot structures

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/v/cluster/topic_table.h Changed lifecycle_markers_t typedef from absl::node_hash_map to chunked_hash_map
src/v/cluster/topic_table.cc Updated snapshot application to use replace() with .values().copy() instead of direct assignment
src/v/cluster/controller_snapshot.h Changed lifecycle markers type in topics_t struct from absl::node_hash_map to chunked_hash_map

@bashtanov bashtanov marked this pull request as draft January 15, 2026 12:37
@bashtanov bashtanov force-pushed the lifecycles-markers-oversized branch from 8c7838e to efc8a80 Compare January 15, 2026 17:54
@bashtanov bashtanov marked this pull request as ready for review January 15, 2026 17:55
@vbotbuildovich
Copy link
Copy Markdown
Collaborator

Retry command for Build#79095

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/cluster_linking_e2e_test.py::ShadowLinkingMetricsTests.test_link_metrics

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

CI test results

test results on build#79095
test_class test_method test_arguments test_kind job_url test_status passed reason test_history
ShadowLinkingMetricsTests test_link_metrics null integration https://buildkite.com/redpanda/redpanda/builds/79095#019bc2df-8436-4280-865b-8b7713f76517 FLAKY 12/21 Test FAILS after retries.Significant increase in flaky rate(baseline=0.0377, p0=0.0000, reject_threshold=0.0100) https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingMetricsTests&test_method=test_link_metrics

Copy link
Copy Markdown
Member

@dotnwat dotnwat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@bashtanov
Copy link
Copy Markdown
Contributor Author

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/cluster_linking_e2e_test.py::ShadowLinkingMetricsTests.test_link_metrics

@piyushredpanda piyushredpanda merged commit f100b58 into redpanda-data:dev Jan 16, 2026
19 checks passed
@vbotbuildovich
Copy link
Copy Markdown
Collaborator

/backport v25.3.x

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

/backport v25.2.x

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

/backport v25.1.x

@vbotbuildovich
Copy link
Copy Markdown
Collaborator

Failed to create a backport PR to v25.1.x branch. I tried:

git remote add upstream https://github.com/redpanda-data/redpanda.git
git fetch --all
git checkout -b backport-pr-29269-v25.1.x-296 remotes/upstream/v25.1.x
git cherry-pick -x efc8a8013c

Workflow run logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants