improve scaling of new clvm compression implementation#707
Merged
Conversation
arvidn
commented
Feb 23, 2026
Pull Request Test Coverage Report for Build 22324685918Details
💛 - Coveralls |
a7ca8cf to
e66efde
Compare
e66efde to
1e30e2b
Compare
1e30e2b to
2e72758
Compare
…e of not always finding the best compression
2e72758 to
c64aea7
Compare
Contributor
Author
|
@cursor review |
0e8a53a to
ad785b5
Compare
ad785b5 to
c64aea7
Compare
5de8926 to
8abf1c3
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
8abf1c3 to
2b7352d
Compare
2b7352d to
22c79e3
Compare
richardkiss
reviewed
Feb 23, 2026
Contributor
|
Okay, I think I see more or less how it works. You're evicting a random parent so there are fewer things to look at. There might be a way to smartly evict potential parents that could never be reached. But also, I think this serialization format is a dead-end: serde 2026 is way better, both faster and better compression, and dead-simple now that interning is implemented. I have a draft PR here #708 |
richardkiss
previously approved these changes
Feb 23, 2026
richardkiss
reviewed
Feb 23, 2026
Co-authored-by: Richard Kiss <him@richardkiss.com>
richardkiss
approved these changes
Feb 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
at the expense of not always finding the best compression.
If the number of parents aren't capped, that list can become very long and very expensive to maintain and traverse. Not having access to every parent when finding a path is not optimal, but I believe this will give a good result most of the time, in the common cases. It definitely makes the algorithm scale a lot better.
The incremental serializer compares against
node_to_bytes_backrefs():Note
Medium Risk
Changes core compression path selection by dropping some parent edges, which can subtly affect compression ratios and edge-case behavior even though the logic remains bounded and well-scoped.
Overview
Improves scalability of the new incremental CLVM compression by capping
TreeCacheparent links per node (MAX_PARENTS = 8) and evicting older entries, trading optimal back-reference selection for bounded memory/CPU duringfind_path().Adds a Criterion benchmark (
many_spends) to stress incremental serialization on large spend-lists and removes the now-unneededserializer-cmpfuzz target. CI wheel builds are also stabilized by pinningmaturinto1.12.2(avoids--sdistartifact changes in1.12.3).Written by Cursor Bugbot for commit dd3679c. This will update automatically on new commits. Configure here.