storage channel splitter for columns#10276
Draft
tbenr wants to merge 1 commit intoConsensys:masterfrom
Draft
Conversation
| new SidecarStorageChannelSplitter( | ||
| serviceConfig.createAsyncRunner("sidecar_storage_query", 1), | ||
| chainStorage, | ||
| chainStorage); |
There was a problem hiding this comment.
Sidecar query parallelism significantly lower than established pattern
Medium Severity
The sidecar_storage_query AsyncRunner is created with parallelism of 1, while the existing storage_query AsyncRunner uses STORAGE_QUERY_CHANNEL_PARALLELISM which equals 20. This 20x reduction in parallelism could cause sidecar query operations to become a bottleneck during high-load scenarios like sync, backfill, or data availability sampling, where many concurrent queries are expected. The stated goal of isolating data column operations "under high load" may be undermined if sidecar queries queue up excessively.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description
Fixed Issue(s)
Documentation
doc-change-requiredlabel to this PR if updates are required.Changelog
Note
Introduces a dedicated query channel and splitter to isolate data column sidecar storage traffic from block/state operations and run queries asynchronously.
SidecarQueryChannelAPI andSidecarStorageChannelSplitterto route sidecar queries via an async runner while keeping updates synchronousSidecarQueryChannelinChainStorageStorageServiceto publish/subscribeSidecarUpdateChannelandSidecarQueryChannelvia the splitterDataColumnSidecarDBto useSidecarQueryChannelinstead ofCombinedChainDataClientBeaconChainControllerto createDataColumnSidecarDBusing event channel publishers forSidecarQueryChannelandSidecarUpdateChannelWritten by Cursor Bugbot for commit d604682. This will update automatically on new commits. Configure here.