Skip to content

feat: 20234: Virtual node cache and virtual data source to use hash chunks#21656

Open
artemananiev wants to merge 61 commits intomainfrom
20234-M-nodecache-datasource-hashchunks
Open

feat: 20234: Virtual node cache and virtual data source to use hash chunks#21656
artemananiev wants to merge 61 commits intomainfrom
20234-M-nodecache-datasource-hashchunks

Conversation

@artemananiev
Copy link
Contributor

@artemananiev artemananiev commented Oct 15, 2025

Fixes: #20234
Signed-off-by: Artem Ananev artem.ananev@hashgraph.com

@artemananiev
fix: 20207: RecordAccessor / RecordAccessorImpl cleanup
9841d45 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Minor typo in RecordAccessor class JavaDoc
30b56f2 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
f026c8b 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
6521fc5 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
feat: 20234: Virtual node cache and virtual data source to use hash c…
029fb0f 
…hunks (first draft)

Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
bd92e9a 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
8931e69 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Introduced VirtualHashChunk.minChunkIdForPaths()
721e49f 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Initial implementation of MerkleDbDataSource.rebuildHashChunks()
5a295b0 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Spotless, tests, minor changes
f1594ab 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
d4b116e 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev artemananiev self-assigned this Oct 15, 2025
@lfdt-bot
Copy link

lfdt-bot commented Oct 15, 2025 •
edited
Loading

✅ Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@artemananiev
Sync with main
4377755 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Minor optimizations to VirtualHasher tasks
9f47b43 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
9f3a97a 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Lazy ChunkHashTask input array initialization
a5ae369 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Added hach chunk cache in MerkleDb data source
7615b90 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
be22a39 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Misc
4ebe309 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Minor fixes
1079d05 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Minor fix in MerkleDbDataSource.snapshot()
7be7d68 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
9257f73 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
4f5f733 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
ReconnectHashLeafFlusher is switched to hash chunks + minor improvements
e9d68b3 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
VirtualNodeCache.preloadHashChunk() is thread-safe now
d156ced 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
3c83f8a 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
VirtualNodeCache: filter hash chunk mutation during merge
c2fe8c2 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Default chunk height set to 1
00369ca 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Debug logs
7e0a7d6 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
imalygin
imalygin reviewed Jan 26, 2026
View reviewed changes
imalygin
imalygin reviewed Jan 26, 2026
View reviewed changes
@artemananiev
Minor cleanup + comments in VirtualHasher
ab24c36 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Minor improvements to how hashes are loaded by reconnect teacher
4d443e5 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
da8c773 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Minor changes proposed by reviewers
65248b2 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@thenswan thenswan mentioned this pull request Jan 28, 2026
Closed
thenswan
thenswan reviewed Jan 28, 2026
View reviewed changes
thenswan
thenswan reviewed Jan 28, 2026
View reviewed changes
thenswan
thenswan reviewed Jan 28, 2026
View reviewed changes
@artemananiev
Minor tweaks to estimated virtual node cache size
d3b03c1 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
artemananiev and others added 9 commits January 29, 2026 10:59
@artemananiev
Minor tweaks and formatting
0e688e0 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Sync with main
ee7e133 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Module dependencies
151b46a 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Virtual map hashing doc, first version
d92d648 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@thenswan @artemananiev
feat: 23112: Update Internal Index Validation (#23174)
bb68014 
Signed-off-by: Nikita Lebedev <nikita.lebedev@limechain.tech>
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
Co-authored-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Added some samples to swirlds-virtualmap hashing.md doc
af89aa0 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
MerkleDbDataSource: pathToHashChunkStore is renamed to idToHashChunkS…
82f65f6 
…tore

Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Reverted unnecessary changes in LegacyCsvWriterTest
2235f38 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev
Spotless
797256d 
Signed-off-by: Artem Ananev <artem.ananev@swirldslabs.com>
@artemananiev artemananiev added Feature Enhancement Enhancing an existing feature driven by business requirements. Typically backwards compatible. Performance Issues related to performance concerns. Platform Virtual Map Platform Tickets pertaining to the platform labels Feb 5, 2026
@artemananiev artemananiev added this to the v0.73 milestone Feb 5, 2026
@artemananiev artemananiev marked this pull request as ready for review February 5, 2026 22:35
@artemananiev artemananiev requested a review from a team as a code owner February 5, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@imalygin imalygin imalygin left review comments

@thenswan thenswan thenswan left review comments

@OlegMazurov OlegMazurov OlegMazurov left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@artemananiev artemananiev

Labels

Feature Enhancement Enhancing an existing feature driven by business requirements. Typically backwards compatible. Performance Issues related to performance concerns. Platform Virtual Map Platform Tickets pertaining to the platform

Projects

None yet

Milestone

v0.73

Development

Successfully merging this pull request may close these issues.

Virtual node cache and virtual data source to use hash chunks

5 participants

@artemananiev @lfdt-bot @imalygin @thenswan @OlegMazurov