Skip to content

feat(validation): Support PrepareForRecord#753

Open
hudem1 wants to merge 101 commits intomainfrom
hudem1/feat/prepare-for-record
Open

feat(validation): Support PrepareForRecord#753
hudem1 wants to merge 101 commits intomainfrom
hudem1/feat/prepare-for-record

Conversation

@hudem1
Copy link
Collaborator

@hudem1 hudem1 commented Mar 13, 2026

Dependent on this NMC PR : NethermindEth/nethermind#10701

This method taking a [start, end] makes sure the state trie exists for each block within that range (reconstructing it if needed), so that when the corresponding RecordBlockCreation calls are made, the state already exists and the witness generation can be made without latency (which would be due to on the fly state reconstruction).

Reconstructed state goes into some memDB that gets pruned gradually when the trie nodes are considered useless (not referenced anymore), otherwise it'd grow indefinitely.
This follows nitro which uses this Referencing/Dereferencing mechanism to keep track of needed trie nodes.

And just like nitro, ReconstructedStateTrieStore writes to this memDB overlay and is backed by the main trie store (as read only) for reading trie nodes from disk (when not present in memDB). When checking for state root existence, it ignores the dirty nodes cache as not reliable as memory pruning could evict nodes there and only part of the state trie could get persisted. So, it checks only the memDB or disk.

hudem1 added 30 commits January 12, 2026 18:30
… that block up to current block's parent header
use case example: Submit retryable tx with empty calldata and that returns early
@hudem1 hudem1 marked this pull request as ready for review March 13, 2026 05:55
Copilot AI review requested due to automatic review settings March 13, 2026 05:55
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 25 out of 25 changed files in this pull request and generated no comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@codecov
Copy link

codecov bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 77.15356% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.22%. Comparing base (f19796a) to head (4a0af7f).

Files with missing lines Patch % Lines
...Arbitrum/Execution/Stateless/StateReconstructor.cs 68.42% 35 Missing and 13 partials ⚠️
...mind.Arbitrum/Execution/ArbitrumExecutionEngine.cs 68.00% 6 Missing and 2 partials ⚠️
...Execution/Stateless/ReconstructedStateTrieStore.cs 94.80% 3 Missing and 1 partial ⚠️
...Execution/ArbitrumExecutionEngineWithComparison.cs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #753      +/-   ##
==========================================
- Coverage   77.24%   77.22%   -0.03%     
==========================================
  Files         185      187       +2     
  Lines       11889    12153     +264     
  Branches     1615     1658      +43     
==========================================
+ Hits         9184     9385     +201     
- Misses       2113     2159      +46     
- Partials      592      609      +17     
Files with missing lines Coverage Δ
src/Nethermind.Arbitrum/ArbitrumPlugin.cs 44.02% <100.00%> (+0.92%) ⬆️
...Nethermind.Arbitrum/Arbos/Programs/StylusParams.cs 94.18% <ø> (ø)
src/Nethermind.Arbitrum/Config/ArbitrumConfig.cs 70.00% <100.00%> (+3.33%) ⬆️
...ethermind.Arbitrum/Data/DigestMessageParameters.cs 92.00% <100.00%> (+0.69%) ⬆️
...itrumWitnessGeneratingBlockProcessingEnvFactory.cs 98.98% <100.00%> (ø)
...c/Nethermind.Arbitrum/Modules/ArbitrumRpcModule.cs 82.14% <100.00%> (+0.66%) ⬆️
...Execution/ArbitrumExecutionEngineWithComparison.cs 0.00% <0.00%> (ø)
...Execution/Stateless/ReconstructedStateTrieStore.cs 94.80% <94.80%> (ø)
...mind.Arbitrum/Execution/ArbitrumExecutionEngine.cs 49.13% <68.00%> (+1.24%) ⬆️
...Arbitrum/Execution/Stateless/StateReconstructor.cs 68.42% <68.42%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines +162 to +187
// Flush the trie dirty cache to disk on shutdown so that the latest processed blocks'
// state is persisted. Without this, Nethermind only persists up to head - PruningBoundary,
// while Nitro persists up to head. On restart, the mismatch causes Nitro to detect an error.
// _api.ProcessExit!.Token.Register(() =>
// {
// ILogger logger = _api.LogManager.GetClassLogger<ArbitrumPlugin>();
// if (logger.IsInfo)
// logger.Info("Flushing trie cache to disk before shutdown...");

// _api.WorldStateManager?.FlushCache(CancellationToken.None);

// // FlushCache persists state but does not update BestPersistedState because
// // TrieStore.AnnounceReorgBoundaries() requires LatestCommittedBlockNumber >= LastPersistedBlockNumber + maxDepth,
// // which is never met when persisting up to the head block itself.
// // Set it directly so that on restart the node head matches the last processed block.
// long? headNumber = _api.BlockTree!.Head?.Number;
// if (headNumber.HasValue)
// {
// if (logger.IsInfo)
// logger.Info($"Setting BestPersistedState to {headNumber.Value}.");
// _api.BlockTree!.BestPersistedState = headNumber.Value;
// }

// if (logger.IsInfo)
// logger.Info("Trie cache flushed.");
// });
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running non-archive node, such as when using memory pruning, the node does not persist every block. Therefore, it does not persist the last processed block. On restart, this causes some mismatch issue where nitro picks up from last processed block and nethermind starts from last processed - PruningBoundary.

To solve that issue, i suggest 2 possible solutions storing the last processed block on shutdown:

  1. either only uncomment this code:
  • pros: no modif to base NMC.
  • cons: will store the whole dirty nodes cache: i currently set MaxUnpersistedBlockCount to 1000 in this PR for the .json config files, meaning 1000 blocks' state will be persisted on shutdown only for to have the last processed block stored.
  1. or use the code in this base NMC PR
  • pros: will persist only the last processed block
  • cons: requires adding those modifications to base NMC (which would then be configurable for nodes other than Arbitrum), so, not sure that's a good solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants