fix(reqresp): clear composed response timeout signals#9029
fix(reqresp): clear composed response timeout signals#9029nflaig merged 2 commits intoChainSafe:unstablefrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a memory leak within the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request effectively addresses a memory leak by introducing a createRespSignal helper to manually manage AbortSignal listeners, replacing AbortSignal.any. The implementation is sound, and the cleanup logic is correctly placed within a finally block to ensure it's always executed. I have one suggestion to further improve the robustness of the new helper function.
nflaig
left a comment
There was a problem hiding this comment.
seems a bit overly complex but given these changes have been tested for 2 days already and fix all memory leaks we have observed it seems fine
|
|
||
| type ClearableSignal = AbortSignal & {clear: () => void}; | ||
|
|
||
| function createRespSignal(signal: AbortSignal | undefined, timeoutMs: number): ClearableSignal { |
There was a problem hiding this comment.
@lodekeeper can you add a comment to the upstream issue: nodejs/node#54614
There was a problem hiding this comment.
Sure! I'll add a comment with our real-world experience from Lodestar's req/resp networking layer. The composed signals from AbortSignal.any() accumulated ~85MB+ of old-space growth over 24h on a mainnet beacon node — exactly the pattern described in that issue. Our workaround in this PR (manual signal composition with explicit cleanup) eliminated the accumulation.
Drafting the comment now — will post shortly.
There was a problem hiding this comment.
Posted on the upstream Node issue here:
I summarized that we saw the same retention pattern in Lodestar's real req/resp networking workload (not just a tight synthetic loop), and linked this PR as the application-side mitigation.
There was a problem hiding this comment.
Well my intention with my comment here was that we would document the upstream issue in our own code/project. But I guess that works 🤷
There was a problem hiding this comment.
@lodekeeper just re-reading @wemeetagain I think he meant you should add a reference in the code to the upstream issue, can you open a follow up PR for that
There was a problem hiding this comment.
Good call — I misread the intent. Opened #9033 to add a JSDoc comment on createRespSignal() referencing the upstream Node.js issue and the investigation context.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #9029 +/- ##
=========================================
Coverage 52.32% 52.32%
=========================================
Files 848 848
Lines 62507 62507
Branches 4603 4603
=========================================
Hits 32705 32705
Misses 29736 29736
Partials 66 66 🚀 New features to boost your workflow:
|
Add JSDoc comment to createRespSignal() documenting why we use manual AbortController composition instead of AbortSignal.any() — it works around a memory leak in Node.js (nodejs/node#54614) where .any() retains references to source signals for the lifetime of the longest-lived signal. Refs: ChainSafe#8969, ChainSafe#9029
## Summary Add a JSDoc comment to `createRespSignal()` in `packages/reqresp/src/request/index.ts` documenting the upstream Node.js issue that motivates the manual `AbortController` composition pattern. Follow-up to #9029 per [review feedback](#9029 (comment)) — the code should document _why_ we avoid `AbortSignal.any()` and link to the upstream issue for future maintainers. ## References - Upstream: nodejs/node#54614 (`AbortSignal.any()` memory leak) - Investigation: #8969 - Fix PR: #9029 ## AI disclosure This PR was authored with AI assistance (Lodekeeper 🌟). Co-authored-by: lodekeeper <lodekeeper@users.noreply.github.com>
|
🎉 This PR is included in v1.41.0 🎉 |
Summary
Open the Lodestar side of the memory-leak mitigation by clearing composed response-timeout signals in req/resp request handling.
This is the Lodestar-local follow-up for the network-thread leak investigation in #8969. It addresses the req/resp timeout-signal retention path by replacing the
AbortSignal.any(...)composition with a clearable composed signal and explicitly clearing listeners once the response stream is done.This PR is intentionally scoped to the req/resp fix only.
The separate residual gossipsub topic-retention issue is being tracked upstream in
ChainSafe/js-libp2p-gossipsub#543, and we can pick that up once a new libp2p release lands.Change
createRespSignal()helper inpackages/reqresp/src/request/index.tsAbortControllerrespSignal.clear()after the response closesValidation
pnpm buildpnpm lint(warning-only: pre-existing unused biome suppression inpackages/light-client/test/unit/webEsmBundle.browser.test.ts)AI disclosure
This PR was authored with AI assistance (Lodekeeper 🌟).