refactor(ops): relay UPDATE streaming task-per-tx (slice C) — #1454 phase 5#3937
Merged
Conversation
…hase 5 Add task-per-transaction drivers for `UpdateMsg::RequestUpdateStreaming` and `UpdateMsg::BroadcastToStreaming` in `operations/update/op_ctx_task.rs` and extend the UPDATE dispatch gate in `node.rs` to route fresh inbound streaming relays through them. Why: streaming UPDATE relays stayed on the legacy state-machine path after slice A. Bringing them onto task-per-tx restores parity with the slice A dedup gate, eliminates the last DashMap-resident relay UPDATE path, and closes the amplifier surface that the non-streaming drivers already avoid. Scope: - New drivers + RAII guard + streaming-specific counters (`RELAY_UPDATE_STREAMING_*`) separate from slice A for observability ramp, but sharing `active_relay_update_txs` (unified tx dedup) and `RELAY_UPDATE_DEDUP_REJECTS` (unified dedup counter). - Both drivers are fire-and-forget: claim stream via `orphan_stream_registry().claim_or_wait`, assemble, deserialize payload, apply via `update_contract`, rely on `BroadcastStateChange` for propagation. No `pipe_stream`, no `send_and_await`. - `BroadcastToStreaming` driver preserves the legacy failure classification (`log_broadcast_to_streaming_failure` + conditional `try_auto_fetch_contract` / `send_summary_back_on_rejection`), the pre-merge dedup cache check, and the proactive summary notification on successful state change. - `log_broadcast_to_streaming_failure` promoted from private to `pub(crate)` so the driver can reuse the shared classifier. - Dispatch gate in `node.rs` routes `RequestUpdateStreaming` and `BroadcastToStreaming` to the new drivers under the same `source_addr.is_some()` AND `!has_update_op(id)` gate as slice A. - Deprecated `UpdateMsg::Broadcasting` stays on the legacy no-op handler; pre-registered `UpdateOp`s (GC / originator-loopback) still fall through to legacy. Structural pin tests cover: driver existence, dedup-gate usage, orphan-stream claim, fire-and-forget guarantee, pre-merge dedup ordering, failure classification, proactive summary spawn, and RAII guard cleanup of streaming-specific counters. Existing slice A pin test `slice_a_drivers_do_not_touch_streaming_variants` re-scoped to the slice A fns only; the legacy-path-only dispatch pin test is replaced with `update_branch_dispatches_streaming_drivers_but_not_broadcasting`. Refs #1454 phase 5 follow-up slice C.
Update operations.md and crates/core/CLAUDE.md to reflect that streaming UPDATE relay variants now run on the task-per-tx path alongside PUT streaming. Only the deprecated `UpdateMsg::Broadcasting` wire variant remains on the legacy no-op handler (scheduled for Phase 6 wire cleanup).
Contributor
Rule Review: No issues foundRules checked: WarningsNone. Info
Checklist against applicable rules:
Rule review against |
Follow-up on PR #3937 review: 1. Pin: streaming dispatch arms (`RequestUpdateStreaming`, `BroadcastToStreaming`) must be gated on `!has_update_op(id)`. Guards the negative case that was only covered by positive pins previously — prevents slice C drivers from stealing pre-registered UpdateOps away from the legacy state machine (GC / originator-loopback paths still need `load_or_init`/`push`). 2. Document why tx dedup is held across `claim_or_wait`'s timeout window. Attacker with bogus `stream_id` can park one tx entry per bogus tx for up to `STREAM_CLAIM_TIMEOUT` (60s). Releasing earlier would allow duplicate metadata messages to spawn parallel drivers on the same tx, amplifying work and telemetry. Bounded, per-tx, and visible via `RELAY_UPDATE_STREAMING_INFLIGHT`. 3. Pin: `log_broadcast_to_streaming_failure` must remain `pub(crate)`. Slice C driver reuses this classifier for `try_auto_fetch_contract` / `send_summary_back_on_rejection` branching; re-privatising (e.g. during Phase 6 cleanup) would break the streaming broadcast failure path silently. No behavioural change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Issue #1454 Phase 5 migrates relay operations to the task-per-transaction
driver pattern. Slice A (PR #3910) migrated non-streaming UPDATE relays
(
UpdateMsg::RequestUpdate,UpdateMsg::BroadcastTo). Streamingvariants (
RequestUpdateStreaming,BroadcastToStreaming) and thedeprecated
Broadcastingwire variant were left on the legacystate-machine path — meaning the last DashMap-resident relay UPDATE
amplifier surface (pre-registered
UpdateOp+load_or_init/push)remained uncovered by the slice A dedup gate.
Solution
Add task-per-transaction drivers for both streaming variants in
operations/update/op_ctx_task.rs:start_relay_request_update_streaming— claims inbound stream viaorphan_stream_registry().claim_or_wait, assembles, deserializesUpdateStreamingPayload, applies locally viaupdate_contract,emits
update_successtelemetry. Fire-and-forget: propagation isvia
BroadcastStateChangeafter local apply.start_relay_broadcast_to_streaming— same stream-claim + assembleflow, plus pre-merge dedup cache check (amplifier mitigation), WASM
merge,
update_broadcast_appliedtelemetry, proactive summaryspawn on success, and the legacy failure classifier
(
log_broadcast_to_streaming_failure+ conditionaltry_auto_fetch_contract/send_summary_back_on_rejection).Key simplification vs PUT slice B: UPDATE streaming relays are
fire-and-forget. No downstream
pipe_stream, no upstream reply, norequest-response bookkeeping.
Dispatch gate in
node.rsroutes both variants under the samesource_addr.is_some()AND!has_update_op(id)gate as slice A.Deprecated
UpdateMsg::Broadcastingstays on the legacy no-ophandler; pre-registered
UpdateOps (GC / originator-loopback) stillfall through to legacy.
Shared counter
RELAY_UPDATE_DEDUP_REJECTSand shared tx setactive_relay_update_txsensure dedup is unified across slice A +slice C; separate
RELAY_UPDATE_STREAMING_*counters give operatorsindependent visibility during the ramp.
Testing
tests + 3 updated node.rs dispatch pin tests).
cargo test -p freenet --libpasses (2462 passed, 16ignored).
cargo clippy -p freenet --all-targets -- -D warningsclean.cargo fmtclean.Structural pin tests cover: driver existence, dedup-gate usage (both
drivers increment
RELAY_UPDATE_DEDUP_REJECTS), orphan-stream claimvia
claim_or_wait, fire-and-forget guarantee (nosend_and_await,no
pipe_stream), pre-merge dedup ordering (BroadcastToStreamingcalls
broadcast_dedup_cache.check_and_insertbeforeupdate_contract), failure classification, proactive summary spawn,and RAII guard cleanup of streaming-specific counters
(
RELAY_UPDATE_STREAMING_INFLIGHT,RELAY_UPDATE_STREAMING_COMPLETED_TOTAL).Linux-only multi-loopback streaming end-to-end tests remain covered
by existing
/freenet:linux-testinfrastructure.Fixes
Partial progress on #1454 Phase 5. After this merges, only the
deprecated
UpdateMsg::Broadcastingwire variant + legacy streamingarms in
update.rs:871-1366remain — Phase 6 wire cleanup willdelete both.