Skip to content

Hotfix blocked messages with Holochain debug fork#642

Merged
lucksus merged 4 commits intodevfrom
blocked-message-debugging
Jan 2, 2026
Merged

Hotfix blocked messages with Holochain debug fork#642
lucksus merged 4 commits intodevfrom
blocked-message-debugging

Conversation

@lucksus
Copy link
Member

@lucksus lucksus commented Dec 29, 2025

We've encountered "blocked_message_count" in our Holochain network stats, correlated with syncing lagging between the blocked node. This is a bug in Holochain that appears when network connections are flaky. See this PR in Kitsune2: holochain/kitsune2#417.

This needs more investigation in Holochain/Kitsune. My quickfix in that branch is not a sufficient fix because it breaks blocking when it's needed (see comments on that PR).

Currently we don't have membranes and no language with important validation code that would need warrents being executed. So we can run with above quick fix.

This branch switches to a temporary branch in Holochain/Kitsune2 in our forks until the issue is resolved in Holochian.

Also removes some warnings.


Note

Mitigates network "blocked messages" by moving to forked Holochain/Kitsune2 and refreshing dependencies.

  • Networking: Patch workspace to use kitsune2* from fix/unexpected-blocked-messages; Holochain crates retargeted to blocked-messages-debugging
  • Deps: Widespread updates in Cargo.lock (notably hyper 1.7.0, reqwest alignment, sbd-server 0.4.0, opentelemetry crates), several version pins/adjustments
  • SurrealDB: Remove global service init/singleton (init_surreal_service, global getter) and related test/setup calls; rely on per‑perspective instances; simplify retry_surreal_op signature
  • CLI/UI: Minor cleanup in REPL imports; change system tray to show_menu_on_left_click(false)

Written by Cursor Bugbot for commit 5e5aa83. This will update automatically on new commits. Configure here.

Summary by CodeRabbit

  • Chores

    • Updated dependency branches for holochain and kitsune2 components to support debugging instrumentation.
  • Refactor

    • Simplified internal service initialization and retry logic patterns.
    • Removed unused import statements.

✏️ Tip: You can customize this high-level summary in your review settings.

lucksus and others added 4 commits December 16, 2025 14:00
The kitsune2 patches from holochain weren't being applied because Cargo
patches need to be in the workspace root, not in individual crates.

Added explicit patches for kitsune2 crates to use lucksus/kitsune2
fix/unexpected-blocked-messages branch which contains:
- Fix for default-to-blocked behavior when no access decision exists
- Debug logging to distinguish network vs warrant-based blocking

This ensures ad4m builds use the fixed kitsune2 version.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 29, 2025

📝 Walkthrough

Walkthrough

Patches kitsune2 crates from a forked branch to address blocked-messages issues. Removes global SurrealDB singleton initialization and refactors retry logic to reduce parameter coupling. Updates dependency branches for holochain-related crates and API calls.

Changes

Cohort / File(s) Summary
Dependency Management
Cargo.toml, rust-executor/Cargo.toml
Added [patch.crates-io] section to override kitsune2-related crates (api, core, kitsune2, dht, gossip, bootstrap_srv, bootstrap_client) from forked repo pinned to fix/unexpected-blocked-messages. Updated holochain, holochain_cli_bundle, and holochain_types to use blocked-messages-debugging branch instead of 0.6.0-coasys.
SurrealDB Removal
rust-executor/src/lib.rs, rust-executor/src/surreal_service/mod.rs
Deleted SurrealDB initialization block from lib.rs including init_surreal_service() call and error handling. Removed public functions pub async fn init_surreal_service() and pub async fn get_surreal_service() from surreal_service module, eliminating global singleton pattern.
Refactoring & Updates
rust-executor/src/perspectives/perspective_instance.rs, cli/src/repl.rs, ui/src-tauri/src/system_tray.rs
Simplified retry_surreal_op() signature by removing link parameter; updated call sites in update_surreal_cache() accordingly. Narrowed import in repl.rs by removing Style from syntect::highlighting. Updated Tauri API call from .menu_on_left_click(false) to .show_menu_on_left_click(false).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • Upgrade Holochain to 0.5.4 #615: Updates holochain-related dependency branches in rust-executor/Cargo.toml, directly paralleling the dependency resolution changes in this PR.
  • Bump Holochain to 0.5.6 #622: Modifies holochain/holochain_cli_bundle/holochain_types source branches in Cargo.toml files, overlapping with the dependency patching strategy here.
  • Update to HC 0.5.3-rc0 #612: Changes holochain and kitsune crate dependencies across manifests, similar in scope to the multi-crate patching introduced in this PR.

Poem

🐰 Kitsune guards fly swift and true,
SurrealDB fades from global view,
Retry loops shed their heavy chains,
Dependencies dance through new domains,
What once was tangled now runs lean! ✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Hotfix blocked messages with Holochain debug fork' directly aligns with the PR's main objective: temporarily switching to a Holochain/Kitsune2 debug fork to address a blocked messages issue in network syncing.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on January 4

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

setup_wallet();
Ad4mDb::init_global_instance(":memory:").unwrap();
AgentService::init_global_test_instance();
init_prolog_service().await;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test missing prolog service initialization will panic

The test_perspective_isolation_surreal_query test removes the init_prolog_service().await call but still invokes create_perspective().await, which internally calls ensure_prolog_engine_pool(). This function calls get_prolog_service().await, which panics with "PrologServiceInterface not set" if the prolog service hasn't been initialized. The test will fail at runtime.

Fix in Cursor Fix in Web

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
rust-executor/src/perspectives/perspective_instance.rs (1)

1662-1692: Surreal retry helper is reasonable; string‑match is a bit brittle

The new retry_surreal_op helper gives you bounded retries on Surreal read/write conflicts with simple linear backoff and logs any non‑conflict failures, which is appropriate given Surreal is just a cache.

The only minor concern is relying on the exact error message substring "Failed to commit transaction due to a read or write conflict"; if SurrealDB changes its wording, retries will silently stop matching. If the Surreal client exposes a structured error type or code for conflicts, consider matching on that instead (or at least centralizing the message constant).

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e997730 and 5e5aa83.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (7)
  • Cargo.toml
  • cli/src/repl.rs
  • rust-executor/Cargo.toml
  • rust-executor/src/lib.rs
  • rust-executor/src/perspectives/perspective_instance.rs
  • rust-executor/src/surreal_service/mod.rs
  • ui/src-tauri/src/system_tray.rs
💤 Files with no reviewable changes (1)
  • rust-executor/src/surreal_service/mod.rs
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-11-13T12:18:07.305Z
Learnt from: lucksus
Repo: coasys/ad4m PR: 630
File: bootstrap-languages/p-diff-sync/hc-dna/zomes/perspective_diff_sync/Cargo.toml:28-29
Timestamp: 2025-11-13T12:18:07.305Z
Learning: For Holochain 0.6.0-rc.0 (branch 0.6.0-rc.0-coasys), the correct and compatible HDI/HDK pairing is: hdi version 0.7.0-rc.0 and hdk version 0.6.0-rc.0. HDI is intentionally one minor version ahead of HDK in this release.

Applied to files:

  • rust-executor/Cargo.toml
📚 Learning: 2025-11-13T12:17:37.357Z
Learnt from: lucksus
Repo: coasys/ad4m PR: 630
File: bootstrap-languages/p-diff-sync/hc-dna/workdir/dna.yaml:2-2
Timestamp: 2025-11-13T12:17:37.357Z
Learning: For Holochain 0.6.0-rc.0, the correct manifest_version in DNA YAML files (dna.yaml) is 0, not 1. The mr_bundle library in Holochain 0.6.0-rc.0 errors when manifest_version is set to 1. The versioning appears to have been reset in this release.

Applied to files:

  • rust-executor/Cargo.toml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Cursor Bugbot
🔇 Additional comments (4)
cli/src/repl.rs (1)

6-6: Import cleanup is correct

Removing the unused Style import and keeping only ThemeSet matches actual usage and just silences a warning; no behavioral impact.

rust-executor/src/lib.rs (1)

37-41: SurrealDB init removal here is consistent with per‑perspective setup

Dropping init_surreal_service from the imports (and from run()) fits with the new design where each PerspectiveInstance owns its own SurrealDBService. This keeps executor startup focused on core services only; just ensure all runtime call‑sites that need Surreal now construct a service explicitly (as in the tests) and there are no lingering calls to the old initializer elsewhere.

rust-executor/src/perspectives/perspective_instance.rs (1)

3286-3330: Test setup now matches per‑perspective Surreal usage

Importing SurrealDBService and constructing it directly in setup() / create_perspective() keeps tests aligned with the new “one Surreal instance per perspective” model and avoids relying on any removed global initializer. The in‑memory DB plus explicit service construction makes these tests good coverage for the new path.

ui/src-tauri/src/system_tray.rs (1)

61-61: ✓ Tauri 2 API is correct

The .show_menu_on_left_click(false) call is the correct Tauri 2.x API. It properly replaces the older menu_on_left_click method and preserves the behavior of disabling the tray menu on left-click.

Comment on lines +9 to +19

[patch.crates-io]
# DEBUG: Using forked kitsune2 with debugging instrumentation and fixed default behavior
# These patches ensure ad4m uses the fixed kitsune2 that doesn't default to blocking
kitsune2_api = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
kitsune2_core = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
kitsune2 = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
kitsune2_dht = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
kitsune2_gossip = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
kitsune2_bootstrap_srv = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
kitsune2_bootstrap_client = { git = "https://github.com/lucksus/kitsune2.git", branch = "fix/unexpected-blocked-messages" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# First, check if Cargo.toml exists and display the patch section
if [ -f "Cargo.toml" ]; then
  echo "=== Cargo.toml [patch.crates-io] section ==="
  sed -n '/\[patch\.crates-io\]/,/^\[/p' Cargo.toml | head -20
else
  echo "Cargo.toml not found"
fi

# Try to run cargo tree if cargo is available
if command -v cargo &> /dev/null; then
  echo ""
  echo "=== Attempting cargo tree for kitsune2_api ==="
  cargo tree -p kitsune2_api 2>&1 | head -30 || echo "cargo tree failed or package not found"
else
  echo "Cargo command not available"
fi

# Check if there are any TODO comments related to kitsune2 patches
echo ""
echo "=== Searching for TODO comments related to kitsune2 or patches ==="
rg -i "todo.*kitsune2|todo.*patch" --max-count=20 || echo "No TODO comments found"

# Check for any commit hashes in the Cargo.toml to see if commits are pinned
echo ""
echo "=== Checking for commit-based patches ==="
rg "rev\s*=" Cargo.toml || echo "No rev= fields found (not pinned to commits)"

Repository: coasys/ad4m

Length of output: 1565


Branch-pinned kitsune2 patches lack reproducibility and cleanup tracking

The [patch.crates-io] for all kitsune2* crates correctly redirects to your forked branch and will work for the hotfix. However:

  • Pinning to a branch instead of a commit is fragile: If the branch is force-pushed or reset, the build's behavior changes unpredictably. For reproducibility and CI stability, specify a rev field with an explicit commit hash.
  • No TODO to track patch removal: Add a comment or GitHub issue to remove this patch once the upstream Kitsune2 fix is merged and released. Without it, this temporary workaround risks becoming permanent technical debt.

Example improvement:

kitsune2_api = { git = "https://github.com/lucksus/kitsune2.git", rev = "<commit-hash>" }  # TODO: Remove once lucksus/kitsune2#XXX is merged upstream
🤖 Prompt for AI Agents
In Cargo.toml around lines 9 to 19, the [patch.crates-io] entries use branch
pins for multiple kitsune2 crates which is non-reproducible and lacks cleanup
tracking; update each entry to pin to a specific commit by replacing the branch
with a rev = "<commit-hash>" value (use the exact commit hash you tested) and
add a short TODO comment on each line referencing the upstream PR/issue (e.g.,
"TODO: remove once lucksus/kitsune2#<PR> is merged") so CI builds are
deterministic and the temporary patch is tracked for removal.

Comment on lines +98 to +100
holochain = { version = "0.6.0", features = ["test_utils", "default", "backend-go-pion"], git = "https://github.com/coasys/holochain.git", branch = "blocked-messages-debugging" }
holochain_cli_bundle = { version = "0.6.0", git = "https://github.com/coasys/holochain.git", branch = "blocked-messages-debugging" }
holochain_types = { version = "0.6.0", git = "https://github.com/coasys/holochain.git", branch = "blocked-messages-debugging" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# First, locate and examine the rust-executor/Cargo.toml file
find . -path "*/rust-executor/Cargo.toml" -type f

Repository: coasys/ad4m

Length of output: 81


🏁 Script executed:

#!/bin/bash
# Examine the specific lines in rust-executor/Cargo.toml around holochain dependencies
if [ -f "rust-executor/Cargo.toml" ]; then
  echo "=== Lines around 98-100 ==="
  sed -n '90,110p' rust-executor/Cargo.toml
  echo ""
  echo "=== All holochain dependencies ==="
  grep -n "holochain" rust-executor/Cargo.toml
fi

Repository: coasys/ad4m

Length of output: 2176


🏁 Script executed:

#!/bin/bash
# Check for HDI/HDK dependencies as the learnings mention version pairing requirements
if [ -f "rust-executor/Cargo.toml" ]; then
  echo "=== HDI/HDK dependencies in rust-executor ==="
  grep -n "hdi\|hdk" rust-executor/Cargo.toml
fi

Repository: coasys/ad4m

Length of output: 100


🏁 Script executed:

#!/bin/bash
# Check workspace structure for other Cargo.toml files that might have holochain deps
find . -name "Cargo.toml" -type f | head -20 | while read f; do
  if grep -q "holochain\|hdi\|hdk" "$f" 2>/dev/null; then
    echo "=== $f ==="
    grep -n "holochain\|hdi\|hdk" "$f" | head -10
  fi
done

Repository: coasys/ad4m

Length of output: 3946


🏁 Script executed:

#!/bin/bash
# Check git log to understand the blocked-messages-debugging branch context
cd rust-executor
git log --oneline --decorate -n 1 Cargo.toml 2>/dev/null || echo "Not a git repo in sandbox"
cd ..

Repository: coasys/ad4m

Length of output: 158


🏁 Script executed:

#!/bin/bash
# Check if there's any documentation or comments about the blocked-messages-debugging branch
grep -r "blocked-messages-debugging" . --include="*.md" --include="*.toml" 2>/dev/null | head -20

Repository: coasys/ad4m

Length of output: 575


Holochain debug branch pinning and rollback plan

Pointing holochain* crates at the blocked-messages-debugging branch is appropriate for this Kitsune2 fix, but without commit pinning, the build depends on a moving branch. This is acceptable for a temporary hotfix, but pin to a specific commit hash and track a follow-up task to revert back to the upstream 0.6.0-coasys branch once the Kitsune2 fix lands upstream.

🤖 Prompt for AI Agents
In rust-executor/Cargo.toml around lines 98 to 100, the holochain* dependencies
are currently referencing the moving branch "blocked-messages-debugging"; change
each dependency to pin to the specific commit SHA (use rev = "<commit-hash>" or
replace branch with the commit notation for the git dependency) so the build is
deterministic, add a short inline comment noting this is a temporary hotfix and
reference a follow-up task/issue to revert the three dependencies back to the
upstream 0.6.0-coasys branch once the Kitsune2 fix is merged, and commit the
updated Cargo.toml.

@lucksus lucksus merged commit c8109b3 into dev Jan 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant