Replace IPC busy-loop polling with async fs.watch by gavrielc · Pull Request #6 · qwibitai/nanoclaw

gavrielc · 2026-02-01T18:16:32Z

Use fs.watch for event-driven file watching instead of setTimeout polling
Replace synchronous fs calls (readdirSync, readFileSync, unlinkSync, renameSync)
with async fs.promises equivalents to avoid blocking the event loop
Add simple coalescing via pending flags to prevent redundant processing
Remove unused IPC_POLL_INTERVAL config constant

https://claude.ai/code/session_01Lccp65bZyEMMbDdF7d9E4j

- Use fs.watch for event-driven file watching instead of setTimeout polling - Replace synchronous fs calls (readdirSync, readFileSync, unlinkSync, renameSync) with async fs.promises equivalents to avoid blocking the event loop - Add simple coalescing via pending flags to prevent redundant processing - Remove unused IPC_POLL_INTERVAL config constant https://claude.ai/code/session_01Lccp65bZyEMMbDdF7d9E4j

gavrielc · 2026-02-01T18:41:24Z

Incompatible with PR #3's per-group IPC directory structure. fs.watch would need to be redesigned to watch multiple group directories. Closing for now.

New Module: memory-summarizer.ts - needsSummarization(): Check if conversation exceeds thresholds - summarizeConversation(): Generate summary via Gemini CLI - getMemoryContext(): Get formatted summary for injection Config (config.ts): - MEMORY.SUMMARIZE_THRESHOLD_CHARS: 50K chars trigger - MEMORY.MAX_CONTEXT_MESSAGES: 100 messages max - MEMORY.SUMMARY_PROMPT: Customizable summary template Database (db.ts): - memory_summaries table with upsert support - getGroupMessageStats() for threshold checking - getMessagesForSummary() for batch retrieval - deleteOldMessages() for archival Integration: - ContainerInput.memoryContext field added - agent-runner injects memory before prompt - index.ts imports getMemoryContext for each execution This enables automatic conversation compression when context grows too large, preserving important information while reducing token usage.

Closes qwibitai#6, relates to qwibitai#4 ## Changes ### `container/build.sh` - Default (no args) builds with `git rev-parse --short HEAD` SHA tag **and** re-tags as `latest` - Optional `REGISTRY` env var: push both SHA tag and `latest` to a registry after build - Explicit tag argument still works for one-off builds; skips `latest` re-tag in that case - Build output now prints both tags and includes rollback instructions ### `setup/container.ts` - Derives SHA tag via `git rev-parse --short HEAD` at build time (falls back to `'unknown'` in non-git environments) - Builds with SHA tag, then re-tags as `latest` - Adds `SHA_TAG` field to all `emitStatus` calls — setup output now records which exact commit was built - Test run uses the SHA-tagged image ## Verification After merge, `./container/build.sh` will produce: ``` nanoclaw-agent:<sha> ← immutable, traceable nanoclaw-agent:latest ← always current ``` To confirm a running container's commit: ```bash docker inspect nanoclaw-agent:latest --format '{{.Config.Labels}}' # or compare the SHA tag against git log ``` To roll back: ```bash docker tag nanoclaw-agent:<previous-sha> nanoclaw-agent:latest ``` Co-authored-by: Tony Santopietro <asantopietro@gmail.com> Reviewed-on: https://gitea.cluster.lab1.lan/asantopietro/nanoclaw/pulls/7 Co-authored-by: nanoclaw-bot <nanoclaw@nowhere.net> Co-committed-by: nanoclaw-bot <nanoclaw@nowhere.net>

Implements token optimization via RTK with clean enable/disable via config. - Added `RTK_ENABLED` (default: true) - Install RTK in containers - Added `RTK_AUTO_HOOK` (default: false) - Auto-intercept Bash commands - Both configurable via environment variables - Install RTK from GitHub releases (rtk-linux-x86_64) - Controlled by RTK_ENABLED build arg (default: true) - RTK auto-hook initialization controlled by RTK_AUTO_HOOK build arg - Build script passes config as build args - Added comprehensive RTK usage guide - Manual prefix mode examples (default) - Auto-hook mode explanation (experimental) - Token savings analytics with `rtk gain` - When to use/not use RTK - Added 6 RTK configuration tests - Test RTK_ENABLED defaults and behavior - Test RTK_AUTO_HOOK defaults and behavior - All 48 tests passing **Default (RTK enabled, manual prefix):** ```bash ./container/build.sh ``` Agents use `rtk git status`, `rtk npm test`, etc. for 60-90% token savings. **Disable RTK:** ```bash RTK_ENABLED=false ./container/build.sh ``` **Enable auto-hook (experimental):** ```bash RTK_AUTO_HOOK=true ./container/build.sh ``` All Bash commands transparently become `rtk <command>`. Use built-in RTK analytics (zero code): ```bash rtk gain # Last 7 days savings rtk gain --days 30 # Last 30 days rtk gain --json # Machine-readable ``` 1. Default: RTK enabled with manual prefix (this PR) 2. Test: Run 1-2 weeks, check `rtk gain` for actual savings 3. Evaluate: Review token consumption metrics 4. Decision: Enable `RTK_AUTO_HOOK=true` if results are good Closes qwibitai#6 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

…security Proposal #4 — Pre-Implementation Ticket Check (REQUIRED) Adds mandatory comment read before any implementation. Blocks start if comments contain "don't implement", "wait for", "architecture only", etc. Evidence: KRE-196 was implemented despite a comment saying architecture plan only — causing rework and mid-session user notification (2026-03-30). Proposal #5 — QA Delegation Enforcement Adds explicit FORBIDDEN clause: running lint/typecheck/tests via Bash does NOT satisfy the QA gate. Creating and immediately deleting a QA team does NOT satisfy the gate. Only a named QA subagent reporting PASS or FAIL counts. Evidence: 3rd consecutive session (2026-03-26, 2026-03-27, 2026-03-30) where PM bypassed QA subagent with shell commands. Proposal #6 — Token Security: No Literal Tokens in Task Prompts Adds rule to use GH_TOKEN=$(~/github-credential-helper.sh) instead of embedding resolved ghs_... values in SendMessage content. Evidence: 81 hardcoded token occurrences in session f497c6f8 JSONL transcript. Approved by Blake (operator) via Slack on 2026-03-30. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Critical bugs: - Bug qwibitai#4: Remove sequence from content hash to prevent duplicates on repeated PreCompact calls. Hash is now sha256(session_id:role:content). - Bug qwibitai#5: storeMessages() now returns count of newly inserted messages. PreCompact only creates leaf summaries for newly inserted messages, preventing re-summarization of already-stored content. Should-fix: - #1/qwibitai#10: Add dbInitialized flag to skip redundant schema setup on repeated initLcmDatabase() calls (fast path for MCP tool calls). - qwibitai#6: Depth-capped condensation now attempts API summarization first, falls back to truncated concatenation with 10K token cap instead of unbounded blob. - qwibitai#7: Skip API call entirely when neither ANTHROPIC_API_KEY nor ANTHROPIC_AUTH_TOKEN is set, go straight to deterministic fallback. Nice-to-have: - qwibitai#3: Remove duplicate LcmMessage/LcmSummary from src/types.ts. Single source of truth is container/agent-runner/src/lcm-store.ts.

…ation qwibitai#5 Warm container: Already supported via 30min IDLE_TIMEOUT + IPC piping. Docker startup is only 0.4s; Claude Code init is the real bottleneck. Full elimination requires embedded SDK (future). qwibitai#6 Frozen memory snapshot + prompt caching: - Query memU once at container start for relevant context - Inject as immutable system prompt prefix - Enables Anthropic's automatic prefix caching (50-75% token savings) - Memory context frozen for session duration (no mid-turn re-queries) qwibitai#7 Smart model routing: Skipped — Agent SDK doesn't expose model selection in query options. Would require SDK changes or CLI wrapper. qwibitai#8 FTS5 session search: - Added FTS5 virtual table on messages for full-text cross-session search - Auto-synced via INSERT/DELETE triggers - searchMessages() function for keyword-based message recall - Complements memU's semantic search with fast keyword search Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

All 7 container features re-applied after confirming the earlier hang was caused by API rate limiting, not code issues. SDK v0.2.76 confirmed to support abortController. #1 AbortController idle timeout (5min, configurable) qwibitai#3 Per-group .mcp.json config (auto-discovered servers + tools) qwibitai#6 Frozen memU memory snapshot (query once at start, hybrid RAG) qwibitai#11 Skill-as-markdown auto-loading from /workspace/group/skills/ qwibitai#12 Structured compaction summary (last-compaction-summary.md) qwibitai#13 Peer channel MCP tools (peer_send, peer_status via SSH) qwibitai#14 Recall MCP tool (FTS5 cross-session message search) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Read HTTP Content-Type header from audio download response instead of hardcoding audio/ogg; falls back to Matrix event info.mimetype - Map MIME type to correct file extension for OpenAI Whisper (webm, ogg, wav, flac, mp3, mp4) — fixes 400 errors from Chrome-based Element Web which sends audio/webm;codecs=opus - Add HTTP response status check before attempting transcription - Docs: add issues qwibitai#5 (container image loss) and qwibitai#6 (voice transcription MIME mismatch) to DEBUG_CHECKLIST.md with diagnosis commands - Docs: add Linux systemd commands to Service Management section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Two issues exposed when the new "nanoclaw using OneCLI" assertion (qwibitai#6 from the adversarial review) was deployed alongside the still-stale "credential proxy listening" check from section 6. Bug 1 — SIGPIPE under pipefail: journalctl ... | grep -q "Credential layer..." worked when run by hand but FAILed inside dr-test.sh under set -uo pipefail. Root cause: grep -q exits on first match, closing the pipe; journalctl gets SIGPIPE on its next write and exits 141; pipefail surfaces 141 as the pipeline exit code; the && branch doesn't fire and the || prints FAIL even though the line was present. Fix: capture journalctl output into a variable first, then grep against the variable. No pipeline → no SIGPIPE. Bug 2 — stale section 6 check in OneCLI mode: "credential proxy listening" greps ss -tln for :3001. With OneCLI active, credential-proxy.ts is not started, so the check always reports FAIL — a false positive. The dr report from #aios-alerts has been "FAILED — N check(s) need attention" since the Phase 2 flag flip and the section 7 nanoclaw-using-OneCLI check exposed it. Fix: skip the port 3001 check when ONECLI_API_KEY is set in /opt/nanoclaw/.env, reporting "skipped — OneCLI mode" as detail. Phase 3 will delete both this check and credential-proxy.ts entirely. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Critical bugs: - Bug qwibitai#4: Remove sequence from content hash to prevent duplicates on repeated PreCompact calls. Hash is now sha256(session_id:role:content). - Bug qwibitai#5: storeMessages() now returns count of newly inserted messages. PreCompact only creates leaf summaries for newly inserted messages, preventing re-summarization of already-stored content. Should-fix: - #1/qwibitai#10: Add dbInitialized flag to skip redundant schema setup on repeated initLcmDatabase() calls (fast path for MCP tool calls). - qwibitai#6: Depth-capped condensation now attempts API summarization first, falls back to truncated concatenation with 10K token cap instead of unbounded blob. - qwibitai#7: Skip API call entirely when neither ANTHROPIC_API_KEY nor ANTHROPIC_AUTH_TOKEN is set, go straight to deterministic fallback. Nice-to-have: - qwibitai#3: Remove duplicate LcmMessage/LcmSummary from src/types.ts. Single source of truth is container/agent-runner/src/lcm-store.ts.

Three parallel subagent reviews (correctness, tests, template/bot-flow) of 6e33f39 + f859aa6 flagged three material gaps. All addressed here. TEMPLATE — recoverable-error retry loop (Agent 3 finding qwibitai#4) The existing success:false handler at L567 said "If error exists, present it in {{LANGUAGE}}" — which degrades into "sorry, registration failed" when the engine rejects register_person with the new missing- fields error. The bot would just show the error to the user instead of parsing which fields are missing and asking for them. Added a new "Recoverable-error retry loop" bullet right after the generic handler that teaches the bot to: 1. Parse the missing-field list from the error text 2. Ask the user in ONE concise question for only those specific fields 3. Retry the SAME register_person call with the complete payload 4. Only degrade to "sorry, failed" if the user refuses or retry errors This is a two-turn conversation, not a failure. TEMPLATE — Cross-Board Assignee Guard 4-field reminder (Agent 3 qwibitai#6) M8 already cross-referenced the offer_register branch for handling cross-board reassignment rejects, but didn't inline the 4-field rule. Added an explicit note on the offer_register diagnose step so a reader landing there directly understands the hierarchy-board requirement without chasing the cross-reference to L545. TESTS — 3 new cases closing Agent 2 coverage gaps container/agent-runner/src/taskflow-engine.test.ts: 1. Hierarchy board missing ALL three fields (phone + group_name + group_folder) → error message lists all three. Guards the dynamic `missing.join(', ')` output so the bot can ask for everything in one prompt instead of discovering the fields one by one across multiple retries. 2. Legacy board with max_depth = NULL → canDelegateDown() returns false, so the validation does NOT fire and a 3-field register_person call succeeds. Pre-hierarchy installs (before the hierarchy schema was added) still have max_depth NULL in the boards table; this regression guard prevents my validation from breaking them. 3. offer_register on a LEAF board → message does NOT include the division/sigla ask. Counterpart to the existing assertion that the sigla IS present on the hierarchy fixture. Locks down the canDelegateDown() branch in buildOfferRegisterError. 218 container engine tests pass (up from 214). DEFERRED (pre-existing, out of scope): Agent 3 qwibitai#2 suggested injecting a pre-resolved {{IS_HIERARCHY_BOARD}} boolean into the generator instead of using literal {{HIERARCHY_LEVEL}} < {{MAX_DEPTH}} comparisons that render as "3 < 3" in rendered group prompts. The current pattern is functional — LLMs read "3 < 3 = false" correctly — but it's cognitively more expensive than a pre-resolved flag. Changing this would require generator refactoring and touches L293, L294, L534, L545 and other places that use the same pattern. Out of scope for the Edilson fix; worth a separate refactor commit later. REGEN 11 groups/*/CLAUDE.md re-rendered to pick up the L545 retry-loop and cross-board 4-field additions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…index Three parallel review agents (reuse, quality, efficiency) flagged actionable items on Phase 2: 1. TYPE SAFETY (Agent 1 qwibitai#5, Agent 2 qwibitai#4): The pending_approval response field was not declared in UpdateResult, and new AdminResult fields (offer_register, merged, source_archived, notes_added) were only returned via 'as any' casts. Fixed: - UpdateResult extended with pending_approval: { request_id, target_chat_jid, message, parent_board_id } - AdminResult extended with offer_register, merged, source_archived, notes_added (used by merge_project and handle_subtask_approval) - All 4 'as any' casts in the Phase 2 code removed - Also cleaned up `parentBoard?.group_jid ?? null` dead fallback (the null case was already made unreachable by the earlier fail-fast guard) 2. UNUSED INDEX (Agent 3 qwibitai#6): idx_subtask_requests_status on (status, target_board_id) was never queried — the dominant query is the PK lookup on request_id (O(1)). Removed the index and the drift-guard test that asserted its existence. Added a comment explaining the decision so a future scan-by-pending query can reinstate it. Deferred (acceptable at current scale): - subtask_requests grows unbounded: zero boards have opted into approval mode, so no current users. Revisit when adoption warrants. - N+1 insert in approve loop: typical batches < 10 subtasks, better-sqlite3 caches prepared statements. - decision field conflates handle_subtask_approval + process_minutes_ decision: matches existing codebase pattern. - Reject/approve notification duplication: only 2 sites with small variation, helper extraction would save ~2 lines. 236 engine / 365 skill tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Risk/execution overhaul — makes the bot safe to run on real money and replaces per-coin tuning with a universal volatility-normalized formula. Fix qwibitai#3 — Exchange-side SL/TP (real-money safety net): - Entry market orders now include attachAlgoOrds with ATR-derived slTriggerPx + tpTriggerPx, so positions are protected on the exchange even if the bot goes offline between cron cycles. - Main bot: new syncExchangeSl() amends the live conditional algo (cancel-and-recreate fallback via orders-algo-pending) whenever the local trailing-SL tier advances, keeping exchange SL in lockstep. - Store slAlgoClOrdId + currentExchangeSlPrice in state.openData. Fix qwibitai#4 — Session rules actually enforced: - New getCurrentSession() (UTC: asian<8, london<13, ny). - Entry logic now applies session_rules[session].max_leverage and size_multiplier instead of only passing them to Claude for logging. Fix #5a — OI divergence with real previous OI: - OKX has no public per-instrument historical OI endpoint, so roll a state.oiHistory cache: pushOiSample() writes current OI each cycle, pickOiPrev() returns the oldest sample inside a 2–8h window. - analyzeMarket() signature extended with oiPrev; both call sites (hold loop + scan loop) pass it through. - Kills the `analyzeOIDivergence(oiCurrent, oiCurrent, ...)` no-op that silently disabled 4 OI signal weights. Fix #5b — Funding-rate trap F&G fusion: - analyzeFundingTrap() now receives the real fg object instead of hardcoded {value: 50}; `combined_with_fear/greed` flags finally fire. Fix qwibitai#6 — Risk dialed down: - risk_per_trade_main: 0.08 → 0.01 (8x reduction; 1% per trade) - hard_limits.max_risk_per_trade: 0.08 → 0.02 (ceiling for selfEvolve) - max_positions_main: 2 → 4 (room for ETH/SOL/XRP/DOGE/SUI scan) - max_drawdown_main: 0.20 → 0.15 (tighter DD circuit-breaker) - Added risk_per_trade_lab 0.005, max_positions_lab 4, max_leverage_lab 5 - instruments[] now matches WATCHLIST (ETH/SOL/XRP/DOGE/SUI) - Params version bump 0 → 1 with update_reason=atr_universal_fix_v1 Universal per-instrument formula (replaces hardcoded per-coin tuning): - New atr(rawCandles, 14) helper computes True Range on 1H candles. - New getInstrumentProfile() returns {atr, atrPct, sizeMult, maxLev, k_sl=1.5, k_tp=3.0}. sizeMult is log-normalized from 24h USD volume ($10M→0.4, $1B→1.0). maxLev is volatility-capped: ~0.12/atrPct. - analyzeMarket() now computes and returns the profile per instrument. - Entry sizing replaces equity*risk*lev*sizeM/ctUsdVal with: riskUsd = equity × risk% × session × liquidity × signal notional = riskUsd × (price / (k_sl × ATR)) sz = notional / ctUsdVal So SUI/DOGE get smaller positions automatically (higher ATR%) while keeping the same USD risk as ETH. No per-coin tuning required. - Initial SL/TP prices derive from ATR: SL = entry ± 1.5×ATR, TP = entry ± 3.0×ATR (1:2 R:R), passed both to attachAlgoOrds and stored in state.openData for the tier-ladder override below. Trailing-SL ladder (main bot): - At t0 (hwm below t1 threshold), prefer the stored ATR-derived atrSlUplRatio over the fixed -7% margin-% default, so the initial stop scales with the coin's natural volatility. - Tiers t1..t5 unchanged; still drive exchange-SL sync on advance. Lab bot (okx-trader-lab.js): - Same helpers/fixes (ATR, profile, session rules, OI cache, FR fix, attachAlgoOrds at entry). - Skips exchange-side SL tier sync because lab's pnlPct is equity-scale rather than uplRatio-scale — initial attach SL/TP still provides the safety net.

…ons, DB schema changes - registered_groups: PK is now (jid, session_group), not global - sessions: PK is (group_folder, session_group), adds last_node tracking - messages: bot responses now stored (is_bot_message=1) - New env vars: SESSION_GROUP, CLAUDE_MODEL documented in all .env examples - Syncthing: added nanoclaw-main-sessions and nanoclaw-trading-sessions folders - Dev workflow: updated copy command to include db.ts, index.ts, task-scheduler.ts - New customizations: qwibitai#6 SESSION_GROUP isolation, qwibitai#7 CLAUDE_MODEL Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

gavrielc closed this Feb 1, 2026

TomGranot deleted the claude/async-file-watcher-szF5V branch February 12, 2026 15:51

gabi-simons mentioned this pull request Feb 26, 2026

fix(ipc): solve multi-channel collision and agent staleness #533

Closed

3 tasks

roi-shikler-thenvoi mentioned this pull request Mar 31, 2026

feat: add Thenvoi Platform channel integration thenvoi/nanoclaw-thenvoi#2

Closed

7 tasks

Nate-Vish mentioned this pull request Mar 31, 2026

Agent memory system redesign #1356

Open

5 tasks

gkarolyi pushed a commit to gkarolyi/nanoclaw that referenced this pull request Apr 7, 2026

feat: add /stop session command to interrupt running agent (qwibitai#6)

a2c1d7d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace IPC busy-loop polling with async fs.watch#6

Replace IPC busy-loop polling with async fs.watch#6
gavrielc wants to merge 1 commit intomainfrom
claude/async-file-watcher-szF5V

gavrielc commented Feb 1, 2026

Uh oh!

gavrielc commented Feb 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants