feat(channel/bb): group mention gating — require_mention_in_groups#2585
Closed
maxtongwang wants to merge 115 commits intozeroclaw-labs:mainfrom
Closed
feat(channel/bb): group mention gating — require_mention_in_groups#2585maxtongwang wants to merge 115 commits intozeroclaw-labs:mainfrom
maxtongwang wants to merge 115 commits intozeroclaw-labs:mainfrom
Conversation
Adds @claude mention support on PRs using anthropics/claude-code-action@v1. Mirrors the workflow from maxtongwang/openclaw. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds use_sticky_comment: true to claude-review workflow so all review updates are consolidated into a single comment instead of flooding the PR with multiple comments. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds BlueBubblesChannel — iMessage via self-hosted BlueBubbles macOS server. - Webhook mode: receives new-message events at POST /bluebubbles - Sender normalization: strips imessage:/sms:/auto: prefixes, lowercases emails - fromMe caching: FIFO VecDeque+HashMap cache with reply context injection - Optional inbound webhook auth: Authorization: Bearer via webhook_secret config - Outbound send: POST /api/v1/message/text with ?password= query param + tempGuid - Health check: GET /api/v1/ping (matches OpenClaw probeBlueBubbles) - constant_time_eq() for secret comparison; manual Debug redacts password Closes #1 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a second BlueBubbles endpoint for a personal Apple ID that stores incoming iMessages to shared memory but never calls the LLM or sends a reply. This enables multi-tenant iMessage setups where one ZeroClaw process handles both a bot account (POST /bluebubbles) and a personal account (POST /bluebubbles-personal) from two BlueBubbles instances. Changes: - src/config/schema.rs: add bluebubbles_personal: Option<BlueBubblesConfig> to ChannelsConfig (reuses existing BlueBubblesConfig struct) - src/gateway/mod.rs: add bluebubbles_personal field to AppState, wire construction, register POST /bluebubbles-personal route, and implement handle_bluebubbles_personal_webhook handler Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- [build] rustc-wrapper = "sccache" in .cargo/config.toml — caches compiled crates across clean builds; sccache auto-starts on first use - [profile.dev] split-debuginfo = "unpacked" in Cargo.toml — skips dsymutil on macOS, eliminating post-link debug-symbol packaging delay Note: mold 2.40.4 dropped Mach-O support so it cannot be used as a macOS linker; split-debuginfo is the primary link-time win on macOS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Convert Discord-style markdown in LLM responses to BB Private API
attributedBody segments for iMessage rendering:
- **bold** → {bold: true}
- *italic* → {italic: true}
- ~~strikethrough~~ → {strikethrough: true}
- __underline__ → {underline: true}
Markers nest arbitrarily; plain-text fallback strips markers.
Update system prompt to instruct LLM to use rich markdown + emoji.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add start_typing/stop_typing to BlueBubblesChannel using BB Private API
POST /api/v1/chat/{chatGuid}/typing. Indicator refreshes every 4s since
BB typing events expire in ~5s. Gateway handler starts indicator before
LLM call and stops it (both success and error paths) before sending reply.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- markdown_to_attributed_body: convert **bold**, *italic*, ~~strike~~, __underline__, `code` (→bold), # headers (→bold), ``` blocks (plain) into BB Private API attributedBody segments - start_typing/stop_typing: background loop refreshing BB typing indicator every 4s while LLM processes; aborted on response - EFFECT_MAP + extract_effect(): LLM can append [EFFECT:name] tags (slam, loud, gentle, invisible-ink, confetti, balloons, fireworks, lasers, love, celebration, echo, spotlight); stripped from content, passed as effectId to BB Private API - Updated channel_delivery_instructions with full style + effect guide - 38 unit tests passing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add ignore_senders to BlueBubblesConfig and BlueBubblesChannel - Personal webhook handler skips storing messages where sender is in ignore_senders - Configure ignore_senders = ["tongtong901005@gmail.com"] in config.toml - Fix release-fast profile: lto=false, codegen-units=16, opt-level=2 for fast local builds with sccache support Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Direct LLM to use web_fetch on canonical sources (ESPN, wttr.in, etc.) for sports, weather, news — not just web_search snippets - Remove "be concise" instruction that was cutting the tool loop short - Instruct bot to complete full research before replying - Keep all text style and effect guidance Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…l endpoint Port all three CodeRabbit improvements from feat/channel-bluebubbles to fork main and extend them consistently to the bluebubbles_personal endpoint: - typing_handles: replace single Mutex<Option<JoinHandle>> with Mutex<HashMap<String, JoinHandle>> keyed by chat GUID so concurrent conversations do not cancel each other's typing indicators - is_sender_ignored(): new method checked before is_sender_allowed() in parse_webhook_payload so ignore_senders always takes precedence over the allowlist; removes the now-redundant inline ignore check from the personal handler since parse_webhook_payload handles it upstream - Secret wiring: add bluebubbles and bluebubbles_personal password + webhook_secret to decrypt_channel_secrets and encrypt_channel_secrets - Debug impl: add ignore_senders field to BlueBubblesConfig debug output - Tests: add three unit tests covering ignore_senders exact match, precedence over allowlist, and empty-list no-op behaviour (41 total) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Conflicts resolved: - .gitignore: keep fork's test-config.toml entry alongside upstream result - Cargo.lock: accept upstream (no new deps from BB channel) - src/config/schema.rs: keep BlueBubblesConfig + bluebubbles/personal fields alongside upstream GitHubConfig; wire both into encrypt/decrypt - src/gateway/mod.rs: keep BB + personal handlers alongside upstream GitHub handler; update run_gateway_chat_with_tools calls to 3-arg signature; add missing BB fields to new GitHub test fixtures Also fixes: - src/providers/cursor.rs: add missing quota_metadata field (pre-existing upstream bug now caught by stricter ChatResponse struct) - BB handler: update sanitize_gateway_response to 3-arg form matching upstream API change (adds leak_guard parameter) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BlueBubbles channel now downloads and transcribes audio attachments when `[transcription]` is configured. iMessage sends voice memos in Core Audio Format (CAF) which Groq Whisper does not accept natively; CAF files are converted to WAV via ffmpeg before upload. Changes: - transcription.rs: add `convert_caf_to_wav` — detects CAF by extension, writes to temp files (seekable input required), runs `ffmpeg -ar 16000 -ac 1`, cleans up on all exit paths; returns Ok(None) for non-CAF so callers reuse original bytes - bluebubbles.rs: add `transcription` field, `with_transcription` builder, `AudioAttachment` struct, `extract_audio_attachments`, `download_attachment` (percent-encoding inline), `download_and_transcribe`, and public async `parse_webhook_payload_with_transcription`; gracefully falls back to `<media:audio>` placeholder when transcription is disabled, ffmpeg is absent, or the API call fails - gateway/mod.rs: wire `config.transcription.clone()` into both BB channel constructions; replace synchronous `parse_webhook_payload` call with async `parse_webhook_payload_with_transcription` in both webhook handlers No new Cargo dependencies — uses tokio::process and tokio::fs already present. No new config keys — wired through the existing `[transcription]` block. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors OpenClaw approach: read attachment from `original_path` in BB webhook payload (local filesystem, no REST download) and shell out to the `whisper` CLI. Python whisper handles CAF (iMessage voice memos) natively via ffmpeg — no pre-conversion step needed. - Remove convert_caf_to_wav + ffmpeg + Groq HTTP download path - Add transcribe_audio_local: resolves whisper in PATH and common Homebrew/system paths, runs with --model turbo --output_format txt, reads <tmpdir>/<stem>.txt, cleans up on all exit paths - extract_audio_attachments: use originalPath instead of guid - transcribe_local: read file directly, no REST API call - Fix pre-existing build break: add OauthConfig/GoogleOauthConfig schema types + re-export so oauth.rs compiles Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
BB webhook attachments do not include originalPath; extract guid instead
and download bytes via /api/v1/attachment/{guid}/download, then write to
a temp file and pass to the local whisper CLI for transcription.
BB converts CAF→MP3 before webhooking so the download is typically
audio/mpeg — no pre-conversion needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two fixes:
1. Instead of REST API download, read the BB-converted MP3 directly from
the local filesystem at the predictable path:
~/Library/Application Support/bluebubbles-server/Convert/{guid}/{name}.mp3
BB converts CAF→MP3 before sending the webhook; since ZeroClaw runs on
the same Mac as BB, no network call is needed.
2. Spawn webhook processing in background (tokio::spawn) so the handler
returns 200 immediately. BB times out webhooks at 30 s, but whisper
transcription takes longer — caused 408 timeouts with the previous
synchronous approach.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BB's transferName in webhooks is inconsistent — sometimes includes .mp3
already, causing triple-extension paths. Instead of constructing the
filename from transferName, scan the Convert/{guid}/ directory for the
first audio file (mp3/m4a/aac/wav/ogg). This is robust to any BB
filename convention.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace incorrect \!escaped.contains('&') assertion — HTML entity encoding uses & characters (&, <, etc.), so this always failed. Assert contains("&") instead.
Replaces broken filesystem-based audio transcription with BB REST API download. Adds dual-backend transcription (local whisper CLI primary, Groq API fallback), inbound tapback surfacing via updated-message webhooks, and outbound reactions via add_reaction/remove_reaction on the Channel trait. Key changes: - download_attachment_bytes: 25 MB cap (Content-Length + body), 30s HTTP timeout - resolve_whisper_bin: OnceLock cache — binary search runs once at startup - parse_tapback_event: surfaces ❤️/👍/👎/😂/‼️ /❓ as [Tapback] system messages - add_reaction/remove_reaction: BB Private API /api/v1/message/react - 58 tests, all green
fetch_with_http_provider used Policy::none() but returned Ok(redirected_url) instead of fetching the target. Replace response with the fetched redirect target and fall through to shared body processing.
Brings in upstream commits through 3726d82. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> # Conflicts: # .cargo/config.toml # .gitignore # Cargo.toml # src/channels/bluebubbles.rs # src/channels/mod.rs # src/config/mod.rs # src/config/schema.rs # src/gateway/mod.rs
…nc parse Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ng indicator (#14) - Replace slow Python whisper (~30–90 s) with whisper-cli (whisper.cpp, Metal-accelerated, ~1–2 s) for local audio transcription. - Pre-convert all non-WAV audio to 16 kHz mono WAV via ffmpeg; the brew build of whisper-cli only reliably reads WAV. - Use OnceLock for binary/model path resolution to avoid repeated lookups. - Start typing indicator before transcription runs so users see feedback immediately rather than waiting for the full whisper phase. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…k reactions, and whisper-cli fast-path Extends the BlueBubbles iMessage channel with the following capabilities on top of the existing base implementation: - REST API audio download: fetches voice memo attachments directly from the BlueBubbles server via its REST API, enabling reliable access without filesystem access. - Tapback reactions: parses and surfaces reaction events (heart, thumbs up, thumbs down, ha ha, !!, ?) so the agent can understand user sentiment and react appropriately. - Rich text / attributed body: parses Apple's attributedBody format to extract plain text from styled iMessage content, improving message fidelity for formatted messages and inline attachments. - Typing indicator before transcription: sends a typing indicator immediately when a voice memo is detected, giving users real-time feedback before the potentially slower transcription step. - Local whisper-cli fast-path (whisper.cpp): prefers the locally installed `whisper-cli` (from whisper.cpp, e.g. via `brew install whisper-cpp`) over the Groq API when available. On Apple Silicon with Metal acceleration this cuts transcription latency from 30-90 s to ~1-2 s. WAV files are passed directly; non-WAV audio is converted via ffmpeg before passing to whisper-cli. Falls back to the Groq API automatically when whisper-cli is not installed. - Stderr diagnostics: surfaces whisper-cli stderr output in agent logs when transcription fails, making it easier to debug environment issues. Files changed: - src/channels/bluebubbles.rs: AudioAttachment struct, REST download, tapback parsing, rich text extraction, typing indicator wiring - src/channels/transcription.rs: local whisper-cli fast-path with ffmpeg pre-conversion and graceful Groq API fallback - src/gateway/mod.rs: wire .with_transcription() into channel construction; call parse_webhook_payload_with_transcription (async)
- Redact reqwest password URL from error messages with `.without_url()` in download_attachment_bytes, add_reaction, and remove_reaction - Replace racy `as_nanos()` temp-file names with `uuid::Uuid::new_v4()` in bluebubbles.rs (audio download) and transcription.rs (ffmpeg/whisper) - Gate typing indicator on fromMe/allowlist checks before `start_typing()` to avoid leaking presence for ignored/disallowed senders - Add `tokio::time::timeout(120s)` + `kill_on_drop(true)` for ffmpeg, whisper-cli, and python whisper subprocesses to bound execution time - Add Intel Homebrew model paths (`/usr/local/opt/whisper-cpp/...`) to MODELS array alongside Apple Silicon paths - Move transcription+LLM loop into `tokio::spawn` background task and return 202 ACCEPTED immediately to stay within gateway timeout budget Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move startup warning from hot-path mention_required_but_missing to with_mention_gating constructor so it fires once at config load, not on every inbound group message - Skip wildcard "*" sentinel when falling back to allowed_senders for keyword resolution; "*" means allow-all and must not be a literal mention keyword - Pre-lowercase mention_keyword in with_mention_gating to avoid repeated allocation on every inbound message comparison - Add 11 unit tests covering all branches: DM passthrough, global flag, keyword match/mismatch, case-insensitive match, fallback to allowed_senders, wildcard skip, no-keyword always-block, and all three per-group override combinations
- Reject blank/whitespace-only mention_keyword (empty string would match
every message via contains(""), silently disabling gating); trim and
filter in with_mention_gating before pre-lowercasing
- Fix typing indicator side-effect: apply mention gate before
start_typing in parse_webhook_payload_with_transcription; voice notes
can never contain a keyword, so use "" as the text argument to let the
group-policy/per-group override decide
- Document tapback bypass: tapbacks intentionally skip mention gating
since reactions carry no user-typed text and can never contain the
keyword; add explanatory comment in parse_tapback_event
- Add 2 unit tests for empty and whitespace-only keyword edge cases
Document require_mention_in_groups, mention_keyword, and per-group override config in the BlueBubbles (§4.20) config block of channels- reference.md. Addresses missing documentation flagged in PR review. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…IDs from Debug output
…note; add BB to ops keyword matrix - Correct comment: 'explicitly @mentioned' → case-insensitive keyword match - Add: 'DMs are never mention-gated.' to comment block and to policy prose - Add silent-drop behavior sentence after group_allow_from policy prose - Add BlueBubbles row to operations keyword matrix (between Nextcloud Talk and iMessage)
…ll locales vi (full update): - channels-reference: add §4.15 BlueBubbles with full config (dm_policy, group_policy, group_allow_from, send_read_receipts, text_chunk_limit, chunk_mode, mention gating); add BlueBubbles row to ops keyword matrix §7.2; add BlueBubbles to rg filter §7.1 - config-reference: add [channels_config.bluebubbles] section with full key table el (full update): - channels-reference: add §4.1 BlueBubbles config snippet with policy fields - config-reference: add BlueBubbles toml example with dm_policy/group_policy/send_read_receipts zh-CN/ja/ru/fr (hub parity): - channels-reference: add BlueBubbles to channel list in scope section - config-reference: add update note listing new bluebubbles config keys
…blesConfig Debug impl
…dact mention_keyword in Debug
The per-group mention override map (groups) was omitted from the localized config-reference update notes in the French, Japanese, and Russian translations. Add it to each locale's changelog summary alongside the other new keys.
Author
|
@coderabbitai resolve The docs/i18n/{fr,ja,ru}/config-reference.md files already include |
Contributor
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 1927
Resolving all related review comments now. [resolve] 🧠 Learnings used |
Author
|
@coderabbitai full review |
Contributor
✅ Actions performedFull review triggered. |
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
- Add missing `groups` key to zh-CN config-reference update notice (fr/ja/ru already had it from prior commit) - Preserve whisper-cli error in transcribe_audio_local when Python fallback also fails — combined error message surfaces both causes (same fix as applied in PR #2495 transcription path)
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
…4.20 Link was pointing to §4.15 which doesn't exist; BlueBubbles is §4.20 in channels-reference.md. Update the VI config-reference cross-link.
…ument mention_keyword warning - oauth.rs: include state token in PKCE filename to prevent concurrent /auth/google starts from clobbering each other's session file; validate state format before constructing path (prevent path traversal) - mod.rs: fix stale comment "Returning 200" → "Returning 202" to match actual HTTP 202 Accepted response - config-reference.md: document that startup emits a warning when require_mention_in_groups is set but no mention keyword can be resolved Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g disabled Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- gateway/oauth: remove unused redirect field, fix TOCTOU race in revoke path, log Google token revocation failures - gateway/mod: hash sender PII before logging, fire-and-forget mark_read calls - i18n/vi: add BlueBubbles exception note and update allowed_senders list - i18n/el: update allowlist policy section with BlueBubbles dm_policy note
…okio::fs Replace std::fs::remove_dir_all with tokio::fs::remove_dir_all in the transcribe_with_whisper_cpp error path on read failure. The previous implementation was calling a blocking filesystem operation inside an async context and bypassing temp directory cleanup atomicity.
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Author
|
Closing — no longer needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mainBlueBubblesConfig:require_mention_in_groups: bool(global flag, defaultfalse),mention_keyword: Option<String>(keyword to match, defaults to firstallowed_sendersentry), andgroups: HashMap<String, BlueBubblesGroupConfig>(per-chat-GUID overrides). When a group message arrives and mention gating is active, the message text is case-insensitively checked for the keyword; non-matching messages are silently dropped (no response, no error sent). Per-group overrides can enable or disable gating for individual group GUIDs.Label Snapshot (required)
risk: lowsize: Schannel,configchannel: bluebubblesChange Metadata
featurechannelLinked Issue
Supersede Attribution (required when
Supersedes #is used)N/A
Validation Evidence (required)
Security Impact (required)
Privacy and Data Hygiene (required)
mention_keywordis a bot-trigger word, not PII.Compatibility / Migration
require_mention_in_groupsdefaults tofalse; existing deployments unaffected.i18n Follow-Through (required when docs or user-facing wording changes)
Human Verification (required)
groupsmap uses global flag.Side Effects / Blast Radius (required)
src/channels/bluebubbles.rs(message dispatch),src/config/schema.rs(three new fields + nested struct).mention_keywordcould silently drop all group messages. Warning is emitted on startup when no keyword is resolvable.Agent Collaboration Notes (recommended)
AGENTS.md+CONTRIBUTING.md.Rollback Plan (required)
require_mention_in_groups/mention_keyword/groupsfrom schema + revert dispatch check.require_mention_in_groups = truein config.require_mention_in_groups = true.Risks and Mitigations
mention_keywordcan be spoofed (any group member types it).allowed_senders/group_allow_fromremain the security gates.Summary by CodeRabbit
New Features
Documentation
Improvements
Chores
Review Fixes (Round 2)
Missing mention-gating docs: Added
require_mention_in_groups,mention_keyword, and per-group override config documentation to the BlueBubbles section (§4.20) ofdocs/channels-reference.md.Also propagated from base branches: all fixes from PRs #2461–#2584.
Review Fixes (Round 6)
oauth.rs: propagate HTTP client build failure — no silent timeout lossoauth.rs: validate PKCE state before consuming session file (CSRF fix)oauth.rs: use POST form body for token revocation (RFC 7009 compliance)transcription.rs: explicit fail on non-UTF-8 temp paths, not empty-string passthroughtranscription.rs: clean up partial ffmpeg WAV on conversion failurebluebubbles.rs: remove attachment GUID from error logclaude-review.yml: fix step-level PR condition (!= ''→!= null)Review Fixes (Round 7)
bluebubbles.rs: mention keyword comparison now lowercases both sides — previously only the incoming text was lowercased, so a keyword fromallowed_senderswith uppercase letters would never matchschema.rs:BlueBubblesConfigcustom Debug output redactsgroupsHashMap to count only — map keys are chat GUIDs containing phone numbers/emailsReview Fixes (Round 8)
docs/channels-reference.md: fix mention-gating comment block — changed "explicitly @mentioned" to "contain a mention keyword"; added note "Case-insensitive keyword match"; added "DMs are never mention-gated." to both comment and policy prose paragraphdocs/channels-reference.md: add silent-drop note after group_allow_from prose — documents that keyword mismatch results in silent drop with no response and no errordocs/channels-reference.md: add BlueBubbles (gateway) row to ops keyword matrix §7.2 — lists startup, auth, and failure log keywords for fast triagechannels-reference.md;[channels_config.bluebubbles]key table added toconfig-reference.md; ops matrix row and rg filter updatedchannels-reference.mdandconfig-reference.mdchannels-reference.mdscope updated to mention BlueBubbles;config-reference.mdupdate notes list new bluebubbles config keysReview Fixes (cargo fmt)
cargo fmt --allto files changed in this PR's diff — collapsed single-use iterator chains and reformatted match arm bodies to comply with project rustfmt settings