Skip to content

feat: replace file-based IPC with Unix sockets#667

Closed
echthesia wants to merge 3 commits intoqwibitai:mainfrom
echthesia:feat/unix-socket-ipc
Closed

feat: replace file-based IPC with Unix sockets#667
echthesia wants to merge 3 commits intoqwibitai:mainfrom
echthesia:feat/unix-socket-ipc

Conversation

@echthesia
Copy link
Copy Markdown

@echthesia echthesia commented Mar 3, 2026

Summary

  • Replace polling-based file IPC between host and container agents with NDJSON over Unix sockets for near-instant message delivery
  • Host creates a per-group socket at data/ipc/{group}/nc.sock before container spawn; container MCP server and agent-runner connect as clients
  • Snapshot files (current_tasks.json, available_groups.json) remain as files — they're pre-start state, not ongoing IPC
  • Eliminates two polling loops (1s host, 500ms container) and all atomic-rename/readdir/unlink bookkeeping

Changes

Core

  • New src/ipc-socket.ts: NdjsonParser class and IpcSocketServer with per-group socket lifecycle
  • src/ipc.ts: Removed startIpcWatcher() and polling loop; kept IpcDeps interface and processTaskIpc()
  • src/config.ts: Removed IPC_POLL_INTERVAL
  • src/group-queue.ts: sendMessage() and closeStdin() now use socket instead of file writes
  • src/container-runner.ts: Socket create before spawn, destroy on close; simplified IPC dir to single mkdirSync
  • src/index.ts: Wired createIpcSocketServer() before scheduler loop, added shutdown handler

Container

  • container/agent-runner/src/index.ts: New IpcClient class replaces file polling with socket event handling; connectIpcSocket() with retry backoff
  • container/agent-runner/src/ipc-mcp-stdio.ts: sendIpcMessage() over lazy socket connection replaces writeIpcFile()
  • container/Dockerfile: Removed stale messages/, tasks/, input/ subdirs

Skills

  • Apple Container: Updated container-runner.ts modify + Dockerfile modify
  • Gmail: Updated container-runner.ts and agent-runner index.ts modify files
  • X-integration: Converted writeIpcFile() to sendIpcMessage() in agent.ts
  • Telegram swarm: Updated SKILL.md IPC references

Other

  • Version bump to 1.3.0 with [BREAKING] CHANGELOG entry
  • Updated docs (SPEC.md, debug skill, main group CLAUDE.md)
  • NdjsonParser unit tests + socket integration tests

Test plan

  • npm run typecheck — clean compile
  • npm test — 299 passing (12 new), no regressions
  • ./container/build.sh — rebuild container image
  • Send a message via channel, verify socket-based IPC end-to-end
  • Test scheduled tasks (schedule_task, pause_task, cancel_task)
  • Verify follow-up messages arrive instantly (no 500ms polling delay)
  • Verify idle timeout and close signal work via socket

🤖 Generated with Claude Code

echthesia and others added 3 commits March 2, 2026 21:13
Replace polling-based file IPC between host and container agents with
NDJSON over Unix sockets for near-instant message delivery.

Host creates a per-group socket at data/ipc/{group}/nc.sock before
container spawn. Container MCP server and agent-runner connect as
clients. Messages flow as newline-delimited JSON. Snapshot files
(current_tasks.json, available_groups.json) remain as files.

- Add src/ipc-socket.ts with NdjsonParser and IpcSocketServer
- Remove startIpcWatcher() and IPC_POLL_INTERVAL
- Update group-queue to send via socket instead of file writes
- Update container-runner with socket lifecycle (create/destroy)
- Rewrite container agent-runner with IpcClient (socket-based input)
- Rewrite container MCP server to use socket instead of writeIpcFile()
- Remove stale IPC subdirs from Dockerfiles
- Update docs (SPEC.md, debug skill, main group CLAUDE.md)
- Add NdjsonParser unit tests and socket integration tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update Apple Container and Gmail skill modify files for socket IPC
  (remove IPC subdirs, simplify mkdirSync)
- Convert X-integration agent.ts from writeIpcFile to sendIpcMessage
- Update Telegram swarm SKILL.md references to socket-based IPC
- Update Gmail index.ts.intent.md to reference IpcClient
- Bump core_version to 1.3.0 in affected skill manifests
- Bump package.json to 1.3.0
- Add [BREAKING] CHANGELOG entry with migration instructions
- Remove unused private socket field from IpcClient
- Remove sandbox special-casing from socket tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of letting invalid JSON warnings leak to test output,
mock logger.warn and assert the correct lines are logged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@echthesia
Copy link
Copy Markdown
Author

Closed due to macOS not supporting bind mounts of UNIX sockets

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant