perf: fix multi-tenant inference latency (per-conversation locking + workspace indexing)

[henrypark133]

Problem

Multi-tenant conversational turns were taking ~20s E2E for messages that required only ~1.7s of actual LLM inference. Trace analysis revealed two bottlenecks in the bridge layer:

• 6s pre-LLM gap: bridge::router blocked before launching the Python orchestrator
• 7s post-execution gap: bridge blocked after thread execution finished before logging completed

Root cause: every engine state write triggered full FTS/vector document chunking via workspace.write(), flooding the shared DB connection pool (default: 10 connections) with chunk INSERTs. This cascaded into lock contention in ConversationManager where a global RwLock was held across slow DB writes — serializing all concurrent tenants.

Changes

Fix 1 — Skip workspace indexing for engine runtime state

Engine execution state files (engine/.runtime/, engine/projects/, engine/orchestrator/failures.json, engine/README.md) are machine-generated blobs, not semantic documents. They no longer trigger reindex_document_with_metadata(), version row creation, or FTS/vector indexing on write. The normalize_path() does not resolve .. segments — an explicit !path.contains("..") guard prevents path traversal bypass.

Fix 2 — Per-conversation mutex in ConversationManager

Replaces RwLock<HashMap<ConversationId, ConversationSurface>> with RwLock<HashMap<ConversationId, Arc<Mutex<ConversationSurface>>>>. The global RwLock is now a directory only (held for brief HashMap lookups). A single get_conversation_lock() helper encapsulates the two-step access pattern and enforces the lock ordering invariant in one place.

• Different conversations now run fully in parallel
• Closes the TOCTOU double-spawn race in handle_user_message
• Fixes record_thread_outcome 7s post-execution gap
• record_thread_outcome and clear_conversation now propagate Err when conversation not in memory (previously silently returned Ok)
• bootstrap_user upserts channel_user_index even for already-present conversations (repairs stale index entries after rollback)
• list_conversations pre-filters via channel_user_index before acquiring per-conv locks — O(matched) not O(all)
• entries_snapshot clone deferred to None branch only (inject/resume paths skip the allocation)

Fix 3 — DATABASE_POOL_SIZE default 10→30

Immediate pool headroom while structural fixes take effect. Configurable via env var. .env.example updated.

Tests

• concurrent_handle_user_message_spawns_one_thread — two concurrent messages to same conversation; asserts exactly one thread spawned (validates TOCTOU fix)
• record_thread_outcome_unknown_conv_returns_err — validates error propagation fix
• engine_projects_path_skips_indexing_but_knowledge_does_not — validates workspace fix coverage
• runtime_path_writes_produce_no_versions — validates versioning suppression

267 tests passing, 0 failures, 0 clippy warnings

Known limitations (documented in code)

• list_conversations is a best-effort snapshot, not point-in-time consistent (accepted trade-off of two-phase collect)
• get_or_create_conversation has a narrow concurrent-creation race where a caller that races between insert and rollback may hold a transiently unpersisted ConversationId (documented with comment)
• record_thread_outcome in-memory mutations are not rolled back if save_conversation fails (accepted, documented)
• clear_engine_conversation in router.rs has a pre-existing TOCTOU where threads spawned concurrently may be cleared without being stopped (pre-existing, out of scope)

[gemini-code-assist]

Code Review

This pull request introduces a granular locking strategy for the ConversationManager by wrapping individual conversations in Arc<Mutex<...>>, enabling parallel processing of messages across different conversations. It also optimizes database performance by increasing the default connection pool size and excluding machine-generated runtime state files from indexing and versioning. A performance concern was raised regarding list_conversations, which currently iterates over all conversations to filter by user ID; utilizing the channel_user_index for pre-filtering is suggested to reduce lock contention and improve efficiency in multi-tenant environments.

[Copilot]

Pull request overview

Reduces multi-tenant inference latency by removing expensive workspace indexing from engine runtime-state writes and by eliminating cross-tenant serialization in the in-memory conversation router via per-conversation locking.

Changes:

• Skip FTS/vector indexing and versioning for machine-generated engine runtime-state paths on Workspace::write().
• Replace a global conversations RwLock with per-conversation Mutexes in ConversationManager to allow parallel turns across conversations and eliminate double-spawn races.
• Increase the default DATABASE_POOL_SIZE (10 → 30) and update .env.example.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
src/workspace/mod.rs	Adds engine runtime path classification and short-circuits write-time metadata/versioning/indexing for those paths; adds tests for indexing/version suppression.
crates/ironclaw_engine/src/runtime/conversation.rs	Introduces per-conversation mutex locking and refactors message routing/outcome recording; adds concurrency-focused tests.
src/config/database.rs	Raises default DB pool size to reduce contention under load.
.env.example	Documents the new default pool size and deployment guidance.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[henrypark133]

Thanks for the review @gemini-code-assist! The list_conversations suggestion was a good catch — applied your exact suggestion in 60608d8. It now pre-filters via channel_user_index before acquiring any per-conversation locks, reducing lock scope from O(all conversations) to O(user's conversations). Thread resolved.

[gemini-code-assist]

That sounds great, @henrypark133. Reducing the lock scope to O(user's conversations) is a significant improvement for multi-tenant performance. The implementation details look solid, and the added tests should help prevent regressions in this area. Thanks for the update!

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.