Skip to content

fix: detect session file rotation in bootstrap — compaction never triggers#190

Open
glfruit wants to merge 1 commit intoMartian-Engineering:mainfrom
glfruit:fix/session-file-rotation-bootstrap
Open

fix: detect session file rotation in bootstrap — compaction never triggers#190
glfruit wants to merge 1 commit intoMartian-Engineering:mainfrom
glfruit:fix/session-file-rotation-bootstrap

Conversation

@glfruit
Copy link
Copy Markdown

@glfruit glfruit commented Mar 26, 2026

Bug: Session file rotation breaks bootstrap — compaction never triggers, context overflows

Environment

  • lossless-claw: 0.5.2
  • OpenClaw: latest (gateway daemon mode)
  • Platform: macOS (darwin arm64), Node.js v25.x

Description

When OpenClaw rotates a session file (creates a new UUID.jsonl on /reset or natural rotation), LCM's bootstrap() method fails to detect and handle the change. The new session file's messages are never ingested into the LCM database, compaction never triggers, and the main session's context grows unbounded until it hits the context limit.

Root Cause

In src/engine.ts, the bootstrap() method tracks sessions by absolute file path in the conversation_bootstrap_state table. When a session file rotates:

  1. Line 1735bootstrapState.sessionFilePath === params.sessionFile is false (new path ≠ old path), so the "file unchanged" fast path is skipped. ✅ Correct so far.

  2. Line 1752 — Same path comparison in the incremental append path. Also skipped. ✅ Correct.

  3. Line 1844 — Falls through to reconcileSessionTail(). This reads the new file's messages and tries to find a "anchor" message that exists in both the new file and the DB. Since the new file is from a post-reset session, no messages overlap with the old DB data → anchorIndex = -1 → returns { importedMessages: 0, hasOverlap: false }.

  4. Line 1908 — Only updates bootstrapState when reconcile.hasOverlap === true. Since it's false, the old file path remains in the DB.

  5. Line 1910conversation.bootstrappedAt is true from the prior session → returns "already bootstrapped".

Result: A dead loop. Every subsequent bootstrap call sees the same stale path, takes the same reconcile path, finds no overlap, and never ingests new messages. Compaction depends on DB token counts which never grow → compaction never triggers.

Evidence from production

  • compaction_events table: 0 rows (never compacted)
  • conversations table: session tracked wrong .jsonl file (815KB, stale)
  • LCM produced 200 summaries on wrong/deleted files
  • Two agents hit context overflow with compactionAttempts: 0/1

Reproduction Steps

  1. Start an agent session that uses LCM as its context engine
  2. Verify LCM is working (bootstrap succeeds, summaries are created)
  3. Execute /reset or trigger session file rotation in OpenClaw
  4. Send messages in the new session
  5. Observe: new messages never appear in LCM DB
  6. Observe: compaction never triggers
  7. Eventually: context overflow

Proposed Fix

Add session file rotation detection in bootstrap(), right after fetching bootstrapState and before the path-comparison guards. When a path change is detected, purge all conversation data and re-bootstrap as a fresh conversation.

// After getting bootstrapState, before the "file unchanged" check:

if (
  bootstrapState &&
  bootstrapState.sessionFilePath !== params.sessionFile
) {
  console.error(
    `[lcm] bootstrap: session file rotated for session ${params.sessionId}: ` +
      `"${bootstrapState.sessionFilePath}" → "${params.sessionFile}" — resetting conversation ${conversationId}`,
  );
  // Purge all data for this conversation
  this.db.prepare(`DELETE FROM context_items WHERE conversation_id = ?`).run(conversationId);
  this.db.prepare(`DELETE FROM summary_messages WHERE summary_id IN (SELECT summary_id FROM summaries WHERE conversation_id = ?)`).run(conversationId);
  this.db.prepare(`DELETE FROM summary_parents WHERE summary_id IN (SELECT summary_id FROM summaries WHERE conversation_id = ?) OR parent_id IN (SELECT summary_id FROM summaries WHERE conversation_id = ?)`).run(conversationId, conversationId);
  this.db.prepare(`DELETE FROM summaries WHERE conversation_id = ?`).run(conversationId);
  this.db.prepare(`DELETE FROM conversation_bootstrap_state WHERE conversation_id = ?`).run(conversationId);
  this.db.prepare(`DELETE FROM large_files WHERE conversation_id = ?`).run(conversationId);
  this.db.prepare(`DELETE FROM messages_fts WHERE rowid IN (SELECT message_id FROM messages WHERE conversation_id = ?)`).run(conversationId);
  const msgCount = this.db.prepare(`DELETE FROM messages WHERE conversation_id = ?`).run(conversationId).changes;
  console.error(`[lcm] bootstrap: purged ${msgCount} messages and all summaries for conversation ${conversationId}`);
  // Reset local state — existingCount must also be 0 for first-import path
  bootstrapState = null;
  existingCount = 0;
}

Also requires changing const existingCountlet existingCount and const bootstrapStatelet bootstrapState.

Trade-offs

  • Old summaries are lost on rotation. This is acceptable because the conversation was reset — old summaries refer to a conversation that no longer exists.
  • Alternative approaches (archiving old data, creating new conversation records) are more complex and provide little value since the old session is gone.

Secondary Issue (unrelated)

ZAI_API_KEY was found empty (length=0) in the LCM auth cascade, which would cause summarization to fail auth even if bootstrap worked. This is a configuration issue, not a code bug.

When OpenClaw rotates a session file (new UUID.jsonl on /reset or natural
rotation), the bootstrap method previously:

1. Detected the path mismatch (new path ≠ stored path)
2. Fell through to reconcileSessionTail()
3. Found no message overlap with the old DB data → hasOverlap=false
4. Skipped persistBootstrapState() (only called when hasOverlap=true)
5. Returned 'already bootstrapped' — never ingesting new messages

This created a dead loop: every subsequent bootstrap hit the same stale
path, compaction never triggered, and context grew unbounded.

Fix: detect the path mismatch immediately after fetching bootstrapState,
purge all conversation data (context_items, summaries, messages, FTS),
reset bootstrapState=null and existingCount=0, then fall through to the
first-import path which handles the new file correctly.

Trade-off: old summaries are lost on rotation, which is acceptable since
the old conversation no longer exists.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant