Skip to content

fix: let pi handle context overflow instead of stopping the loop#45

Open
elecnix wants to merge 1 commit into
davebcn87:mainfrom
elecnix:fix-context-compaction
Open

fix: let pi handle context overflow instead of stopping the loop#45
elecnix wants to merge 1 commit into
davebcn87:mainfrom
elecnix:fix-context-compaction

Conversation

@elecnix
Copy link
Copy Markdown

@elecnix elecnix commented Apr 9, 2026

Problem

When context becomes nearly full during an autoresearch run, the extension was proactively calling ctx.abort() and disabling autoresearchMode, stopping the experiment loop instead of letting it continue.

Comparison with PR #41

PR #41 takes a different approach: it keeps the proactive isContextExhausted check but adds an autoCompactResume: true config option to compact and resume instead of stopping.

PR #41 This PR (#45)
Proactive check kept (opt-in via autoCompactResume) removed
Trigger token-history prediction overflow error from LLM
Fires when at start of run_experiment only when LLM actually overflows
Default stops (preserves original behavior) stops (less code)

Why this PR removes the proactive check:

  1. It fires before the LLM responds — based on predictions (token history), not actual overflow
  2. Predictions are unreliable — the chars/4 heuristic is conservative; large experiment output doesn't mean the LLM will overflow
  3. pi already handles this_checkCompaction detects actual context overflow errors from the LLM, auto-compacts, and retries automatically
  4. If the prediction is wrong, you've aborted for nothing — the turn stops, the session reconstructs, and you've added latency for no reason

What This PR Does

Remove the proactive isContextExhausted check entirely:

  • Overflow mid-experiment: pi's _checkCompaction detects the overflow error, compacts, retries. The experiment subprocess keeps running (detached) and its result gets logged normally.
  • Overflow before experiment starts: same flow — pi compacts, the loop continues.
  • State is always persisted to autoresearch.jsonl, so agent_endagent_start resumes cleanly.

Changes

  • Removed: isContextExhausted(), estimateTokensPerIteration(), hasRoomForNextIteration(), CONTEXT_SAFETY_MARGIN (dead code)
  • Removed: needsCompaction flag from AutoresearchRuntime (never wired up)
  • Removed: the proactive abort block in run_experiment — replaced with a comment
  • Kept: recordIterationTokens() and iterationTokenHistory (useful ASI diagnostics)

Why not PR #41's approach?

PR #41's autoCompactResume config is a reasonable alternative if you want to preserve the predictive check for early warning. But:

  • The prediction fires before the LLM responds, so it can't know if overflow will actually happen
  • A wrong prediction adds latency and complexity for no benefit
  • pi's built-in overflow handling already covers the real case

Testing

  • node --check extensions/pi-autoresearch/index.ts passes
  • Diff: 9 insertions, 32 deletions

When context is nearly full, the extension was proactively calling ctx.abort()
and disabling autoresearchMode, which stopped the loop instead of letting it
continue.

pi already has robust built-in context overflow handling:
- _checkCompaction detects context overflow errors from the LLM
- It automatically compacts the session (summarizes old history)
- Then retries the turn

The proactive token-history check (isContextExhausted) was redundant and
actively harmful — it would fire too early based on predictions, before
the LLM even responded, interrupting running experiments unnecessarily.

Changes:
- Removed isContextExhausted() and related helpers (estimateTokensPerIteration,
  hasRoomForNextIteration, CONTEXT_SAFETY_MARGIN)
- Removed the proactive abort in run_experiment that called ctx.abort()
  when context was predicted to be nearly full
- The token tracking (recordIterationTokens, iterationTokenHistory) stays
  — it logs useful diagnostics in ASI for experiment analysis
- Removed needsCompaction flag (was unnecessary complexity)

Now when context overflows, pi auto-compacts and retries automatically.
When it overflows mid-experiment, the experiment still completes and gets
logged, and the loop auto-resumes in the next turn.
@elecnix elecnix marked this pull request as ready for review April 9, 2026 14:36
dddgogogo added a commit to dddgogogo/pi-autoresearch that referenced this pull request Apr 18, 2026
fix: let pi handle context overflow instead of stopping the loop
@davebcn87
Copy link
Copy Markdown
Owner

I want to rework how we handle context on pi-autoresearch. The idea that I have is that every experiment calls programatically /tree so you could run it indefinetely by moving to the first experiment start checkpoint.

This is not yet possible with current pi API. We'll have some implementation side on the pi codebase so that can work.

That will make compaction unnecessary, so don't want to add more complexity as this problem will go away soon.

@elecnix
Copy link
Copy Markdown
Author

elecnix commented Apr 20, 2026

@davebcn87 I think I understand what you're saying. If every experiment starts at a checkpoint, the context essentially starts from scratch.

In this case, the solution to context overflow would be to throw away the experiment, instead of stopping the loop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants