Skip to content

fix(tools): enforce read-before-write guard for write_file#2554

Open
nsalvacao wants to merge 4 commits intoQwenLM:mainfrom
nsalvacao:fix/write-file-read-before-write-2499
Open

fix(tools): enforce read-before-write guard for write_file#2554
nsalvacao wants to merge 4 commits intoQwenLM:mainfrom
nsalvacao:fix/write-file-read-before-write-2499

Conversation

@nsalvacao
Copy link
Copy Markdown
Contributor

TLDR

Prevents write_file from overwriting existing files with hallucinated/truncated content by adding a 3-layer defense: system prompt instruction, tool description warning, and a runtime content-loss guard that blocks writes when proposed content is significantly shorter than the original.

Screenshots / Video Demo

Before (the problem): The agent calls write_file on an existing 200-line file without reading it first, replacing it with ~20 lines of hallucinated content. Data loss.

After (the fix): When the proposed content is <50% of the original file length, the write is blocked with an educational error:

Warning: Proposed content (17 chars) is significantly shorter than the existing file
(200 chars), which may indicate data loss. Please use read_file to read the current
file content before overwriting. If you intend to replace the file with shorter
content, read the file first to confirm your intent.

The original file is preserved. The LLM is guided to read first, then retry.

Dive Deeper

Approach & Rationale

Problem: write_file is stateless — no instruction or mechanism tells the LLM to read before writing. The edit tool avoids this by design (its old_string param forces knowledge of current content), but write_file has no equivalent.

Alternatives considered:

  1. Session-level read tracking — Track which files the LLM has read and refuse writes to unread files. Rejected: requires architectural changes to the tool system (shared state across tool invocations) that are beyond the scope of a bugfix.
  2. Pre-hook validation — Use the hook system to intercept write_file calls. Rejected: hooks are user-configurable, not a reliable default safeguard.
  3. Content-loss heuristic (chosen) — Compare proposed content length to original. Simple, stateless, catches the most damaging case. Combined with prompt-level instructions for broader coverage.

Decision: Defense-in-depth with 3 complementary layers, each independently useful:

Layer File What
System prompt prompts.ts "Read Before Write" mandate in Core Mandates
Tool description write-file.ts Warning in schema, following edit tool pattern
Runtime guard write-file.ts Block writes when proposed.length / original.length < 0.5 (files ≥100 chars)

Guard bypass conditions: new files, small files (<100 chars), empty files, user-modified content.

Reviewer Test Plan

  1. Run unit tests: npx vitest run packages/core/src/tools/write-file.test.ts (6 new tests for the guard)
  2. Run prompt snapshot tests: npx vitest run packages/core/src/core/prompts.test.ts
  3. Manual test:
    • Create a file with >100 chars of content
    • Ask the agent to write to that file with significantly shorter content (without reading first)
    • Expected: write blocked with content-loss warning
    • Then ask the agent to read the file first, then write
    • Expected: write proceeds normally
Edge case Guard behavior
New file creation Skipped (file doesn't exist)
Empty existing file Skipped (length < 100)
Small file (<100 chars) Skipped (below threshold)
Content >50% of original Skipped (ratio check passes)
User-modified content Skipped (explicit bypass)

Testing Matrix

🍏 🪟 🐧
npm run
npx
Docker
Podman - -
Seatbelt - -

Linked issues / bugs

Fixes #2499

Part of QwenLM#2499 — new error type for the content-loss guard
in write_file tool.

Generated by Nuno Salvação <nuno.salvacao@gmail.com> & Co-Authored with Nexo <nexo.modeling@gmail.com>
Instructs the LLM to always read existing files before overwriting
them with write_file, and to prefer edit for targeted changes.

Part of QwenLM#2499

Generated by Nuno Salvação <nuno.salvacao@gmail.com> & Co-Authored with Nexo <nexo.modeling@gmail.com>
Updates tool schema description to instruct the LLM to read existing
files before overwriting them, following the pattern established by
the edit tool (edit.ts:26).

Part of QwenLM#2499

Generated by Nuno Salvação <nuno.salvacao@gmail.com> & Co-Authored with Nexo <nexo.modeling@gmail.com>
When write_file is called on an existing file (>=100 chars) and the
proposed content is less than 50% of the original length, the write is
blocked with an educational error instructing the LLM to read the file
first. The guard is bypassed when content was modified by the user.

Fixes QwenLM#2499

Generated by Nuno Salvação <nuno.salvacao@gmail.com> & Co-Authored with Nexo <nexo.modeling@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent overwrites files using write_file without reading existing content, leading to data loss.

1 participant