Problem
When running zeptoclaw agent --template coder -m "Read file X, find bugs, fix them, verify with tests", the LLM:
- Reads the buggy file correctly
- Writes its own test suite that matches the buggy behavior (asserting wrong expected values)
- Runs its own tests — which pass because they test the wrong things
- Reports "NO BUGS FOUND" / "ALL TESTS PASSED"
- Never actually edits the original file
Expected
The agent should:
- Use the existing
test_calculator.py to verify (run it first to see failures)
- Edit the original file in-place using
edit_file
- Re-run the existing tests to confirm the fix
Root cause
The coder template system prompt doesn't instruct the LLM to:
- Check for existing test files before writing new ones
- Use
edit_file to fix bugs in-place rather than write_file to create new files
- Run existing tests as the verification step
Reproduction
cp -R test-coding-pristine test-coding-zepto
./target/release/zeptoclaw agent --template coder -m "Read the file test-coding-zepto/calculator.py, find all the bugs, and fix them. Show me what you changed, and verify the result with tests."
python3 -m unittest test-coding-zepto/test_calculator.py -v # still 6/6 FAIL
Suggested fix
Update the coder template system prompt to include instructions like:
- "Always check for existing test files first and run them to understand the current state"
- "Fix bugs by editing the original file with edit_file, not writing new files"
- "Verify by running the existing test suite, not by writing your own tests"
Problem
When running
zeptoclaw agent --template coder -m "Read file X, find bugs, fix them, verify with tests", the LLM:Expected
The agent should:
test_calculator.pyto verify (run it first to see failures)edit_fileRoot cause
The coder template system prompt doesn't instruct the LLM to:
edit_fileto fix bugs in-place rather thanwrite_fileto create new filesReproduction
Suggested fix
Update the coder template system prompt to include instructions like: