SDD strict-TDD flow loses apply-progress evidence across apply passes and verify rejects reconstructed evidence

[danxorzum]

Pre-flight Checklist

• I have searched existing issues and this is not a duplicate
• I understand that PRs will be rejected if the linked issue does not have status:approved

📝 Bug Description

In strict-TDD SDD workflows, sdd-apply can report successful progress while a later sdd-verify still blocks the change because the apply-progress artifact is missing, incomplete, overwritten by a later pass, or reconstructed post-hoc.
This creates a mismatch between implementation reality and process validation:

• the code changes may already be correct
• tests, typecheck, lint, and coverage may pass
• but verify still fails because strict-TDD evidence was not durably preserved in a cumulative way
  The issue appears when a change requires multiple apply passes. Earlier TDD evidence can be lost or replaced, while verify expects a single auditable cumulative artifact for the whole change.

🔄 Steps to Reproduce

1. Start a strict-TDD SDD change in a project where verify requires task-level TDD evidence.
2. Run an initial sdd-apply and complete part of the implementation.
3. Persist sdd/{change-name}/apply-progress.
4. Run sdd-verify and receive blockers that require a corrective apply pass.
5. Run a second sdd-apply that updates the same apply-progress topic.
6. Run sdd-verify again.

✅ Expected Behavior

• apply-progress should behave as a cumulative artifact for the entire change.
• Each apply pass should:
  1. read the current artifact,
  2. merge new evidence into the previous evidence,
  3. preserve earlier task rows,
  4. save the full accumulated artifact.
• sdd-verify should be able to audit the final evidence without blocking on missing or reconstructed records when the work was actually preserved correctly during execution.

❌ Actual Behavior

• a later apply pass can effectively overwrite or replace earlier strict-TDD evidence
• verify then reports that:
  • apply-progress is missing,
  • or the TDD evidence table is incomplete,
  • or the artifact was reconstructed post-hoc and therefore does not count as original evidence
• this causes repeated apply/verify loops even when the implementation and runtime verification are already green

Gentle AI Version

v2.8.1

Operating System

macOS

AI Agent / Client

OpenCode

📋 Affected Area

Other

💡 Logs / Error Output

Additional Context

Impact
This causes:

• false-positive apply completions
• repeated correction loops
• strict-TDD deadlocks
• loss of trust in the SDD flow
• auditability gaps for RED / GREEN / TRIANGULATE / REFACTOR evidence
  Likely root cause
  There appears to be a process mismatch between:
• apply behavior that treats apply-progress like a per-pass or per-batch snapshot
• verification behavior that expects a durable cumulative artifact
  Proposal to fix
  Use read-merge-write semantics for sdd/{change-name}/apply-progress:
• before saving, always read the existing artifact
• merge the new pass into the previous evidence
• never overwrite the artifact with only the latest batch
  Also consider requiring explicit per-task evidence fields such as:
• task ID
• test files
• production files
• RED / GREEN / TRIANGULATE / REFACTOR notes
• command run
• result / exit code

[Alan-TheGentleman]

Fixed in v1.18.3 — release.

Root cause confirmed: Two compounding issues:

1. topic_key: sdd/{change}/apply-progress uses upsert semantics — same key = UPDATE, not INSERT. Each pass overwrote prior progress with no history.
2. Sub-agents started with fresh context and never read prior apply-progress before saving their own.

Fix:

1. Orchestrator searches for existing apply-progress before launching continuation batches and tells the sub-agent: "PREVIOUS APPLY-PROGRESS EXISTS. You MUST read it first via mem_search + mem_get_observation, merge your new progress with the existing progress, and save the combined result. Do NOT overwrite — MERGE."
2. sdd-apply has a new Step 2b that reads previous apply-progress and skips already-completed tasks
3. Step 6 (save) now requires merging: the final artifact must include ALL previously completed tasks PLUS new completions in a single combined artifact

To test:

brew upgrade gentle-ai
gentle-ai sync

Then run sdd-apply across multiple batches and verify that the apply-progress artifact accumulates state instead of overwriting it.