Skip to content

Stabilize approval matrix write-file command#14968

Merged
aibrahim-oai merged 1 commit intomainfrom
codex/approval-matrix-write-file-stability
Mar 17, 2026
Merged

Stabilize approval matrix write-file command#14968
aibrahim-oai merged 1 commit intomainfrom
codex/approval-matrix-write-file-stability

Conversation

@aibrahim-oai
Copy link
Copy Markdown
Collaborator

What is flaky

The approval-matrix WriteFile scenario is flaky. It sometimes fails in CI even though the approval logic is unchanged, because the test delegates the file write and readback to shell parsing instead of deterministic file I/O.

Why it was flaky

The test generated a command shaped like printf ... > file && cat file. That means the scenario depended on shell quoting, redirection, newline handling, and encoding behavior in addition to the approval system it was actually trying to validate. If the shell interpreted the payload differently, the test would report an approval failure even though the product logic was fine.

That also made failures hard to diagnose, because the test did not log the exact generated command or the parsed result payload.

How this PR fixes it

This PR replaces the shell-redirection path with a deterministic python3 -c script that writes the file with Path.write_text(..., encoding='utf-8') and then reads it back with the same UTF-8 path. It also logs the generated command and the resulting exit code/stdout for the approval scenario so any future failure is directly attributable.

Why this fix fixes the flakiness

The scenario no longer depends on shell parsing and redirection semantics. The file contents are produced and read through explicit UTF-8 file I/O, so the approval test is measuring approval behavior instead of shell behavior. The added diagnostics mean a future failure will show the exact command/result pair instead of looking like a generic intermittent mismatch.

Co-authored-by: Codex <noreply@openai.com>
@aibrahim-oai aibrahim-oai requested a review from pakrym-oai March 17, 2026 18:25
@aibrahim-oai aibrahim-oai merged commit 4d9d4b7 into main Mar 17, 2026
54 of 56 checks passed
@aibrahim-oai aibrahim-oai deleted the codex/approval-matrix-write-file-stability branch March 17, 2026 20:52
@github-actions github-actions bot locked and limited conversation to collaborators Mar 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant