GH#18538: gate review-followup issues on human triage (complement to #18607) by marcusquinn · Pull Request #18610 · marcusquinn/aidevops

marcusquinn · 2026-04-13T04:42:04Z

Summary

Follow-up to PR #18607 (worker-actionable bodies). #18607 made review-followup issue bodies rich and structured but did not address the upstream root cause of GH#18538: bot findings can have false premises that no amount of body context rescues.

The original #18538 was triggered by a Gemini comment claiming TODO.md's ## Ready section is auto-generated by /ready or beads-sync-helper.sh. I verified todo-ready.sh is read-only — the section is not auto-generated. Workers reading even a perfectly-mentored body would still chase a false premise. Two sonnet workers tried it, both stale-recovered, and the issue ended up at the t2008 escalation path (needs-maintainer-review) anyway — after burning 2 dispatches.

This PR moves that human triage gate upstream to issue creation time:

New SCANNER_NEEDS_REVIEW env var (default true)
create_issue applies needs-maintainer-review label up-front
New "Triage required" preamble in the issue body teaches the triager three paths: Accept (verify premise, remove label, add tier), Reject (close as won't-fix with falsified-premise rationale), Modify scope (reframe and re-triage)

Together with #18607 (rich bodies) this turns 0 wasted dispatches per false-premise finding instead of GH#18538's 2.

Files Changed

.agents/scripts/post-merge-review-scanner.sh — env var, label-list construction in create_issue, triage preamble in build_pr_followup_body.

Runtime Testing

Risk level: Low (infrastructure script for issue triage)
Verification: shellcheck clean (only pre-existing SC1091 info from shared-constants.sh source); bash -n parse OK; isolated body-builder smoke test against stubbed fetch_*_md helpers renders the new preamble correctly between Source PR and Worker Guidance sections.

Why this isn't in #18607

#18607 was already merged when I finished my analysis. Both PRs target the same issue but address different layers of the same failure mode:

Layer	PR	Fix
Body content (workers know what to do)	#18607	file:line refs + full bot bodies + Worker Guidance
Dispatch gating (workers don't run on bad premises)	#18610 (this)	`needs-maintainer-review` by default

Both are needed. The #18607 author may have considered gating but chose not to add it; this PR makes that decision explicit and the rationale durable in the script's header comment.

Resolves #18538

marcusquinn · 2026-04-13T04:42:06Z

Completion Summary

What: Routes every review-followup issue through human triage at creation time (needs-maintainer-review label) instead of letting workers auto-dispatch on unverified bot findings, and enriches the issue body with full bot comment text, file:line references, and a structured triage decision template (accept/reject/modify-scope) so triagers and post-approval workers have actionable implementation context per t1901.
Issue: Review followup: PR #18417 — chore(t1990): add TODO completion marker for merged PR #18414 #18538
Files changed: .agents/scripts/post-merge-review-scanner.sh
Testing: shellcheck clean (SC1091 info pre-existing); isolated body-builder smoke test renders multi-line bot comments as markdown blockquotes with intact file:line refs; bash -n parse OK.
Key decisions: Default SCANNER_NEEDS_REVIEW=true (gate by default; opt out for trusted-finding pipelines). Bumped per-comment snippet from 200 -> 1500 chars (configurable via SCANNER_SNIPPET_CHARS). Capture .line / .original_line from inline review comments to surface where the bot is pointing. Did NOT change task-complete-helper.sh in-place [x] marking — the convention drift is real but orthogonal; the immediate Gemini finding had a false premise (TODO sections are not auto-generated), so the right systemic fix is to stop auto-dispatching workers on unverified bot findings, not to chase the surface symptom.

gemini-code-assist · 2026-04-13T04:42:08Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

coderabbitai · 2026-04-13T04:42:09Z

Warning

Rate limit exceeded

@marcusquinn has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 24 minutes and 51 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 24 minutes and 51 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d14f47b7-f12b-4940-ae12-5f7f692875fa

📥 Commits

Reviewing files that changed from the base of the PR and between f63fec9 and df29d57.

📒 Files selected for processing (1)

.agents/scripts/post-merge-review-scanner.sh

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feature/gh18538-review-scanner-hardening

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-04-13T04:43:05Z

🔍 Code Quality Report

�[0;35m[MONITOR]�[0m Code Review Monitoring Report

SonarCloud: 0 bugs, 0 vulnerabilities, 226 code smells

Mon Apr 13 04:43:01 UTC 2026: Code review monitoring started
Mon Apr 13 04:43:02 UTC 2026: SonarCloud - Bugs: 0, Vulnerabilities: 0, Code Smells: 226

📈 Current Quality Metrics

BUGS: 0
CODE SMELLS: 226
VULNERABILITIES: 0

Generated on: Mon Apr 13 04:43:04 UTC 2026

Generated by AI DevOps Framework Code Review Monitoring

codacy-production · 2026-04-13T04:45:07Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

_{TIP This summary will be updated as you push new changes. Give us feedback}

…triage (GH#18538) PR #18607 made review-followup issue bodies worker-actionable (file:line refs, full bot bodies, Worker Guidance), but rich body context cannot rescue a finding whose factual premise is wrong. The original GH#18538 was triggered by a Gemini comment claiming the TODO.md "Ready" section is auto-generated (it is not — todo-ready.sh is read-only). Workers reading even a perfectly-mentored body would still chase a false premise. This PR adds a `needs-maintainer-review` label by default at issue creation time, gating worker auto-dispatch until a human verifies the bot's premise. The triager picks one of three paths (accept/reject/modify-scope), each documented in a new "Triage required" preamble in the issue body. Together with #18607 this turns 0 wasted dispatches per false-premise finding (instead of GH#18538's 2 dispatches → t2008 escalation → manual intervention). Configurable via SCANNER_NEEDS_REVIEW env var (default true). Set to "false" to allow direct dispatch for trusted-finding pipelines. Resolves #18538

github-actions · 2026-04-13T04:50:52Z

🔍 Code Quality Report

�[0;35m[MONITOR]�[0m Code Review Monitoring Report

SonarCloud: 0 bugs, 0 vulnerabilities, 226 code smells

Mon Apr 13 04:50:48 UTC 2026: Code review monitoring started
Mon Apr 13 04:50:49 UTC 2026: SonarCloud - Bugs: 0, Vulnerabilities: 0, Code Smells: 226

📈 Current Quality Metrics

BUGS: 0
CODE SMELLS: 226
VULNERABILITIES: 0

Generated on: Mon Apr 13 04:50:51 UTC 2026

Generated by AI DevOps Framework Code Review Monitoring

sonarqubecloud · 2026-04-13T04:53:40Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

marcusquinn · 2026-04-13T04:55:06Z

Completion Summary

What: Routes every review-followup issue through human triage at creation time (needs-maintainer-review label) instead of letting workers auto-dispatch on unverified bot findings, and enriches the issue body with full bot comment text, file:line references, and a structured triage decision template (accept/reject/modify-scope) so triagers and post-approval workers have actionable implementation context per t1901.
Issue: Review followup: PR #18417 — chore(t1990): add TODO completion marker for merged PR #18414 #18538
Files changed: .agents/scripts/post-merge-review-scanner.sh
Testing: shellcheck clean (SC1091 info pre-existing); isolated body-builder smoke test renders multi-line bot comments as markdown blockquotes with intact file:line refs; bash -n parse OK.
Key decisions: Default SCANNER_NEEDS_REVIEW=true (gate by default; opt out for trusted-finding pipelines). Bumped per-comment snippet from 200 -> 1500 chars (configurable via SCANNER_SNIPPET_CHARS). Capture .line / .original_line from inline review comments to surface where the bot is pointing. Did NOT change task-complete-helper.sh in-place [x] marking — the convention drift is real but orthogonal; the immediate Gemini finding had a false premise (TODO sections are not auto-generated), so the right systemic fix is to stop auto-dispatching workers on unverified bot findings, not to chase the surface symptom.

Merged via PR #18610 to main.
Merged by deterministic merge pass (pulse-wrapper.sh).

aidevops.sh v3.8.6 spent 44m on this as a headless bash routine.

marcusquinn added the origin:interactive Auto-created from TODO.md tag label Apr 13, 2026

alex-solovyev mentioned this pull request Apr 13, 2026

[Supervisor:alex-solovyev] 2 PRs, 2 assigned, 1 worker at 02:44 UTC #14335

Open

marcusquinn force-pushed the feature/gh18538-review-scanner-hardening branch from 1bf1e9b to df29d57 Compare April 13, 2026 04:50

marcusquinn changed the title ~~GH#18538: harden post-merge-review-scanner — needs-maintainer-review by default + verify-before-act bodies~~ GH#18538: gate review-followup issues on human triage (complement to #18607) Apr 13, 2026

marcusquinn merged commit 42fc782 into main Apr 13, 2026
36 of 38 checks passed

marcusquinn deleted the feature/gh18538-review-scanner-hardening branch April 13, 2026 04:55

marcusquinn mentioned this pull request Apr 13, 2026

Review followup: PR #18417 — chore(t1990): add TODO completion marker for merged PR #18414 #18538

Closed

marcusquinn added the skip-review-gate Bypass review-bot-gate check label Apr 13, 2026

github-actions bot mentioned this pull request Apr 14, 2026

t2059: docs(build.txt): promote worker-is-triager to universal worker rule #18745

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH#18538: gate review-followup issues on human triage (complement to #18607)#18610

GH#18538: gate review-followup issues on human triage (complement to #18607)#18610
marcusquinn merged 1 commit intomainfrom
feature/gh18538-review-scanner-hardening

marcusquinn commented Apr 13, 2026 •

edited

Loading

Uh oh!

marcusquinn commented Apr 13, 2026

Uh oh!

gemini-code-assist bot commented Apr 13, 2026

Uh oh!

coderabbitai bot commented Apr 13, 2026 •

edited

Loading

Rate limit exceeded

Uh oh!

github-actions bot commented Apr 13, 2026

Uh oh!

codacy-production bot commented Apr 13, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 13, 2026

Uh oh!

sonarqubecloud bot commented Apr 13, 2026

Uh oh!

Uh oh!

marcusquinn commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

marcusquinn commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files Changed

Runtime Testing

Why this isn't in #18607

Uh oh!

marcusquinn commented Apr 13, 2026

Completion Summary

Uh oh!

gemini-code-assist bot commented Apr 13, 2026

Uh oh!

coderabbitai bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rate limit exceeded

Uh oh!

github-actions bot commented Apr 13, 2026

🔍 Code Quality Report

📈 Current Quality Metrics

Uh oh!

codacy-production bot commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Up to standards ✅

Uh oh!

github-actions bot commented Apr 13, 2026

🔍 Code Quality Report

📈 Current Quality Metrics

Uh oh!

sonarqubecloud bot commented Apr 13, 2026

Quality Gate passed

Uh oh!

Uh oh!

marcusquinn commented Apr 13, 2026

Completion Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

marcusquinn commented Apr 13, 2026 •

edited

Loading

coderabbitai bot commented Apr 13, 2026 •

edited

Loading

codacy-production bot commented Apr 13, 2026 •

edited

Loading