Add gh-aw workflow for automated prior fix regression detection by kubaflo · Pull Request #34768 · dotnet/maui

kubaflo · 2026-04-01T10:27:09Z

Description

Adds a gh-aw workflow (copilot-regression-check) and supporting infrastructure that automatically detects when PRs remove lines that were added by prior bug fixes.

The Problem

Three P/0 regressions (#34634, #34635, #34636) shipped in 10.0.60 because PRs unknowingly reverted lines from prior bug fix PRs. Example: PR #33908 removed || parent is IMauiRecyclerView added by PR #32278 to fix issue #32436.

What It Does

When a PR deletes lines from implementation files, the workflow:

Runs Detect-Regressions.ps1 to extract deleted lines from the diff
Uses git blame to trace each line to its originating commit
Checks the commit message AND the linked PR body for issue references (fixes #XXXX)
Flags removals of prior bug fix lines as 🔴 Critical

Triggers

Trigger	When	Gate
`pull_request`	Any PR (no paths filter for workflow indexing)	Gate step skips if no `.cs/.xaml` impl files changed
`workflow_dispatch`	Manual — enter PR number	None
`issue_comment` (`/check-regressions`)	Comment on PR	None

Security Model

Same as copilot-evaluate-tests (PR #34548): sandboxed agent, scrubbed credentials, safe-outputs (max 1 comment), fork support via Checkout-GhAwPr.ps1, SHA-pinned actions.

Files

.github/workflows/copilot-regression-check.md — gh-aw workflow source
.github/workflows/copilot-regression-check.lock.yml — Compiled workflow
.github/skills/pr-finalize/scripts/Detect-Regressions.ps1 — Detection script
.github/skills/pr-finalize/tests/eval.yaml — 11 eval scenarios
.github/instructions/regression-detection.instructions.md — Global instructions
.github/skills/pr-finalize/SKILL.md — Updated with regression check as Phase 2 Step 1

Verified

$ pwsh Detect-Regressions.ps1 -PRNumber 33908
🔴 Prior Fix Regression Check: FAILED
File: MauiWindowInsetListener.cs
Removed: || parent is IMauiRecyclerView
Origin: commit 0b30658 (PR #32278)
References: #32301, #32436

Supersedes #34656 (moved from PR branch to feature branch for workflow indexing).

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Adds Prior Fix Regression Check that detects when PRs remove lines that were added by prior bug fixes (e.g. PR #33908 removing a guard added by PR #32278 to fix issue #32436). Files: - copilot-regression-check.md/.lock.yml — gh-aw workflow - Detect-Regressions.ps1 — detection script (checks commit msgs + PR bodies) - eval.yaml — 11 eval scenarios - regression-detection.instructions.md — global instructions - SKILL.md — updated pr-finalize skill with regression check Triggers: pull_request (no paths filter, gate step instead), issue_comment (/check-regressions), workflow_dispatch (pr_number) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-04-01T10:27:18Z

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34768

Or

Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34768"

Copilot

Pull request overview

Adds a new gh-aw workflow and supporting skill/script documentation to detect when PRs delete lines that were originally introduced as part of prior bug fixes (to help prevent “reverted-fix” regressions in MAUI).

Changes:

Introduces copilot-regression-check gh-aw workflow (source + compiled lock) to run the prior-fix regression analysis on PRs / slash-command / manual dispatch.
Adds Detect-Regressions.ps1 to identify deleted non-trivial lines, trace their origin via git blame, and flag removals tied to issue-fix commits/PRs.
Updates pr-finalize skill (plus eval scenarios and repo instructions) to make the regression check a mandatory first step in Phase 2.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
.github/workflows/copilot-regression-check.md	New gh-aw workflow source defining triggers, gating, and the agent prompt for regression checking
.github/workflows/copilot-regression-check.lock.yml	Compiled gh-aw workflow (auto-generated) implementing the workflow execution jobs
.github/skills/pr-finalize/scripts/Detect-Regressions.ps1	PowerShell implementation of deleted-line → blame → issue reference detection and report generation
.github/skills/pr-finalize/tests/eval.yaml	Eval scenarios validating regression detection behavior and pr-finalize workflow expectations
.github/skills/pr-finalize/SKILL.md	Adds “Prior Fix Regression Check” as mandatory first step of Phase 2 code review
.github/instructions/regression-detection.instructions.md	New guidance doc explaining the reverted-fix regression pattern and how to detect it
.github/copilot-instructions.md	Adds global Copilot review guidance to run the regression detection first on line removals
.github/aw/actions-lock.json	Adds gh-aw action pin for `github/gh-aw-actions/setup@v0.62.5`

Copilot · 2026-04-01T10:32:04Z

.github/workflows/copilot-regression-check.md

+    env:
+      GH_TOKEN: ${{ github.token }}
+      PR_NUMBER: ${{ inputs.pr_number }}
+    run: pwsh .github/scripts/Checkout-GhAwPr.ps1


The regression check relies on git blame to trace deleted lines to their originating commits, but the gh-aw generated workflow checks out the repo with the default shallow depth (no fetch-depth: 0). In a shallow clone, git blame often returns boundary commits (leading ^) and git log -1 <hash> can fail, causing the script to miss regressions. Consider adding a pre-agent step that unshallows the repo history (e.g., git fetch --unshallow --tags or an explicit deep fetch of the base branch) before the agent runs Detect-Regressions.ps1 so blame is reliable.

Suggested change

run: pwsh .github/scripts/Checkout-GhAwPr.ps1

run: pwsh .github/scripts/Checkout-GhAwPr.ps1

- name: Ensure full git history for regression analysis

run: |

# If there's no git repo yet, nothing to do

if ! git rev-parse --is-inside-work-tree > /dev/null 2>&1; then

echo "No git repository found; skipping unshallow."

exit 0

fi

# Only attempt to unshallow when the repository is shallow

if git rev-parse --is-shallow-repository > /dev/null 2>&1 && [ "$(git rev-parse --is-shallow-repository)" = "true" ]; then

echo "Repository is shallow; fetching full history for reliable git blame…"

# Prefer unshallow, fall back to an explicit deep fetch if needed

git fetch --unshallow --tags || git fetch --depth=2147483647 --tags

else

echo "Repository is already unshallowed; no additional fetch required."

fi

Copilot · 2026-04-01T10:32:05Z

.github/skills/pr-finalize/scripts/Detect-Regressions.ps1

+# Parse the diff to find deleted lines per file
+$CurrentFile = $null
+$FileDeletes = @{}  # file -> list of (lineContent, lineNumber)
+$CurrentLineNumber = 0
+$InHunk = $false
+
+foreach ($line in $DiffContent -split "`n") {
+    if ($line -match '^diff --git a/.+ b/(.+)$') {
+        $CurrentFile = $Matches[1]
+        $InHunk = $false
+        continue
+    }
+
+    if ($line -match '^\+\+\+ b/(.+)$') {
+        $CurrentFile = $Matches[1]
+        continue
+    }
+
+    if ($line -match '^--- ') {
+        continue
+    }
+
+    if ($line -match '^@@ -(\d+)') {
+        $CurrentLineNumber = [int]$Matches[1]
+        $InHunk = $true
+        continue
+    }
+
+    if (-not $InHunk -or -not $CurrentFile) { continue }
+    if (-not (Is-ImplementationFile $CurrentFile)) { continue }
+
+    if ($line.StartsWith('-')) {
+        $lineContent = $line.Substring(1)
+        if (-not (Is-TrivialLine $lineContent)) {
+            if (-not $FileDeletes.ContainsKey($CurrentFile)) {
+                $FileDeletes[$CurrentFile] = @()
+            }
+            $FileDeletes[$CurrentFile] += [PSCustomObject]@{
+                LineContent  = $lineContent
+                LineNumber   = $CurrentLineNumber
+            }
+        }
+        $CurrentLineNumber++
+    }


LineNumber is captured from the diff hunks but never used later; instead the script searches git blame output by matching only the first ~40 characters of the deleted line. This can match the wrong line when common prefixes repeat, leading to false positives/negatives. Since you already track the original line number, consider blaming the exact line range (e.g., git blame <base> -L <n>,<n> -- <file>) to get an unambiguous originating commit.

Copilot · 2026-04-01T10:32:05Z

.github/skills/pr-finalize/scripts/Detect-Regressions.ps1

+                # If the commit message itself has no issue refs but references a PR number,
+                # check the PR body on GitHub — many PRs put "Fixes #XXXX" only in the
+                # PR description, not in the commit message body.
+                if ($issueRefs.Count -eq 0 -and $prRefs.Count -gt 0) {
+                    foreach ($prNum in $prRefs) {
+                        try {
+                            $prBody = gh pr view $prNum --repo $env:GITHUB_REPOSITORY --json body --jq '.body' 2>&1
+                            if ($LASTEXITCODE -eq 0 -and $prBody) {
+                                $prIssueRefs = @(Get-IssueReferences ($prBody | Out-String))
+                                if ($prIssueRefs.Count -gt 0) {
+                                    $issueRefs = $prIssueRefs
+                                    break
+                                }


The script calls gh pr view ... --repo $env:GITHUB_REPOSITORY when trying to infer issue references from a referenced PR body. GITHUB_REPOSITORY is set in Actions, but it’s often unset for local runs (the README example shows running this script locally), which will cause gh to error and silently skip a key signal. Consider defaulting the repo from gh repo view --json nameWithOwner or omitting --repo when already inside a git clone, and only using --repo when the env var is present.

Copilot · 2026-04-01T10:32:05Z

.github/skills/pr-finalize/SKILL.md

+```bash
+# Step 1: Get all deleted lines from non-test implementation files
+gh pr diff XXXXX | grep "^-" | grep -v "^---" | grep -v "^-.*\(test\|Test\|spec\|Spec\)" > /tmp/deleted_lines.txt
+
+# Step 2: For each deleted non-trivial line (not blank, not comment-only, not brace-only),
+# find where it came from using git blame on the base branch
+# The script automates this:
+pwsh .github/skills/pr-finalize/scripts/Detect-Regressions.ps1 -PRNumber XXXXX


The Step 1 command intended to collect deleted lines from non-test implementation files doesn’t actually filter by file path; it filters out deleted lines whose content contains “test/spec”, and it can still include deletions from test files. This makes the documented manual procedure inaccurate and may hide real regressions (e.g., guards containing “Test”). Consider updating the instructions to filter by changed file paths (or just recommend running Detect-Regressions.ps1, which already does path-based filtering).

Copilot · 2026-04-01T10:32:06Z

.github/instructions/regression-detection.instructions.md

+
+```bash
+# Check who added a specific line and why
+git blame main -- src/path/to/file.cs | grep -F "deleted line content"


This guidance uses git blame main -- ..., but in many local clones/CI environments the base branch ref is origin/main (and main might not exist as a local branch). Consider using git blame origin/main -- ... (or explicitly noting “your base branch”) to avoid copy/paste failures.

Suggested change

git blame main -- src/path/to/file.cs | grep -F "deleted line content"

git blame origin/main -- src/path/to/file.cs | grep -F "deleted line content"

# (Use your repository's base branch ref instead of origin/main if different.)

kubaflo added the area-ai-agents Copilot CLI agents, agent skills, AI-assisted development label Apr 1, 2026

Copilot AI review requested due to automatic review settings April 1, 2026 10:27

kubaflo mentioned this pull request Apr 1, 2026

[Skills] Add regression detection to pr-finalize skill and eval scenarios #34656

Closed

Copilot started reviewing on behalf of kubaflo April 1, 2026 10:28 View session

github-actions bot mentioned this pull request Apr 1, 2026

[aw] Prior Fix Regression Check failed #34769

Open

Copilot AI reviewed Apr 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add gh-aw workflow for automated prior fix regression detection#34768

Add gh-aw workflow for automated prior fix regression detection#34768
kubaflo wants to merge 1 commit intomainfrom
feature/copilot-regression-check

kubaflo commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Copilot AI Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-    run: pwsh .github/scripts/Checkout-GhAwPr.ps1
+    run: pwsh .github/scripts/Checkout-GhAwPr.ps1
+  - name: Ensure full git history for regression analysis
+    run: |
+      # If there's no git repo yet, nothing to do
+      if ! git rev-parse --is-inside-work-tree > /dev/null 2>&1; then
+        echo "No git repository found; skipping unshallow."
+        exit 0
+      fi
+      # Only attempt to unshallow when the repository is shallow
+      if git rev-parse --is-shallow-repository > /dev/null 2>&1 && [ "$(git rev-parse --is-shallow-repository)" = "true" ]; then
+        echo "Repository is shallow; fetching full history for reliable git blame…"
+        # Prefer unshallow, fall back to an explicit deep fetch if needed
+        git fetch --unshallow --tags || git fetch --depth=2147483647 --tags
+      else
+        echo "Repository is already unshallowed; no additional fetch required."
+      fi

	git blame main -- src/path/to/file.cs \| grep -F "deleted line content"
	git blame origin/main -- src/path/to/file.cs \| grep -F "deleted line content"
	# (Use your repository's base branch ref instead of origin/main if different.)

Conversation

kubaflo commented Apr 1, 2026

Description

The Problem

What It Does

Triggers

Security Model

Files

Verified

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants