diff --git a/.github/README-AI.md b/.github/README-AI.md index 843637a2988c..9eeb1f0532ca 100644 --- a/.github/README-AI.md +++ b/.github/README-AI.md @@ -4,8 +4,8 @@ This folder contains instructions and configurations for AI coding assistants wo ## Available Agents -### PR Agent -The PR agent is a unified 4-phase workflow for investigating issues and reviewing/working on PRs. It handles everything from context gathering through test verification, fix exploration, and creating PRs or review reports. +### PR Review Agent +The pr-review skill is a 4-phase orchestrator for investigating issues and reviewing/working on PRs. It invokes dedicated phase skills (pr-preflight, pr-gate, try-fix, pr-report) for context gathering, test verification, fix exploration, and review reports. ### Sandbox Agent The sandbox agent is your general-purpose tool for working with the .NET MAUI Sandbox app. Use it for manual testing, PR validation, issue reproduction, and experimentation with MAUI features. @@ -84,13 +84,13 @@ copilot please write UI tests for issue #12345 ``` -**PR Agent:** +**Try-Fix-Validate Agent:** ```bash # Start GitHub Copilot CLI with agent support copilot -# Invoke the pr agent -/agent pr +# Invoke the pr-review skill +/skill pr-review # Fix an issue or review a PR please fix issue #12345 @@ -112,7 +112,7 @@ please review https://github.com/dotnet/maui/pull/XXXXX 3. **Choose your agent** from the dropdown: - `sandbox-agent` for manual testing and experimentation - `write-tests-agent` for writing tests (invokes appropriate skill) - - `pr` for reviewing and working on existing PRs + - `pr-review skill` for reviewing and working on existing PRs 4. **Enter a task** in the text box: - For sandbox testing: `Please test PR #32479` @@ -146,9 +146,9 @@ Automated testing specialist for the .NET MAUI test suite: 4. **Cross-Platform** - Tests on iOS, Android, Windows, and MacCatalyst 5. **Automated Workflow** - Uses `BuildAndRunHostApp.ps1` to handle building, deployment, and logging to `CustomAgentLogsTmp/UITests/` -### PR Agent +### PR Review Agent -Unified 4-phase workflow for issue investigation and PR work: +Unified 4-phase orchestrator for issue investigation and PR work: 1. **Pre-Flight** - Context gathering from issues/PRs 2. **Gate** - Verify tests exist and catch the issue (mandatory checkpoint) @@ -199,9 +199,11 @@ Agents work with **time budgets as estimates for planning**, not hard deadlines: ## File Structure -### Agent Definitions -- **`agents/pr.md`** - PR workflow phases 1-2 (Pre-Flight, Gate) -- **`agents/pr/post-gate.md`** - PR workflow phases 3-4 (Fix, Report) +### Agent & Skill Definitions +- **`skills/pr-review/SKILL.md`** - PR Review orchestrator (invokes phase docs and try-fix skill) +- **`pr-review/pr-preflight.md`** - Phase 1: Context gathering (phase doc, not a standalone skill) +- **`pr-review/pr-gate.md`** - Phase 2: Test verification (phase doc, not a standalone skill) +- **`pr-review/pr-report.md`** - Phase 4: Final recommendation (phase doc, not a standalone skill) - **`agents/sandbox-agent.md`** - Sandbox agent for testing and experimentation - **`agents/write-tests-agent.md`** - Test writing agent (dispatches to skills like write-ui-tests) @@ -209,8 +211,6 @@ Agents work with **time budgets as estimates for planning**, not hard deadlines: Agent files in the `.github/agents/` directory: -- **`agents/pr.md`** - PR workflow phases 1-2 (Pre-Flight, Gate) -- **`agents/pr/post-gate.md`** - PR workflow phases 3-4 (Fix, Report) - **`agents/sandbox-agent.md`** - Sandbox app testing and experimentation - **`agents/write-tests-agent.md`** - Test writing (invokes skills like write-ui-tests) @@ -250,12 +250,12 @@ Reusable skills in `.github/skills/` that agents can invoke: - **`verify-tests-fail-without-fix/`** - Verifies UI tests catch bugs (auto-detects mode based on git diff) - **`write-ui-tests/`** - Creates UI tests for issues following MAUI conventions - **`write-xaml-tests/`** - Creates XAML unit tests for parsing, XamlC, and source generation issues -- **`azdo-build-investigator/`** - MAUI-specific CI investigation context (works with `ci-analysis` from arcade-skills plugin) +- **`azdo-build-investigator/`** - Investigates CI failures for PRs (build errors, Helix test logs, binlog analysis) via dotnet/arcade-skills plugin ### Recent Improvements (January 2026) -**PR Agent Consolidation:** -1. **Unified PR Agent** - Replaced separate `issue-resolver` and `pr-reviewer` agents with single 4-phase `pr` agent +**Agent Consolidation:** +1. **Unified PR Review Orchestrator** - Replaced separate `issue-resolver` and `pr-reviewer` agents with 4-phase `pr-review` skill that orchestrates `pr-preflight`, `pr-gate`, `try-fix`, and `pr-report` phase skills 2. **try-fix Skill** - New skill for exploring independent fix alternatives with empirical testing 3. **Skills Integration** - Added `verify-tests-fail-without-fix` and `write-ui-tests` skills for reusable test workflows 4. **Agent/Skills Guidelines** - New instruction files for authoring agents and skills @@ -364,8 +364,8 @@ For issues or questions about the AI agent instructions: ## Metrics **Agent Files**: -- 4 agent files (pr.md, pr/post-gate.md, sandbox-agent.md, write-tests-agent.md) -- 5 skills (try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator) +- 4 agent files (pr-review skill.md, sandbox-agent.md, write-tests-agent.md) +- 5 skills (pr-review, try-fix, verify-tests-fail-without-fix, write-ui-tests, write-xaml-tests, azdo-build-investigator) + 3 phase docs (pr-preflight, pr-gate, pr-report) - All validated and consistent with consolidated structure **Automation**: @@ -383,4 +383,4 @@ For issues or questions about the AI agent instructions: **Last Updated**: 2026-01-07 -**Note**: These instructions are actively being refined based on real-world usage. PR agent consolidation completed January 2026 (unified 4-phase workflow with try-fix skill). Feedback and improvements are welcome! +**Note**: These instructions are actively being refined based on real-world usage. Agent consolidation completed January 2026 (unified 4-phase workflow with try-fix skill). Feedback and improvements are welcome! diff --git a/.github/agents/pr.md b/.github/agents/pr.md deleted file mode 100644 index e4fdc4163c73..000000000000 --- a/.github/agents/pr.md +++ /dev/null @@ -1,310 +0,0 @@ ---- -name: pr -description: Sequential 4-phase workflow for GitHub issues - Pre-Flight, Gate, Fix, Report. Phases MUST complete in order. ---- - -# .NET MAUI Pull Request Agent - -You are an end-to-end agent that takes a GitHub issue from investigation through to a completed PR. - -## When to Use This Agent - -- ✅ "Fix issue #XXXXX" - Works whether or not a PR exists -- ✅ "Work on issue #XXXXX" -- ✅ "Implement fix for #XXXXX" -- ✅ "Review PR #XXXXX" -- ✅ "Continue working on #XXXXX" -- ✅ "Pick up where I left off on #XXXXX" - -## When NOT to Use This Agent - -- ❌ Just run tests manually → Use `sandbox-agent` -- ❌ Only write tests without fixing → Use `write-tests-agent` - ---- - -## Workflow Overview - -This file covers **Phases 1-2** (Pre-Flight → Gate). - -After Gate passes, read `.github/agents/pr/post-gate.md` for **Phases 3-4**. - -``` -┌─────────────────────────────────────────┐ ┌─────────────────────────────────────────────┐ -│ THIS FILE: pr.md │ │ pr/post-gate.md │ -│ │ │ │ -│ 1. Pre-Flight → 2. Gate │ ──► │ 3. Fix → 4. Report │ -│ ⛔ │ │ │ -│ MUST PASS │ │ (Only read after Gate ✅ PASSED) │ -└─────────────────────────────────────────┘ └─────────────────────────────────────────────┘ -``` - ---- - -## 🚨 Critical Rules - -**Read `.github/agents/pr/SHARED-RULES.md` for complete details on:** -- Phase Completion Protocol (fill ALL pending fields before marking complete) -- Follow Templates EXACTLY (no `open` attributes, no "improvements") -- No Direct Git Commands (use `gh pr diff/view`, let scripts handle files) -- Use Skills' Scripts (don't bypass with manual commands) -- Stop on Environment Blockers (retry once, then skip and continue autonomously) -- Multi-Model Configuration (5 models for Phase 4) -- Platform Selection (must be affected AND available on host) - -**Key points:** -- ❌ Never run `git checkout`, `git switch`, `git stash`, `git reset` - agent is always on correct branch -- ❌ Never stop and ask user - use best judgment to skip blocked phases and continue -- ❌ Never mark phase ✅ with [PENDING] fields remaining - -Phase 3 uses a 5-model exploration workflow. See `post-gate.md` for detailed instructions after Gate passes. - ---- - -## PRE-FLIGHT: Context Gathering (Phase 1) - -> **⚠️ SCOPE**: Document only. No code analysis. No fix opinions. No running tests. - -### ❌ Pre-Flight Boundaries (What NOT To Do) - -| ❌ Do NOT | Why | When to do it | -|-----------|-----|---------------| -| Research git history | That's root cause analysis | Phase 3: 🔧 Fix | -| Look at implementation code | That's understanding the bug | Phase 3: 🔧 Fix | -| Design or implement fixes | That's solution design | Phase 3: 🔧 Fix | -| Form opinions on correct approach | That's analysis | Phase 3: 🔧 Fix | -| Run tests | That's verification | Phase 2: 🚦 Gate | - -### ✅ What TO Do in Pre-Flight - -- Read issue description and comments -- Note platforms affected (from labels) -- Identify files changed (if PR exists) -- Document disagreements and edge cases from comments - -### Step 1: Gather Context (depends on starting point) - -**If starting from a PR:** -```bash -# Fetch PR metadata (agent is already on correct branch) -gh pr view XXXXX --json title,body,url,author,labels,files - -# Find and read linked issue -gh pr view XXXXX --json body --jq '.body' | grep -oE "(Fixes|Closes|Resolves) #[0-9]+" | head -1 -gh issue view ISSUE_NUMBER --json title,body,comments -``` - -**If starting from an Issue (no PR exists):** -```bash -# Fetch issue details directly -gh issue view XXXXX --json title,body,comments,labels -``` - -### Step 2: Fetch Comments - -**If PR exists** - Fetch PR discussion: -```bash -# PR-level comments -gh pr view XXXXX --json comments --jq '.comments[] | "Author: \(.author.login)\n\(.body)\n---"' - -# Review summaries -gh pr view XXXXX --json reviews --jq '.reviews[] | "Reviewer: \(.author.login) [\(.state)]\n\(.body)\n---"' - -# Inline code review comments (CRITICAL - often contains key technical feedback!) -gh api "repos/dotnet/maui/pulls/XXXXX/comments" --jq '.[] | "File: \(.path):\(.line // .original_line)\nAuthor: \(.user.login)\n\(.body)\n---"' - -# Detect Prior Agent Reviews -gh pr view XXXXX --json comments --jq '.comments[] | select(.body | contains("Final Recommendation") and contains("| Phase | Status |")) | .body' -``` - -**If issue only** - Comments already fetched in Step 1. - -**Signs of a prior agent review in comments:** -- Contains phase status table (`| Phase | Status |`) -- Contains `✅ Final Recommendation: APPROVE` or `⚠️ Final Recommendation: REQUEST CHANGES` -- Contains collapsible `
` sections with phase content -- Contains structured analysis (Root Cause, Platform Comparison, etc.) - -**If prior agent review found:** -1. Parse the phase statuses to determine what's already done -2. Import all findings (fix candidates, test results) -3. Resume from whichever phase is not yet complete (or report as done) - -**Do NOT:** -- Start from scratch if a complete review already exists -- Treat the prior review as just "reference material" -- Re-do phases that are already marked `✅ PASSED` - -### Step 3: Document Key Findings - -**If PR exists** - Document disagreements and reviewer feedback: -| File:Line | Reviewer Says | Author Says | Status | -|-----------|---------------|-------------|--------| -| Example.cs:95 | "Remove this call" | "Required for fix" | ⚠️ INVESTIGATE | - -**Edge Cases to Check** (from comments mentioning "what about...", "does this work with..."): -- [ ] Edge case 1 from discussion -- [ ] Edge case 2 from discussion - -### Step 4: Classify Files (if PR exists) - -```bash -gh pr view XXXXX --json files --jq '.files[].path' -``` - -Classify into: -- **Fix files**: Source code (`src/Controls/src/...`, `src/Core/src/...`) -- **Test files**: Tests (`DeviceTests/`, `TestCases.HostApp/`, `UnitTests/`) - -Identify test type: **UI Tests** | **Device Tests** | **Unit Tests** - -**Record PR's fix as reference** (at the bottom of the Fix Candidates table): - -```markdown -| # | Source | Approach | Test Result | Files Changed | Notes | -|---|--------|----------|-------------|---------------|-------| -| PR | PR #XXXXX | [Describe PR's approach] | ⏳ PENDING (Gate) | `file.cs` (+N) | Original PR | -``` - -**Note:** The PR's fix is validated by Gate (Phase 3), NOT by try-fix. try-fix candidates are numbered 1, 2, 3... and are YOUR independent ideas. - -The test result will be updated to `✅ PASS (Gate)` after Gate passes. - -### Step 5: Complete Pre-Flight - -Verify the following before proceeding: -- [ ] Issue summary captured -- [ ] Platform information noted -- [ ] Files changed identified (if PR exists) -- [ ] PR discussion summarized (if PR exists) -- [ ] **Write phase output to `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/pre-flight/content.md`** (see SHARED-RULES.md "Phase Output Artifacts") - ---- - -## 🚦 GATE: Verify Tests Catch the Issue (Phase 2) - -> **SCOPE**: Verify tests exist and correctly detect the fix (for PRs) or reproduce the bug (for issues). - -**⛔ This phase MUST pass before continuing. If it fails, stop and fix the tests.** - -**⚠️ Gate Check:** Pre-Flight must be `✅ COMPLETE` before starting this phase. - -### Step 1: Check if Tests Exist - -**If PR exists:** -```bash -gh pr view XXXXX --json files --jq '.files[].path' | grep -E "TestCases\.(HostApp|Shared\.Tests)" -``` - -**If issue only:** -```bash -# Check if tests exist for this issue number -find src/Controls/tests -name "*XXXXX*" -type f 2>/dev/null -``` - -**If tests exist** → Proceed to verification. - -**If NO tests exist** → Let the user know that tests are missing. They can use the `write-tests-agent` to help create them. - -### Step 2: Select Platform - -**🚨 CRITICAL: Choose a platform that is BOTH affected by the bug AND available on the current host.** - -**Identify affected platforms** from Pre-Flight: -- Check the platforms affected from Pre-Flight context -- Check issue labels (e.g., `platform/iOS`, `platform/Android`) -- Check which platform-specific files the PR modifies - -**Match to available platforms on current host:** - -| Host OS | Available Platforms | -|---------|---------------------| -| Windows | Android, Windows | -| macOS | Android, iOS, MacCatalyst | - -**Select the best match:** -1. Pick a platform that IS affected by the bug -2. That IS available on the current host -3. Prefer the platform most directly impacted by the PR's code changes - -**Example decisions:** -- Bug affects iOS/Windows/MacCatalyst, host is Windows → Test on **Windows** -- Bug affects iOS only, host is Windows → **STOP** - cannot test (ask user) -- Bug affects Android only → Test on **Android** (works on any host) -- Bug affects all platforms → Pick based on host (Windows on Windows, iOS on macOS) - -**⚠️ Do NOT test on a platform that isn't affected by the bug** - the test will pass regardless of whether the fix works. - -### Step 3: Run Verification - -**🚨 MUST invoke as a task agent** to prevent command substitution: - -```markdown -Invoke the `task` agent with agent_type: "task" and this prompt: - -"Invoke the verify-tests-fail-without-fix skill for this PR: -- Platform: [selected platform from Platform Selection above] -- TestFilter: 'IssueXXXXX' -- RequireFullVerification: true - -Report back: Did tests FAIL without fix? Did tests PASS with fix? Final status?" -``` - -**Why task agent?** Running inline allows substituting commands and fabricating results. Task agent runs in isolation and reports exactly what happened. - -See `.github/skills/verify-tests-fail-without-fix/SKILL.md` for full skill documentation. - -### Expected Output (PR with fix) - -``` -╔═══════════════════════════════════════════════════════════╗ -║ VERIFICATION PASSED ✅ ║ -╠═══════════════════════════════════════════════════════════╣ -║ - FAIL without fix (as expected) ║ -║ - PASS with fix (as expected) ║ -╚═══════════════════════════════════════════════════════════╝ -``` - -### If Tests Don't Behave as Expected - -**If tests PASS without fix** → Tests don't catch the bug. Let the user know the tests need to be fixed. They can use the `write-tests-agent` for help. - -### Complete 🚦 Gate - -Verify the following before proceeding: -- [ ] Test result documented (PASSED ✅ or FAILED ❌) -- [ ] Test behavior documented -- [ ] Platform tested noted -- [ ] **Write phase output to `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/gate/content.md`** (see SHARED-RULES.md "Phase Output Artifacts") - ---- - -## ⛔ STOP HERE - -**If Gate is `✅ PASSED`** → Read `.github/agents/pr/post-gate.md` to continue with phases 3-4. - -**If Gate `❌ FAILED`** → Stop. Request changes from the PR author to fix the tests. - ---- - -## Common Pre-Gate Mistakes - -- ❌ **Researching root cause during Pre-Flight** - Just document what the issue says, save analysis for Phase 3 -- ❌ **Looking at implementation code during Pre-Flight** - Just gather issue/PR context -- ❌ **Forming opinions on the fix during Pre-Flight** - That's Phase 3 -- ❌ **Running tests during Pre-Flight** - That's Phase 2 (Gate) -- ❌ **Skipping to Phase 3** - Gate MUST pass first - -## Common Gate Mistakes - -- ❌ **Running Gate verification inline** - Use task agent to prevent command substitution -- ❌ **Using `BuildAndRunHostApp.ps1` for Gate** - That only runs ONE direction; the skill does TWO runs -- ❌ **Using manual `dotnet test` commands** - Doesn't revert/restore fix files automatically -- ❌ **Claiming "fails both ways" from a single test run** - That's fabrication; you need the script's TWO runs -- ❌ **Not waiting for task agent completion** - Script takes 5-10+ minutes; wait for task to return - -**🚨 The verify-tests-fail.ps1 script does TWO test runs automatically:** -1. Reverts fix → runs tests (should FAIL) -2. Restores fix → runs tests (should PASS) - -Never run Gate inline. Always invoke as task agent. diff --git a/.github/agents/pr/PLAN-TEMPLATE.md b/.github/agents/pr/PLAN-TEMPLATE.md deleted file mode 100644 index 2cdeafd2ebd7..000000000000 --- a/.github/agents/pr/PLAN-TEMPLATE.md +++ /dev/null @@ -1,104 +0,0 @@ -# PR Review Plan Template - -**Reusable checklist** for the 4-phase PR Agent workflow. - -**Source documents:** -- `.github/agents/pr.md` - Phases 1-2 (Pre-Flight, Gate) -- `.github/agents/pr/post-gate.md` - Phases 3-4 (Fix, Report) -- `.github/agents/pr/SHARED-RULES.md` - Critical rules (blockers, git, templates) - ---- - -## 🚨 Critical Rules (Summary) - -See `SHARED-RULES.md` for complete details. Key points: -- **Environment Blockers**: Skip blocked phase and continue autonomously (no human operator) -- **No Git Commands**: Never checkout/switch branches - agent is always on correct branch -- **Gate via Task Agent**: Never run inline (prevents fabrication) -- **Multi-Model try-fix**: 5 models, SEQUENTIAL only -- **Follow Templates**: No `open` attributes, no "improvements" - ---- - -## Work Plan - -### Phase 1: Pre-Flight -- [ ] Gather PR metadata (title, body, labels, author) -- [ ] Fetch and read linked issue -- [ ] Fetch PR comments and review feedback -- [ ] Check for prior agent reviews (import and resume if found) -- [ ] Document platforms affected -- [ ] Classify changed files (fix vs test) -- [ ] Document PR's fix approach in Fix Candidates table -- [ ] **Write `PRAgent/pre-flight/content.md`** - -**Boundaries:** No code analysis, no fix opinions, no test running - -### Phase 2: Gate ⛔ -**🚨 Cannot continue if Gate fails** - -- [ ] Check if tests exist (if not, let the user know and suggest using `write-tests-agent`) -- [ ] Select platform (must be affected AND available on host) -- [ ] Invoke via **task agent** (NOT inline): - ``` - "Run verify-tests-fail-without-fix skill - Platform: [X], TestFilter: 'IssueXXXXX', RequireFullVerification: true" - ``` -- [ ] ⛔ If environment blocker: retry once, then skip and document -- [ ] Verify: Tests FAIL without fix, PASS with fix -- [ ] If Gate fails: STOP, request test fixes -- [ ] **Write `PRAgent/gate/content.md`** - -### Phase 3: Fix 🔧 -*(Only if Gate ✅ PASSED)* - -**Round 1: Run try-fix with each model (SEQUENTIAL)** -- [ ] claude-sonnet-4.6 -- [ ] claude-opus-4.6 -- [ ] gpt-5.2 -- [ ] gpt-5.3-codex -- [ ] gemini-3-pro-preview -- [ ] ⛔ If blocker: retry once, skip remaining models, proceed to Report -- [ ] Record: approach, result, files, failure analysis - -**Round 2+: Cross-Pollination (MANDATORY)** -- [ ] Invoke EACH model: "Any NEW fix ideas?" -- [ ] Record responses in Cross-Pollination table -- [ ] Run try-fix for new ideas (SEQUENTIAL) -- [ ] Repeat until ALL 6 say "NO NEW IDEAS" (max 3 rounds) - -**Completion:** -- [ ] Cross-Pollination table has all 6 responses -- [ ] Mark Exhausted: Yes -- [ ] Compare passing candidates with PR's fix -- [ ] Select best fix (results → simplicity → robustness) -- [ ] **Write `PRAgent/try-fix/content.md`** - -### Phase 4: Report 📋 -*(Only if Phases 1-3 complete)* - -- [ ] Run `pr-finalize` skill -- [ ] Generate review: root cause, candidates, recommendation -- [ ] **Write `PRAgent/report/content.md`** -- [ ] Post AI Summary comment (auto-loads from PRAgent/*/content.md): - ```bash - pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber XXXXX - pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber XXXXX - ``` -- [ ] Post PR Finalization comment (separate): - ```bash - pwsh .github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 -PRNumber XXXXX -SummaryFile CustomAgentLogsTmp/PRState/XXXXX/PRAgent/pr-finalize/pr-finalize-summary.md - ``` - ---- - -## Quick Reference - -| Phase | Key Action | Blocker Response | -|-------|------------|------------------| -| Pre-Flight | Gather context | N/A | -| Gate | Task agent → verify script | Skip, report incomplete | -| Fix | Multi-model try-fix | Skip remaining, proceed to Report | -| Report | Post via skill | Document what completed | - -**Never:** Claim success without tests, bypass scripts, stop and ask user diff --git a/.github/agents/pr/SHARED-RULES.md b/.github/agents/pr/SHARED-RULES.md deleted file mode 100644 index 4a5f547f0422..000000000000 --- a/.github/agents/pr/SHARED-RULES.md +++ /dev/null @@ -1,288 +0,0 @@ -# PR Agent: Shared Rules - -This file contains critical rules that apply across all PR agent phases. Referenced by `pr.md`, `post-gate.md`, and `PLAN-TEMPLATE.md`. - ---- - -## Phase Completion Protocol - -**Before changing ANY phase status to ✅ COMPLETE:** - -1. **Review the phase checklist** for the phase you're completing -2. **Verify all required items** are addressed -3. **Write the phase output to `content.md`** (see Phase Output Artifacts below) -4. **Then mark the phase** as ✅ COMPLETE - -**Rule:** Status ✅ means "work complete and verified", not "I finished thinking about it" - ---- - -## Phase Output Artifacts - -**After completing EACH phase, write a `content.md` file to the phase's output directory.** - -This is MANDATORY. The comment scripts (`post-ai-summary-comment.ps1`) read from these files to build the PR comment. - -### Output Directory Structure - -``` -CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/ -├── pre-flight/ -│ └── content.md # Written after Phase 1 -├── gate/ -│ ├── content.md # Written after Phase 2 -│ └── verify-tests-fail/ # Script output from verify-tests-fail.ps1 -├── try-fix/ -│ ├── content.md # Written after Phase 3 (summary of all attempts) -│ └── attempt-{N}/ # Individual attempt outputs from try-fix skill -└── report/ - └── content.md # Written after Phase 4 -``` - -### What Goes in Each content.md - -Each `content.md` should contain the **formatted phase content** — the same content that would appear inside the collapsible `
` section in the PR comment. - -**Pre-Flight example:** -```markdown -**Issue:** #XXXXX - [Title] -**Platforms Affected:** iOS, Android -**Files Changed:** 2 implementation files, 1 test file - -### Key Findings -- [Finding 1] -- [Finding 2] - -### Fix Candidates -| # | Source | Approach | Test Result | Files Changed | Notes | -|---|--------|----------|-------------|---------------|-------| -| PR | PR #XXXXX | [approach] | ⏳ PENDING (Gate) | `file.cs` | Original PR | -``` - -**Gate example:** -```markdown -**Result:** ✅ PASSED -**Platform:** android -**Mode:** Full Verification - -- Tests FAIL without fix ✅ -- Tests PASS with fix ✅ -``` - -**Fix (try-fix) example:** -```markdown -### Fix Candidates -| # | Source | Approach | Test Result | Files Changed | Notes | -|---|--------|----------|-------------|---------------|-------| -| 1 | try-fix | [approach] | ❌ FAIL | 1 file | Why: [reason] | -| 2 | try-fix | [approach] | ✅ PASS | 2 files | Works! | -| PR | PR #XXXXX | [approach] | ✅ PASS (Gate) | 2 files | Original PR | - -**Exhausted:** Yes -**Selected Fix:** PR's fix - [Reason] -``` - -**Report example:** -```markdown -## ✅ Final Recommendation: APPROVE - -### Summary -[Brief summary of the review] - -### Root Cause -[Root cause analysis] - -### Fix Quality -[Assessment of the fix] -``` - -### How to Write the File - -```bash -# Create the directory (idempotent) -mkdir -p "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pre-flight" - -# Write content (use create tool or bash) -cat > "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pre-flight/content.md" << 'EOF' -[phase content here] -EOF -``` - -### Timing - -| Phase | When to Write | -|-------|---------------| -| Pre-Flight | After all context gathered and documented | -| Gate | After verification result received from task agent | -| Fix (try-fix) | After all try-fix models explored and best fix selected | -| Report | After final recommendation determined | - ---- - -## Agent Labels (Automated by Review-PR.ps1) - -After all phases complete, `Review-PR.ps1` automatically applies GitHub labels based on phase outcomes. The agent does NOT need to apply labels — just write accurate `content.md` files. - -### Label Categories - -**Outcome labels** (mutually exclusive — exactly one per PR): -| Label | When Applied | -|-------|-------------| -| `s/agent-approved` | Report recommends APPROVE | -| `s/agent-changes-requested` | Report recommends REQUEST CHANGES | -| `s/agent-review-incomplete` | Agent didn't complete all phases | - -**Signal labels** (additive — multiple can coexist): -| Label | When Applied | -|-------|-------------| -| `s/agent-gate-passed` | Gate phase passes | -| `s/agent-gate-failed` | Gate phase fails | -| `s/agent-fix-win` | Agent found a better alternative fix than the PR | -| `s/agent-fix-pr-picked` | PR's fix is the best — agent couldn't beat it | - -**Tracking label** (always applied): -| Label | When Applied | -|-------|-------------| -| `s/agent-reviewed` | Every completed agent run | - -### How Labels Are Determined - -Labels are parsed from `content.md` files: -- **Outcome**: from `report/content.md` — looks for `Final Recommendation: APPROVE` or `REQUEST CHANGES` -- **Gate**: from `gate/content.md` — looks for `PASSED` or `FAILED` -- **Fix**: from `try-fix/content.md` — looks for alternative selected (win = agent beat PR) vs `Selected Fix: PR` (lose = PR was best) - -**Agent responsibility**: Write clear, parseable `content.md` with standard markers (`✅ PASSED`, `❌ FAILED`, `Selected Fix: PR`, `Final Recommendation: APPROVE`). - ---- - -## No Direct Git Commands - -**Never run git commands that change branch or file state.** - -The agent is always invoked from the correct branch. All file state management is handled by PowerShell scripts (`verify-tests-fail.ps1`, `try-fix`, etc.). - -**What to do instead:** -- Use `gh pr diff` or `gh pr view` to see PR info (read-only GitHub CLI) -- Use `gh pr diff --name-only` to list changed files -- Let scripts handle all file manipulation internally - -**Never run these commands:** -- ❌ `git checkout` (any form) -- ❌ `git switch` -- ❌ `git stash` -- ❌ `git reset` -- ❌ `git revert` -- ❌ `gh pr checkout` -- ❌ `git fetch` (for branch switching purposes) - ---- - -## Use Skills' Scripts - Don't Bypass - -When a skill provides a PowerShell script: -- **Run the script** - don't interpret what it does and do it manually -- **Fix inputs if script fails** - don't bypass with manual `gh` commands -- **Use `-DryRun` to debug** - see what the script would produce before posting -- Scripts handle formatting, API calls, and section management correctly - ---- - -## Stop on Environment Blockers - -If you encounter an environment or system setup blocker that prevents completing a phase: - -### 🚨 Autonomous Execution (Default) - -When running via `Review-PR.ps1`, there is **NO human operator** to respond to questions. - -**NEVER stop and ask the user. NEVER present options and wait for a choice. Nobody will respond.** - -Instead, use your best judgment to continue autonomously: - -1. **Try ONE retry** (install missing tool, kill conflicting process, etc.) -2. **If still blocked after one retry**, SKIP the blocked phase and continue to the next phase -3. **Document what was skipped and why** in your report -4. **Always prefer continuing with partial results** over stopping completely - -**Autonomous decision guide:** - -| Blocker Type | Max Retries | Then Do | -|--------------|-------------|---------| -| Missing tool/driver | 1 install attempt | Skip phase, continue | -| Server errors (500, timeout) | 1 retry | Skip phase, continue | -| Port conflicts | 1 (kill process) | Skip phase, continue | -| Build failures in try-fix | 2 attempts | Skip remaining try-fix models, proceed to Report | -| Configuration issues | 1 fix attempt | Skip phase, continue | - -**Common autonomous decisions:** -- Gate passes but Fix phase is blocked → **Skip Fix, proceed to Report** with Gate results only -- try-fix builds fail for multiple models → **Stop try-fix exploration, proceed to Report** -- A specific platform fails → **Try alternative platform ONCE**, then skip if still blocked -- Gate fails due to environment → **Report as incomplete**, proceed to Report - -### Interactive Mode - -When running with `-Interactive` flag, you MAY ask the user for guidance on blockers. - -### Common Blockers - -- Missing Appium drivers (Windows, iOS, Android) -- WinAppDriver not installed or returning errors -- Xcode/iOS simulators not available (on Windows) -- Android emulator not running or not configured -- Developer Mode not enabled -- Port conflicts (e.g., 4723 in use) -- Missing SDKs or tools -- Server errors (500, timeout, "unknown error occurred") - -### Never Do - -- ❌ Keep trying different fixes after retry limit exceeded -- ❌ Claim "verification passed" when tests couldn't actually run -- ❌ Install multiple tools/drivers without asking between each -- ❌ Spend more than 2-3 tool calls troubleshooting the same blocker -- ❌ **Stop and present options to the user** - choose the best option yourself -- ❌ **Wait for user response** - nobody will respond - ---- - -## Multi-Model Configuration - -Phase 4 uses these 5 AI models for try-fix exploration (run SEQUENTIALLY): - -| Order | Model | -|-------|-------| -| 1 | `claude-sonnet-4.6` | -| 2 | `claude-opus-4.6` | -| 3 | `gpt-5.2` | -| 4 | `gpt-5.3-codex` | -| 5 | `gemini-3-pro-preview` | - -**Note:** The `model` parameter is passed to the `task` tool, which supports model selection. This is separate from agent YAML frontmatter (which is VS Code-only). - -**⚠️ SEQUENTIAL ONLY**: try-fix runs modify the same files and use the same device. Never run in parallel. - ---- - -## Platform Selection - -**Choose a platform that is BOTH affected by the bug AND available on the current host.** - -### Step 1: Identify affected platforms from Pre-Flight -- Check issue labels (e.g., `platform/iOS`, `platform/Android`) -- Check which platform-specific files the PR modifies - -### Step 2: Match to available platforms - -| Host OS | Available Platforms | -|---------|---------------------| -| Windows | Android, Windows | -| macOS | Android, iOS, MacCatalyst | - -### Step 3: Select the best match -1. Pick a platform that IS affected by the bug -2. That IS available on the current host -3. Prefer the platform most directly impacted by the PR's code changes - -**⚠️ Do NOT test on a platform that isn't affected by the bug** - the test will pass regardless of whether the fix works. diff --git a/.github/agents/pr/post-gate.md b/.github/agents/pr/post-gate.md deleted file mode 100644 index 55e4e876aa7b..000000000000 --- a/.github/agents/pr/post-gate.md +++ /dev/null @@ -1,331 +0,0 @@ -# PR Agent: Post-Gate Phases (3-4) - -**⚠️ PREREQUISITE: Only read this file after 🚦 Gate shows `✅ PASSED`.** - -If Gate is not passed, go back to `.github/agents/pr.md` and complete phases 1-2 first. - ---- - -## Workflow Overview - -| Phase | Name | What Happens | -|-------|------|--------------| -| 3 | **Fix** | Invoke `try-fix` skill repeatedly to explore independent alternatives, then compare with PR's fix | -| 4 | **Report** | Deliver result (approve PR, request changes, or create new PR) | - ---- - -## 🚨 Critical Rules - -**All rules from `.github/agents/pr/SHARED-RULES.md` apply here**, including: -- Phase Completion Protocol (fill ALL pending fields before marking complete) -- Stop on Environment Blockers (retry once, then skip and continue autonomously) -- Multi-Model Configuration (5 models, SEQUENTIAL only) - -If try-fix cannot run due to environment issues after one retry, **skip the remaining try-fix models and proceed to Report**. Do NOT stop and ask the user. - -### 🚨 CRITICAL: Environment Blockers in Phase 3 - -The default mode is **non-interactive** — no human can respond to questions. - -If try-fix cannot run due to: -- Missing Appium drivers -- Device/emulator not available -- WinAppDriver not installed -- Platform tools missing -- Build failures unrelated to the fix - -**Use your best judgment to continue autonomously:** -1. Try ONE alternative (e.g., different platform, rebuild) -2. If still blocked, **skip remaining try-fix models and proceed to Report** -3. Document what was skipped and why in the Report phase -4. The PR's fix was already validated by Gate - that's sufficient for a recommendation - ---- - -## 🔧 FIX: Explore and Select Fix (Phase 3) - -> **SCOPE**: Explore independent fix alternatives using `try-fix` skill, compare with PR's fix, select the best approach. - -**⚠️ Gate Check:** Verify 🚦 Gate is `✅ PASSED` before proceeding. - -### 🚨 CRITICAL: try-fix is Independent of PR's Fix - -**The PR's fix has already been validated by Gate (tests FAIL without it, PASS with it).** - -The purpose of Phase 3 is NOT to re-test the PR's fix, but to: -1. **Generate independent fix ideas** - What would YOU do to fix this bug? -2. **Test those ideas empirically** - Actually implement and run tests -3. **Compare with PR's fix** - Is there a simpler/better alternative? -4. **Learn from failures** - Record WHY failed attempts didn't work - -**Do NOT let the PR's fix influence your thinking.** Generate ideas as if you hadn't seen the PR. - -### Step 1: Multi-Model try-fix Exploration - -Phase 3 uses a **multi-model approach** to maximize fix diversity. Each AI model brings different perspectives and may find solutions others miss. - -**⚠️ SEQUENTIAL ONLY**: try-fix runs MUST execute one at a time. They modify the same files and use the same test device. Never run try-fix attempts in parallel. - -#### Round 1: Run try-fix with Each Model - -Run the `try-fix` skill **6 times sequentially**, once with each model (see `SHARED-RULES.md` for model list). - -**For each model**, invoke the try-fix skill: -``` -Invoke the try-fix skill for PR #XXXXX: -- problem: [Description of the bug from issue/PR - what's broken and expected behavior] -- platform: [Use platform selected in Gate phase - must be affected by the bug AND available on host] -- test_command: pwsh .github/scripts/BuildAndRunHostApp.ps1 -Platform [same platform] -TestFilter "IssueXXXXX" -- target_files: - - src/[area]/[likely-affected-file-1].cs - - src/[area]/[likely-affected-file-2].cs - -Generate ONE independent fix idea. Review the PR's fix first to ensure your approach is DIFFERENT. -``` - -**Wait for each to complete before starting the next.** - -**🧹 MANDATORY: Clean up between attempts.** After each try-fix completes (pass or fail), run these commands before starting the next attempt: - -```bash -# 1. Restore any baseline state from the previous attempt (safe no-op if none exists) -pwsh .github/scripts/EstablishBrokenBaseline.ps1 -Restore - -# 2. Restore all tracked files to HEAD (the merged PR state) -# This catches any files the previous attempt modified but didn't restore -git checkout HEAD -- . - -# 3. Remove untracked files added by the previous attempt -# git checkout restores tracked files but does NOT remove new untracked files -git clean -fd --exclude=CustomAgentLogsTmp/ -``` - -**Why this is required:** Each try-fix attempt modifies source files. If an attempt fails mid-way (build error, timeout, model error), it may not run its own cleanup step. Without explicit cleanup, the next attempt starts with a dirty working tree, which can cause missing files, corrupt state, or misleading test results. Use `HEAD` (not just `-- .`) to also restore deleted files. - -#### Round 2+: Cross-Pollination Loop (MANDATORY) - -After Round 1, invoke EACH of the 5 models to ask for new ideas. **No shortcuts allowed.** - -**❌ WRONG**: Using `explore`/`glob`, declaring exhaustion without invoking each model -**✅ CORRECT**: Invoke EACH model via task agent and ask explicitly - -**Steps (repeat until all 6 say "NO NEW IDEAS", max 3 rounds):** - -1. **Compile bounded summary** (max 3-4 bullets per attempt): - - Attempt #, approach (1 line), result (✅/❌), key learning (1 line) - -2. **Invoke each model via task agent:** - ``` - agent_type: "task", model: "[model-name]" - prompt: "Review PR #XXXXX fix attempts: - - Attempt 1: [approach] - ✅/❌ - - Attempt 2: [approach] - ✅/❌ - Do you have any NEW fix ideas? Reply: 'NEW IDEA: [desc]' or 'NO NEW IDEAS'" - ``` - -3. **Record each model's response** in Cross-Pollination table - -4. **For each new idea**: Run try-fix with that model (SEQUENTIAL, wait for completion) - -5. **Exit when**: ALL 5 models say "NO NEW IDEAS" in the same round - -#### try-fix Behavior - -Each `try-fix` invocation (run via task agent with specific model): -1. Learns from prior failed attempts -2. Reverts PR's fix to get a broken baseline -3. Proposes ONE new independent fix idea -4. Implements and tests it -5. Records result (with failure analysis if it failed) -6. Reverts all changes (restores PR's fix) - -See `.github/skills/try-fix/SKILL.md` for full details. - -### Step 2: Compare Results - -After the loop, review the **Fix Candidates** table: - -```markdown -| # | Source | Approach | Test Result | Files Changed | Notes | -|---|--------|----------|-------------|---------------|-------| -| 1 | try-fix | Fix in TabbedPageManager | ❌ FAIL | 1 file | Why failed: Too late in lifecycle | -| 2 | try-fix | RequestApplyInsets only | ❌ FAIL | 1 file | Why failed: Trigger insufficient | -| 3 | try-fix | Reset + RequestApplyInsets | ✅ PASS | 2 files | Works! | -| PR | PR #33359 | [PR's approach] | ✅ PASS (Gate) | 2 files | Original PR | -``` - -**Compare passing candidates:** -- PR's fix (known to pass from Gate) -- Any try-fix attempts that passed - -### Step 3: Select Best Fix - -**Selection criteria (in order of priority):** -1. **Must pass tests** - Only consider candidates with ✅ PASS -2. **Simplest solution** - Fewer files, fewer lines, lower complexity -3. **Most robust** - Handles edge cases, less likely to regress -4. **Matches codebase style** - Consistent with existing patterns - -Update the selected fix: - -```markdown -**Exhausted:** Yes (or No if stopped early) -**Selected Fix:** PR's fix - [Reason] OR #N - [Reason why alternative is better] -``` - -**Possible outcomes:** -- **PR's fix is best** → Approve the PR -- **try-fix found a simpler/better alternative** → Request changes with suggestion -- **try-fix found same solution independently** → Strong validation, approve PR -- **All try-fix attempts failed** → PR's fix is the only working solution, approve PR -- **Multiple passing alternatives** → Select simplest/most robust - -### Step 4: Apply Selected Fix (if different from PR) - -**If PR's fix was selected:** -- No action needed - PR's changes are already in place - -**If a try-fix alternative was selected:** -- Re-implement the fix (you documented the approach in the table) -- Apply the changes to files (do not commit - user handles git) - -### Complete 🔧 Fix - -Verify the following before proceeding: -- [ ] Round 1 completed: All 5 models ran try-fix -- [ ] Cross-pollination completed with responses from ALL 5 models -- [ ] Fix Candidates table has numbered rows for each try-fix attempt -- [ ] Each row has: approach, test result, files changed, notes -- [ ] "Exhausted" field set to Yes (all models confirmed no new ideas) -- [ ] "Selected Fix" populated with reasoning -- [ ] Root cause analysis documented for the selected fix -- [ ] **Write phase output to `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/try-fix/content.md`** (see SHARED-RULES.md "Phase Output Artifacts") - -**🚨 If cross-pollination is missing, you skipped Round 2. Go back and invoke each model.** - ---- - -## 📋 REPORT: Final Report (Phase 4) - -> **SCOPE**: Deliver the final result - either a PR review or a new PR. - -**⚠️ Gate Check:** Verify ALL phases 1-3 are complete before proceeding. - -### Finalize Title and Description - -**Invoke the `pr-finalize` skill** to ensure the PR title and description: -- Accurately reflect the actual implementation -- Provide context for future agents (root cause, key insight, what to avoid) -- Follow the repository's PR template structure - -See `.github/skills/pr-finalize/SKILL.md` for details. - -If creating a new PR (from issue), use the skill's output template to write the PR body. -If reviewing an existing PR, check if title/description need updates and include in review. - -### If Starting from Issue (No PR) - Create PR - -1. **⛔ STOP: Ask user to commit and create PR**: - - Present a summary to the user and wait for them to handle git operations: - > "I've implemented the fix for issue #XXXXX. Here's what needs to be committed: - > - **Selected fix**: Candidate #N - [approach] - > - **Files changed**: [list files] - > - **Tests added**: [list test files] - > - **Other candidates considered**: [brief summary] - > - > Please commit these changes and create a PR when ready. - > Suggested PR title: `[Platform] Brief description of behavior fix` - > - > Use the pr-finalize skill output for the PR body." - - **Do NOT run git commands. User handles commit/push/PR creation.** - -### If Starting from PR - Write Review - -Determine your recommendation based on the Fix phase: - -**If PR's fix was selected:** -- Recommend: `✅ APPROVE` -- Justification: PR's approach is correct/optimal - -**If an alternative fix was selected:** -- Recommend: `⚠️ REQUEST CHANGES` -- Justification: Suggest the better approach from try-fix Candidate #N -- **Tell user:** "I've applied the alternative fix locally. Please review the changes and commit/push to update the PR." - -**If PR's fix failed tests:** -- Recommend: `⚠️ REQUEST CHANGES` -- Justification: Fix doesn't work, suggest alternatives - -**Check title/description accuracy:** -- Run the `pr-finalize` skill to verify title and description match implementation -- If discrepancies found, include suggested updates in review comments - -### Complete 📋 Report - -Verify the following before finishing: -- [ ] Final recommendation determined (APPROVE/REQUEST_CHANGES/COMMENT) -- [ ] Summary of findings prepared -- [ ] Key technical insights documented -- [ ] **Write phase output to `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/report/content.md`** (see SHARED-RULES.md "Phase Output Artifacts") -- [ ] Result presented to user - ---- - -## Common Mistakes in Post-Gate Phases - -- ❌ **Looking at PR's fix before generating ideas** - Generate fix ideas independently first -- ❌ **Re-testing the PR's fix in try-fix** - Gate already validated it; try-fix tests YOUR ideas -- ❌ **Skipping models in Round 1** - All 5 models must run try-fix before cross-pollination -- ❌ **Running try-fix in parallel** - SEQUENTIAL ONLY - they modify same files and use same device -- ❌ **Using explore/glob instead of invoking models** - Cross-pollination requires ACTUAL task agent invocations with each model, not code searches -- ❌ **Assuming "comprehensive coverage" = exhausted** - Only exhausted when all 5 models explicitly say "NO NEW IDEAS" -- ❌ **Not recording cross-pollination responses** - Must track each model's Round 2 response -- ❌ **Not analyzing why fixes failed** - Record the flawed reasoning to help future attempts -- ❌ **Selecting a failing fix** - Only select from passing candidates -- ❌ **Forgetting to revert between attempts** - Each try-fix must start from broken baseline, end with PR restored -- ❌ **Declaring exhaustion prematurely** - All 5 models must confirm "no new ideas" via actual invocation -- ❌ **Rushing the report** - Take time to write clear justification -- ❌ **Skipping cleanup between attempts** - ALWAYS run `-Restore` + `git checkout HEAD -- .` + `git clean -fd --exclude=CustomAgentLogsTmp/` between try-fix attempts (see Step 1) - ---- - -## Common Errors and Recovery - -### skill(try-fix) fails with "ENOENT: no such file or directory" - -**Symptom:** `skill(try-fix) Failed to read skill file: Error: ENOENT: no such file or directory, open '.../.github/skills/try-fix/SKILL.md'` - -**Root cause:** A previous try-fix attempt failed mid-way and left the working tree in a dirty state. Files may have been modified or deleted by `EstablishBrokenBaseline.ps1` without being restored. - -**Fix:** Run cleanup before retrying: -```bash -pwsh .github/scripts/EstablishBrokenBaseline.ps1 -Restore -git checkout HEAD -- . -git clean -fd --exclude=CustomAgentLogsTmp/ -``` - -Then retry the try-fix attempt. The skill file should now be accessible. - -**Prevention:** Always run the cleanup commands between try-fix attempts (see Step 1). - -### try-fix attempt starts with dirty working tree - -**Symptom:** `git status` shows modified files before the attempt starts, or the build fails with unexpected errors from files the attempt didn't touch. - -**Root cause:** Previous attempt didn't restore its changes (crashed, timed out, or model didn't follow Step 8 restore instructions). - -**Fix:** Same as above — run `-Restore` + `git checkout HEAD -- .` + `git clean -fd --exclude=CustomAgentLogsTmp/` to reset to the merged PR state. - -### Build errors unrelated to the fix being attempted - -**Symptom:** Build fails with errors in files the try-fix attempt didn't modify (e.g., XAML parse errors, unrelated compilation failures). - -**Root cause:** Often caused by dirty working tree from a previous attempt. Can also be transient environment issues. - -**Fix:** -1. Run cleanup: `pwsh .github/scripts/EstablishBrokenBaseline.ps1 -Restore && git checkout HEAD -- . && git clean -fd --exclude=CustomAgentLogsTmp/` -2. Retry the attempt -3. If it fails again with the same unrelated error, treat this as an environment/worktree blocker: STOP the try-fix workflow, do NOT continue with the next model, and ask the user to investigate (see "Stop on Environment Blockers"). diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index fb99bc2f87f5..62307bf4a263 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -242,24 +242,18 @@ The repository includes specialized custom agents and reusable skills for specif ### Available Custom Agents -1. **pr** - Sequential 4-phase workflow for reviewing and working on PRs - - **Use when**: A PR already exists and needs review or work, OR an issue needs a fix - - **Capabilities**: PR review, test verification, fix exploration, alternative comparison - - **Trigger phrases**: "review PR #XXXXX", "work on PR #XXXXX", "fix issue #XXXXX", "continue PR #XXXXX" - - **Do NOT use for**: Just running tests manually → Use `sandbox-agent` - -2. **write-tests-agent** - Agent for writing tests. Determines test type (UI vs XAML) and invokes the appropriate skill (`write-ui-tests`, `write-xaml-tests`) +1. **write-tests-agent** - Agent for writing tests. Determines test type (UI vs XAML) and invokes the appropriate skill (`write-ui-tests`, `write-xaml-tests`) - **Use when**: Creating new tests for issues or PRs - **Capabilities**: Test type determination (UI and XAML), skill invocation, test verification - **Trigger phrases**: "write tests for #XXXXX", "create tests", "add test coverage" -3. **sandbox-agent** - Specialized agent for working with the Sandbox app for testing, validation, and experimentation +2. **sandbox-agent** - Specialized agent for working with the Sandbox app for testing, validation, and experimentation - **Use when**: User wants to manually test PR functionality or reproduce issues - **Capabilities**: Sandbox app setup, Appium-based manual testing, PR functional validation - **Trigger phrases**: "test this PR", "validate PR #XXXXX in Sandbox", "reproduce issue #XXXXX", "try out in Sandbox" - - **Do NOT use for**: Code review (use pr agent), writing automated tests (use write-tests-agent) + - **Do NOT use for**: Code review (use `pr-review` skill), writing automated tests (use write-tests-agent) -4. **learn-from-pr** - Extracts lessons from PRs and applies improvements to the repository +3. **learn-from-pr** - Extracts lessons from PRs and applies improvements to the repository - **Use when**: After complex PR, want to improve instruction files/skills based on lessons learned - **Capabilities**: Analyzes PR, identifies failure modes, applies improvements to instruction files, skills, code comments - **Trigger phrases**: "learn from PR #XXXXX and apply improvements", "improve repo based on what we learned", "update skills based on PR" @@ -272,68 +266,84 @@ Skills are modular capabilities that can be invoked directly or used by agents. #### User-Facing Skills -1. **issue-triage** (`.github/skills/issue-triage/SKILL.md`) +1. **pr-review** (`.github/skills/pr-review/SKILL.md`) + - **Purpose**: End-to-end PR review orchestrator — follows phase instructions: pr-preflight, pr-gate, try-fix, pr-report + - **Trigger phrases**: "review PR #XXXXX", "work on PR #XXXXX", "fix issue #XXXXX", "continue PR #XXXXX" + - **Capabilities**: Multi-model fix exploration, test verification, alternative comparison, PR review recommendation + - **Do NOT use for**: Just running tests manually → Use `sandbox-agent` + - **Phase instructions** (in `.github/pr-review/`): + - `pr-preflight.md` — Context gathering from issue/PR + - `pr-gate.md` — Verify tests FAIL without fix, PASS with fix + - `pr-report.md` — Final recommendation + - **Phase skill**: `try-fix` — Multi-model fix exploration + +3. **issue-triage** (`.github/skills/issue-triage/SKILL.md`) - **Purpose**: Query and triage open issues that need milestones, labels, or investigation - **Trigger phrases**: "find issues to triage", "show me old Android issues", "what issues need attention" - **Scripts**: `init-triage-session.ps1`, `query-issues.ps1`, `record-triage.ps1` -2. **find-reviewable-pr** (`.github/skills/find-reviewable-pr/SKILL.md`) +4. **find-reviewable-pr** (`.github/skills/find-reviewable-pr/SKILL.md`) - **Purpose**: Finds open PRs in dotnet/maui and dotnet/docs-maui that need review - **Trigger phrases**: "find PRs to review", "show milestoned PRs", "find partner PRs" - **Scripts**: `query-reviewable-prs.ps1` - **Categories**: P/0, milestoned, partner, community, recent, docs-maui -3. **pr-finalize** (`.github/skills/pr-finalize/SKILL.md`) +5. **pr-finalize** (`.github/skills/pr-finalize/SKILL.md`) - **Purpose**: Verifies PR title and description match actual implementation, AND performs code review for best practices before merge. - **Trigger phrases**: "finalize PR #XXXXX", "check PR description for #XXXXX", "review commit message" - **Used by**: Before merging any PR, when description may be stale - **Note**: Works on any PR - **🚨 CRITICAL**: NEVER use `--approve` or `--request-changes` - only post comments. Approval is a human decision. -4. **learn-from-pr** (`.github/skills/learn-from-pr/SKILL.md`) +6. **learn-from-pr** (`.github/skills/learn-from-pr/SKILL.md`) - **Purpose**: Analyzes completed PR to identify repository improvements (analysis only, no changes applied) - **Trigger phrases**: "what can we learn from PR #XXXXX?", "how can we improve agents based on PR #XXXXX?" - **Used by**: After complex PRs, when agent struggled to find solution - **Output**: Prioritized recommendations for instruction files, skills, code comments - **Note**: For applying changes automatically, use the learn-from-pr agent instead -5. **write-ui-tests** (`.github/skills/write-ui-tests/SKILL.md`) +7. **write-ui-tests** (`.github/skills/write-ui-tests/SKILL.md`) - **Purpose**: Creates UI tests for GitHub issues and verifies they reproduce the bug - **Trigger phrases**: "write UI tests for #XXXXX", "create UI test for issue", "add UI test coverage" - **Output**: Test files that fail without fix, pass with fix -6. **write-xaml-tests** (`.github/skills/write-xaml-tests/SKILL.md`) +8. **write-xaml-tests** (`.github/skills/write-xaml-tests/SKILL.md`) - **Purpose**: Creates XAML unit tests for XAML parsing, compilation, and source generation - **Trigger phrases**: "write XAML tests for #XXXXX", "test XamlC behavior", "reproduce XAML parsing bug" - **Output**: Test files for Controls.Xaml.UnitTests -7. **verify-tests-fail-without-fix** (`.github/skills/verify-tests-fail-without-fix/SKILL.md`) +9. **verify-tests-fail-without-fix** (`.github/skills/verify-tests-fail-without-fix/SKILL.md`) - **Purpose**: Verifies UI tests catch the bug before fix and pass with fix - **Two modes**: Verify failure only (test creation) or full verification (test + fix) - **Used by**: After creating tests, before considering PR complete -8. **run-integration-tests** (`.github/skills/run-integration-tests/SKILL.md`) +10. **azdo-build-investigator** (`.github/skills/azdo-build-investigator/SKILL.md`) + - **Purpose**: Investigates CI failures for PRs — build errors, Helix test logs, and binlog analysis. Delegates to the `ci-analysis` skill from the dotnet/arcade-skills plugin. + - **Trigger phrases**: "check build for PR #XXXXX", "why did PR build fail", "get build status", "what's failing on PR", "Helix failures" + - **Used by**: When investigating CI failures + +11. **run-integration-tests** (`.github/skills/run-integration-tests/SKILL.md`) - **Purpose**: Build, pack, and run .NET MAUI integration tests locally - **Trigger phrases**: "run integration tests", "test templates locally", "run macOSTemplates tests", "run RunOniOS tests" - **Categories**: Build, WindowsTemplates, macOSTemplates, Blazor, MultiProject, Samples, AOT, RunOnAndroid, RunOniOS - **Note**: **ALWAYS use this skill** instead of manual `dotnet test` commands for integration tests -#### Internal Skills (Used by Agents) +#### Internal Skills (Used by Skills/Agents) -9. **try-fix** (`.github/skills/try-fix/SKILL.md`) +12. **try-fix** (`.github/skills/try-fix/SKILL.md`) - **Purpose**: Proposes ONE independent fix approach, applies it, tests, records result with failure analysis, then reverts - - **Used by**: pr agent Phase 3 (Fix phase) - rarely invoked directly by users + - **Used by**: `pr-review` skill Phase 3 (Try-Fix phase) - rarely invoked directly by users - **Behavior**: Reads prior attempts to learn from failures. Max 5 attempts per session. - **Output**: Reports attempt results and failure analysis -### Using Custom Agents +### Using Custom Agents and Skills -**Delegation Policy**: When user request matches agent trigger phrases, **ALWAYS delegate to the appropriate agent immediately**. Do not ask for permission or explain alternatives unless the request is ambiguous. +**Delegation Policy**: When user request matches skill/agent trigger phrases, **ALWAYS invoke the appropriate skill or delegate to the agent immediately**. Do not ask for permission or explain alternatives unless the request is ambiguous. **Examples of correct delegation**: -- User: "Review PR #12345" → Immediately invoke **pr** agent +- User: "Review PR #12345" → Immediately invoke **pr-review** skill - User: "Test this PR" → Immediately invoke **sandbox-agent** -- User: "Fix issue #67890" (no PR exists) → Suggest using `/delegate` command +- User: "Fix issue #67890" → Immediately invoke **pr-review** skill - User: "Write tests for issue #12345" → Immediately invoke **write-tests-agent** **When NOT to delegate**: diff --git a/.github/docs/agent-labels.md b/.github/docs/agent-labels.md index 7025d26f8f2e..2a43521e4c14 100644 --- a/.github/docs/agent-labels.md +++ b/.github/docs/agent-labels.md @@ -26,12 +26,12 @@ Additive — **multiple** can coexist on a single PR. | Label | Color | Description | Applied When | |-------|-------|-------------|--------------| -| `s/agent-gate-passed` | 🟢 `#4CAF50` | AI verified tests catch the bug (fail without fix, pass with fix) | Gate phase passes | -| `s/agent-gate-failed` | 🟠 `#FF9800` | AI could not verify tests catch the bug | Gate phase fails | +| `s/agent-gate-passed` | 🟢 `#4CAF50` | AI verified tests catch the bug (fail without fix, pass with fix) | Validate phase passes | +| `s/agent-gate-failed` | 🟠 `#FF9800` | AI could not verify tests catch the bug | Validate phase fails | | `s/agent-fix-win` | 🟢 `#66BB6A` | AI found a better alternative fix than the PR | Fix phase: alternative selected over PR's fix | | `s/agent-fix-pr-picked` | 🟠 `#FF7043` | AI could not beat the PR fix — PR is the best among all candidates | Fix phase: PR selected as best after comparison | -Gate labels (`gate-passed`/`gate-failed`) are mutually exclusive with each other. Fix labels (`fix-win`/`fix-lose`) are mutually exclusive with each other. +Validate labels (`gate-passed`/`gate-failed`) are mutually exclusive with each other. Fix labels (`fix-win`/`fix-lose`) are mutually exclusive with each other. ### Tracking Label @@ -57,9 +57,9 @@ Applied by MAUI maintainers, not by automation. ``` Review-PR.ps1 -├── Phase 1: PR Agent Review (Copilot CLI) +├── Phase 1: Agent Review (Copilot CLI) │ ├── Pre-Flight → writes content.md -│ ├── Gate → writes content.md +│ ├── Validate → writes content.md │ ├── Fix → writes content.md │ └── Report → writes content.md ├── Phase 2: PR Finalize (optional) @@ -125,7 +125,7 @@ is:pr label:s/agent-reviewed |--------|-------| | Total agent reviews | `is:pr label:s/agent-reviewed` | | Approval rate | Compare `label:s/agent-approved` vs `label:s/agent-changes-requested` counts | -| Gate pass rate | Compare `label:s/agent-gate-passed` vs `label:s/agent-gate-failed` counts | +| Validate pass rate | Compare `label:s/agent-gate-passed` vs `label:s/agent-gate-failed` counts | | Fix win rate | Compare `label:s/agent-fix-win` vs `label:s/agent-fix-pr-picked` counts | | Agent adoption rate | `label:s/agent-fix-implemented` / `label:s/agent-changes-requested` | | Incomplete review rate | `label:s/agent-review-incomplete` / `label:s/agent-reviewed` | @@ -140,7 +140,7 @@ is:pr label:s/agent-reviewed |------|---------| | `.github/scripts/shared/Update-AgentLabels.ps1` | Label helper module (all label logic) | | `.github/scripts/Review-PR.ps1` | Orchestrator that calls `Apply-AgentLabels` in Phase 4 | -| `.github/agents/pr/SHARED-RULES.md` | Documents label system for the PR agent | +| `.github/skills/pr-review/SKILL.md` | Documents label system for the pr-review skill | ### Key Functions @@ -149,7 +149,7 @@ is:pr label:s/agent-reviewed | `Apply-AgentLabels` | Main entry point — parses phases and applies all labels | | `Parse-PhaseOutcomes` | Reads `content.md` files, returns outcome/gate/fix results | | `Update-AgentOutcomeLabel` | Applies one outcome label, removes conflicting ones | -| `Update-AgentSignalLabels` | Adds/removes gate and fix signal labels | +| `Update-AgentSignalLabels` | Adds/removes validate and fix signal labels | | `Update-AgentReviewedLabel` | Ensures tracking label is present | | `Ensure-LabelExists` | Creates or updates a label in the repository | diff --git a/.github/instructions/sandbox.instructions.md b/.github/instructions/sandbox.instructions.md index 6977fa1fdb1e..ddff19634ba2 100644 --- a/.github/instructions/sandbox.instructions.md +++ b/.github/instructions/sandbox.instructions.md @@ -170,7 +170,7 @@ Work with the Sandbox app for manual testing, PR validation, issue reproduction, ## Distinction: Code Review vs. Functional Testing -**Code Review** (pr agent): +**Code Review** (pr-review skill): - Analyzes code quality, patterns, best practices - Reviews test coverage and correctness - Checks for potential bugs or issues in the code itself diff --git a/.github/pr-review/pr-gate.md b/.github/pr-review/pr-gate.md new file mode 100644 index 000000000000..817496178333 --- /dev/null +++ b/.github/pr-review/pr-gate.md @@ -0,0 +1,96 @@ +# PR Gate — Test Verification + +> **⛔ This phase MUST pass before continuing to Try-Fix. If it fails, stop and inform user.** + +> 🚨 Gate verification MUST run via task agent — never inline. + +--- + +## Prerequisites + +- Pre-Flight phase must be ✅ COMPLETE before starting +- Platform must be selected (affected by bug AND available on host) + +### Platform Selection + +Choose a platform that is BOTH affected by the bug AND available on the current host: + +| Host OS | Available Platforms | +|---------|---------------------| +| Windows | Android, Windows | +| macOS | Android, iOS, MacCatalyst | + +⚠️ Do NOT test on a platform unaffected by the bug — the test will pass regardless. + +--- + +## Steps + +1. **Check if tests exist:** + ```bash + gh pr view XXXXX --json files --jq '.files[].path' | grep -E "TestCases\.(HostApp|Shared\.Tests)" + ``` + If NO tests exist → inform user, suggest `write-tests-agent`. Gate is ⚠️ SKIPPED. + +2. **Select platform** — must be affected by bug AND available on host (see Platform Selection above). + +3. **Run verification via task agent** (MUST use task agent — never inline): + ``` + Invoke the `task` agent with this prompt: + + "Invoke the verify-tests-fail-without-fix skill for this PR: + - Platform: {platform} + - TestFilter: 'IssueXXXXX' + - RequireFullVerification: true + + Report back: Did tests FAIL without fix? Did tests PASS with fix? Final status?" + ``` + +**Why task agent?** Running inline allows substituting commands and fabricating results. Task agent runs in isolation. + +--- + +## Expected Result + +``` +╔═══════════════════════════════════════════════════════════╗ +║ VERIFICATION PASSED ✅ ║ +╠═══════════════════════════════════════════════════════════╣ +║ - FAIL without fix (as expected) ║ +║ - PASS with fix (as expected) ║ +╚═══════════════════════════════════════════════════════════╝ +``` + +--- + +## If Gate Fails + +- **Tests PASS without fix** → Tests don't catch the bug. Inform user, suggest `write-tests-agent`. +- **Tests FAIL with fix** → PR's fix doesn't work. Skip Try-Fix, proceed to Report with ⚠️ REQUEST CHANGES. + +--- + +## Output File + +```bash +mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/gate +``` + +Write `content.md`: +```markdown +### Gate Result: {✅ PASSED / ❌ FAILED / ⚠️ SKIPPED} + +**Platform:** {platform} +**Mode:** Full Verification + +- Tests FAIL without fix: {✅/❌} +- Tests PASS with fix: {✅/❌} +``` + +--- + +## Common Mistakes + +- ❌ Running inline — MUST use task agent +- ❌ Using `BuildAndRunHostApp.ps1` — that runs ONE direction; the skill does TWO +- ❌ Claiming results from a single test run — script does TWO runs automatically diff --git a/.github/pr-review/pr-preflight.md b/.github/pr-review/pr-preflight.md new file mode 100644 index 000000000000..0c5ae3f9b0b3 --- /dev/null +++ b/.github/pr-review/pr-preflight.md @@ -0,0 +1,67 @@ +# PR Pre-Flight — Context Gathering + +> **SCOPE:** Document only. No code analysis. No fix opinions. No running tests. + +--- + +## Steps + +1. **Read the issue** — full body + ALL comments via GitHub MCP tools +2. **Find the PR** — read description, diff summary, review comments, inline feedback +3. **Fetch PR discussion** — detect prior agent reviews, import findings if found +4. **Classify files** — separate fix files from test files, identify test type (UI / Device / Unit) +5. **Document edge cases** — from comments mentioning "what about...", "does this work with..." +6. **Record PR's fix** in Fix Candidates table (pending validation) + +```bash +# Fetch PR metadata +gh pr view XXXXX --json title,body,url,author,labels,files + +# Find linked issue +gh pr view XXXXX --json body --jq '.body' | grep -oE "(Fixes|Closes|Resolves) #[0-9]+" | head -1 +gh issue view ISSUE_NUMBER --json title,body,comments + +# PR comments +gh pr view XXXXX --json comments --jq '.comments[] | "Author: \(.author.login)\n\(.body)\n---"' + +# Inline review comments (CRITICAL — often contains key technical feedback) +gh api "repos/dotnet/maui/pulls/XXXXX/comments" --jq '.[] | "File: \(.path):\(.line // .original_line)\nAuthor: \(.user.login)\n\(.body)\n---"' + +# Detect prior agent reviews +gh pr view XXXXX --json comments --jq '.comments[] | select(.body | contains("Final Recommendation") and contains("| Phase | Status |")) | .body' +``` + +**If prior agent review found:** Parse phase statuses, import findings, resume from incomplete phase. + +--- + +## Output File + +```bash +mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/pre-flight +``` + +Write `content.md`: +```markdown +**Issue:** #{IssueNumber} - {Title} +**PR:** #{PRNumber} - {Title} +**Platforms Affected:** {platforms} +**Files Changed:** {count} implementation, {count} test + +### Key Findings +- {Finding 1} +- {Finding 2} + +### Fix Candidates +| # | Source | Approach | Test Result | Files Changed | Notes | +|---|--------|----------|-------------|---------------|-------| +| PR | PR #XXXXX | {approach} | ⏳ PENDING (Gate) | `file.cs` | Original PR | +``` + +--- + +## Common Mistakes + +- ❌ Researching root cause — save for Try-Fix phase +- ❌ Looking at implementation code — just gather context +- ❌ Running tests — that's the Gate phase diff --git a/.github/pr-review/pr-report.md b/.github/pr-review/pr-report.md new file mode 100644 index 000000000000..ffe233ad1172 --- /dev/null +++ b/.github/pr-review/pr-report.md @@ -0,0 +1,86 @@ +# PR Report — Final Recommendation + +> **SCOPE:** Deliver the review recommendation. Output files only — no comments posted. + +> 🚨 **DO NOT post any comments.** This phase only produces output files. + +--- + +## Prerequisites + +- Phases 1-3 (Pre-Flight, Gate, Try-Fix) must be complete before starting + +--- + +## Steps + +1. **Determine recommendation:** + + | Condition | Recommendation | + |-----------|----------------| + | PR's fix selected and Gate passed | `✅ APPROVE` | + | Alternative fix found via Try-Fix | `⚠️ REQUEST CHANGES` — suggest alternative | + | Gate failed | `⚠️ REQUEST CHANGES` — fix doesn't work | + +2. **Write output files** — Save recommendation to `content.md` + +> 🚨 **DO NOT post comments.** This phase only produces output files. +> +> 🚨 **DO NOT run pr-finalize.** That is a separate skill invoked only when the user explicitly requests it. + +--- + +## Output File + +```bash +mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/report +``` + +Write `content.md`: +```markdown +## {✅/⚠️} Final Recommendation: {APPROVE/REQUEST CHANGES} + +### Phase Status +| Phase | Status | Notes | +|---|---|---| +| Pre-Flight | ✅ COMPLETE | {notes} | +| Gate | ✅ PASSED | {platform} | +| Try-Fix | ✅ COMPLETE | {N} attempts, {M} passing | +| Report | ✅ COMPLETE | | + +### Summary +{Brief summary of the review} + +### Root Cause +{Root cause analysis} + +### Fix Quality +{Assessment of the fix} +``` + +--- + +## Agent Labels (Automated) + +After Report completes, `Review-PR.ps1` automatically applies labels based on `content.md` files: + +| Label | When Applied | +|-------|-------------| +| `s/agent-approved` | Report recommends APPROVE | +| `s/agent-changes-requested` | Report recommends REQUEST CHANGES | +| `s/agent-review-incomplete` | Agent didn't complete all phases | +| `s/agent-gate-passed` | Gate phase passes | +| `s/agent-gate-failed` | Gate phase fails | +| `s/agent-fix-win` | Agent found a better alternative | +| `s/agent-fix-pr-picked` | PR's fix was best | +| `s/agent-reviewed` | Every completed run | + +Standard markers in content.md: `✅ PASSED`, `❌ FAILED`, `Selected Fix: PR`, `Final Recommendation: APPROVE`. + +--- + +## Common Mistakes + +- ❌ Rushing the report — take time for clear justification +- ❌ Running git commands — user handles commit/push +- ❌ Posting comments — this phase only produces output files, never posts to GitHub diff --git a/.github/scripts/EstablishBrokenBaseline.ps1 b/.github/scripts/EstablishBrokenBaseline.ps1 index 70a2f8a1c19b..2a6d57506ec2 100644 --- a/.github/scripts/EstablishBrokenBaseline.ps1 +++ b/.github/scripts/EstablishBrokenBaseline.ps1 @@ -335,11 +335,31 @@ if ($Restore) { } } +# ============================================================ +# AUTO-RESTORE: If a previous baseline is still active, restore it first +# ============================================================ +# This prevents the Establish→fail→Establish loop that caused build #13539436 +# to waste 3.7 hours. Instead of erroring on a dirty tree, we detect that a +# prior baseline was never restored and clean it up automatically. + +$existingState = Get-BaselineState +if ($existingState) { + Write-Host "⚠️ Previous baseline still active — auto-restoring before re-establishing..." -ForegroundColor Yellow + + foreach ($file in $existingState.RevertedFiles) { + Write-Host " Restoring: $file" -ForegroundColor Gray + git checkout HEAD -- $file 2>&1 | Out-Null + } + + Remove-BaselineState + Write-Host " Previous baseline restored." -ForegroundColor Green +} + # ============================================================ # FAIL-FAST: Require clean working directory # ============================================================ -# This check ensures every successful baseline establishment started from a clean state. -# If this script completes without error, the baseline was valid - no checkpoint logging needed. +# After auto-restore above, the tree should be clean. If it's still dirty, +# something else is wrong (manual edits, uncommitted work, etc.). $dirtyFiles = git status --porcelain --untracked-files=no 2>$null if ($dirtyFiles) { diff --git a/.github/scripts/Review-PR.ps1 b/.github/scripts/Review-PR.ps1 index 4b96d435f8c1..e8547572e751 100644 --- a/.github/scripts/Review-PR.ps1 +++ b/.github/scripts/Review-PR.ps1 @@ -1,64 +1,40 @@ <# .SYNOPSIS - Runs a PR review using Copilot CLI and the PR Agent workflow. + Runs a PR review using Copilot CLI with skill-based prompts. .DESCRIPTION - This script invokes Copilot CLI to perform a comprehensive 4-phase PR review: + Orchestrates a 5-step PR review by invoking Copilot CLI with skill prompts: - Phase 1: Pre-Flight - Context gathering - Phase 2: Gate - Verify tests catch the bug - Phase 3: Fix - Multi-model exploration of alternatives - Phase 4: Report - Final recommendation - - The script: - - Validates prerequisites (gh CLI, PR exists) - - Validates current branch is not protected (main, release/*, net*.0) - - Merges the PR into the current branch (for isolated testing) - - Creates the state directory - - Invokes Copilot CLI with the pr agent + Step 0: Branch setup - Create review branch from main, merge PR squashed + Step 1: pr-review skill - 4-phase review (Pre-Flight, Gate, Try-Fix, Report) + Step 2: pr-finalize skill - Verify PR title/description match implementation + Step 3: Post AI Summary - Directly runs posting scripts (review + finalize) + Step 4: Apply labels - Apply agent labels based on review results + + By default, the script checks out main and creates a review branch from it. + If squash-merge conflicts, the script posts a comment on the PR and exits. + Use -UseCurrentBranch to create the review branch from the current branch instead. .PARAMETER PRNumber - The GitHub PR number to review (e.g., 33687) + The GitHub PR number to review .PARAMETER Platform - The platform to use for testing. Default is 'android'. - Valid values: android, ios, windows, maccatalyst - -.PARAMETER SkipMerge - If specified, skips merging the PR into the current branch (useful if already merged) + Platform for testing. Valid: android, ios, windows, maccatalyst -.PARAMETER Interactive - If specified, starts Copilot in interactive mode with the prompt. - Default is non-interactive mode (exits after completion). +.PARAMETER UseCurrentBranch + Create the review branch from the current branch instead of main. + By default, the script checks out main before creating the review branch. .PARAMETER DryRun - If specified, shows what would be done without making changes + Show what would be done without making changes -.PARAMETER RunFinalize - If specified, runs the pr-finalize skill after the PR agent completes - to verify PR title/description match the implementation. - -.PARAMETER PostSummaryComment - If specified, runs the ai-summary-comment skill after all other phases complete - to post a combined summary comment on the PR from all phases. +.PARAMETER LogFile + Capture all output via Start-Transcript .EXAMPLE .\Review-PR.ps1 -PRNumber 33687 - Reviews PR #33687 in non-interactive mode (default) using auto-detected platform - -.EXAMPLE - .\Review-PR.ps1 -PRNumber 33687 -Platform ios -SkipMerge - Reviews PR #33687 on iOS without merging (assumes already merged) - -.EXAMPLE - .\Review-PR.ps1 -PRNumber 33687 -Interactive - Reviews PR #33687 in interactive mode (stays open for follow-up questions) - -.NOTES - Prerequisites: - - GitHub CLI (gh) installed and authenticated - - Copilot CLI (copilot) installed - - For testing: Appropriate platform tools (Appium, emulators, etc.) + .\Review-PR.ps1 -PRNumber 33687 -Platform ios + .\Review-PR.ps1 -PRNumber 33687 -UseCurrentBranch #> [CmdletBinding()] @@ -68,30 +44,20 @@ param( [Parameter(Mandatory = $false)] [ValidateSet('android', 'ios', 'windows', 'maccatalyst')] - [string]$Platform, # Optional - agent will determine appropriate platform if not specified + [string]$Platform, [Parameter(Mandatory = $false)] - [switch]$SkipMerge, - - [Parameter(Mandatory = $false)] - [switch]$Interactive, + [switch]$UseCurrentBranch, [Parameter(Mandatory = $false)] [switch]$DryRun, [Parameter(Mandatory = $false)] - [switch]$PostSummaryComment, - - [Parameter(Mandatory = $false)] - [switch]$RunFinalize, - - [Parameter(Mandatory = $false)] - [string]$LogFile # If provided, captures all output via Start-Transcript + [string]$LogFile ) $ErrorActionPreference = 'Stop' -# Start transcript logging if LogFile specified (replaces external tee pipe) if ($LogFile) { $logDir = Split-Path $LogFile -Parent if ($logDir -and -not (Test-Path $logDir)) { @@ -100,13 +66,10 @@ if ($LogFile) { Start-Transcript -Path $LogFile -Force | Out-Null } -# Get repository root $RepoRoot = git rev-parse --show-toplevel 2>$null -if (-not $RepoRoot) { - Write-Error "Not in a git repository" - exit 1 -} +if (-not $RepoRoot) { Write-Error "Not in a git repository"; exit 1 } +# ─── Banner ─────────────────────────────────────────────────────────────────── Write-Host "" Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan Write-Host "║ PR Review with Copilot CLI ║" -ForegroundColor Cyan @@ -120,485 +83,478 @@ if ($Platform) { Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan Write-Host "" -# Step 1: Verify prerequisites +# ─── Prerequisites ──────────────────────────────────────────────────────────── Write-Host "📋 Checking prerequisites..." -ForegroundColor Yellow -# Check GitHub CLI $ghVersion = gh --version 2>$null | Select-Object -First 1 -if (-not $ghVersion) { - Write-Error "GitHub CLI (gh) is not installed. Install from: https://cli.github.com/" - exit 1 -} +if (-not $ghVersion) { Write-Error "GitHub CLI (gh) not installed"; exit 1 } Write-Host " ✅ GitHub CLI: $ghVersion" -ForegroundColor Green -# Check Copilot CLI - use Get-Command (reliable) then get version with merged streams $copilotCmd = Get-Command copilot -ErrorAction SilentlyContinue -if (-not $copilotCmd) { - Write-Error "Copilot CLI is not installed. Install with: npm install -g @github/copilot" - exit 1 -} +if (-not $copilotCmd) { Write-Error "Copilot CLI not installed"; exit 1 } $copilotVersion = (& copilot --version 2>&1 | Out-String).Trim() if (-not $copilotVersion) { $copilotVersion = $copilotCmd.Source } Write-Host " ✅ Copilot CLI: $copilotVersion" -ForegroundColor Green -# Check PR exists -Write-Host " 🔍 Verifying PR #$PRNumber exists..." -ForegroundColor Gray -$prInfo = gh pr view $PRNumber --json title,state,url 2>$null | ConvertFrom-Json -if (-not $prInfo) { - Write-Error "PR #$PRNumber not found or not accessible" - exit 1 -} +$prInfo = gh pr view $PRNumber --json title,state 2>$null | ConvertFrom-Json +if (-not $prInfo) { Write-Error "PR #$PRNumber not found"; exit 1 } Write-Host " ✅ PR: $($prInfo.title)" -ForegroundColor Green -Write-Host " ✅ State: $($prInfo.state)" -ForegroundColor Green -# Step 2: Validate current branch and merge PR -Write-Host "" -$currentBranch = git branch --show-current -Write-Host "📍 Current branch: $currentBranch" -ForegroundColor Yellow +# ─── Shared prompt rules ───────────────────────────────────────────────────── +$platformInstruction = if ($Platform) { + "**Platform for testing:** $Platform" +} else { + "**Platform for testing:** Determine from PR's affected code paths and current host OS." +} -# Check if on a protected branch (main, release/*, net*.0) -$protectedBranches = @('main', 'master') -$isProtected = $protectedBranches -contains $currentBranch -or - $currentBranch -match '^release/' -or - $currentBranch -match '^net\d+\.0$' +$autonomousRules = @" -if ($isProtected) { - Write-Host "" - Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Red - Write-Host "║ ERROR: Cannot run on protected branch! ║" -ForegroundColor Red - Write-Host "╠═══════════════════════════════════════════════════════════╣" -ForegroundColor Red - Write-Host "║ Current branch: $currentBranch" -ForegroundColor Red - Write-Host "║ ║" -ForegroundColor Red - Write-Host "║ This script merges the PR into the current branch. ║" -ForegroundColor Red - Write-Host "║ Protected branches: main, release/*, net*.0 ║" -ForegroundColor Red - Write-Host "║ ║" -ForegroundColor Red - Write-Host "║ Please checkout a working branch first: ║" -ForegroundColor Red - Write-Host "║ git checkout -b pr-review-$PRNumber ║" -ForegroundColor Red - Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Red - Write-Host "" - exit 1 -} +🚨 **AUTONOMOUS EXECUTION:** +- There is NO human operator - NEVER stop and ask for input +- On environment blockers: skip the blocked phase and continue +- Always prefer CONTINUING with partial results over STOPPING +"@ -Write-Host " ✅ Branch '$currentBranch' is not protected" -ForegroundColor Green +# ═════════════════════════════════════════════════════════════════════════════ +# STEP 0: Branch Setup (Create Review Branch & Cherry-Pick PR) +# ═════════════════════════════════════════════════════════════════════════════ -# Merge the PR into the current branch (unless skipped) -if (-not $SkipMerge) { - Write-Host "" - Write-Host "🔀 Merging PR #$PRNumber into current branch..." -ForegroundColor Yellow - - if ($DryRun) { - Write-Host " [DRY RUN] Would fetch and merge PR #$PRNumber" -ForegroundColor Magenta +Write-Host "" +Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Yellow +Write-Host "║ STEP 0: BRANCH SETUP ║" -ForegroundColor Yellow +Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Yellow + +$reviewBranch = "pr-review-$PRNumber" + +if ($DryRun) { + if ($UseCurrentBranch) { + Write-Host "[DRY RUN] Would create review branch '$reviewBranch' from current branch" -ForegroundColor Magenta } else { - # Fetch the PR ref and merge it - Write-Host " 📥 Fetching PR #$PRNumber..." -ForegroundColor Gray - git fetch origin pull/$PRNumber/head:temp-pr-$PRNumber 2>$null + Write-Host "[DRY RUN] Would checkout main, then create review branch '$reviewBranch'" -ForegroundColor Magenta + } + Write-Host "[DRY RUN] Would squash-merge PR #$PRNumber (stops on conflicts)" -ForegroundColor Magenta +} else { + # In CI pipelines, prior steps (Build MSBuild Tasks, Install Appium) may leave + # modified tracked files (e.g. HybridWebView.js) or untracked dirs (e.g. .appium/). + # Clean them so the dirty-tree check doesn't fail on build artifacts. + if ($env:TF_BUILD) { + git checkout -- . 2>$null + git clean -fd -e CustomAgentLogsTmp/ 2>$null + } + + # Check for dirty working tree + $dirtyFiles = git status --porcelain 2>$null + if ($dirtyFiles) { + Write-Error "Working tree is dirty. Please commit or stash changes before running review.`n$dirtyFiles" + exit 1 + } + + # Delete leftover review branch from a previous run (if it exists) + $existingBranch = git branch --list $reviewBranch 2>$null + if ($existingBranch) { + Write-Host " ⚠️ Removing leftover branch '$reviewBranch' from previous run" -ForegroundColor Yellow + git branch -D $reviewBranch 2>$null + } + + # Auto-detect CI environment — in CI, always use current branch + $isCI = $env:CI -or $env:TF_BUILD -or $env:GITHUB_ACTIONS -or $env:BUILD_BUILDID + if ($isCI -and -not $UseCurrentBranch) { + Write-Host " 🤖 CI environment detected — using current branch instead of main" -ForegroundColor Cyan + $UseCurrentBranch = $true + } + + # Capture original branch so error paths can restore it (not `git checkout -` which is unreliable) + $originalBranch = git branch --show-current 2>$null + if (-not $originalBranch) { $originalBranch = git rev-parse HEAD 2>$null } + + if (-not $UseCurrentBranch) { + # Default: checkout main first + Write-Host " 📌 Checking out main branch..." -ForegroundColor Cyan + git checkout main 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { Write-Error "Failed to checkout main"; exit 1 } + $pullOutput = git pull origin main --ff-only 2>&1 if ($LASTEXITCODE -ne 0) { - # Try fetching from the PR's head repository (for fork PRs) - $prDetails = gh pr view $PRNumber --json headRepositoryOwner,headRefName 2>$null | ConvertFrom-Json - if ($prDetails) { - $forkOwner = $prDetails.headRepositoryOwner.login - $headRef = $prDetails.headRefName - Write-Host " 📥 PR is from fork: $forkOwner, fetching..." -ForegroundColor Gray - git fetch "https://github.com/$forkOwner/maui.git" "${headRef}:temp-pr-$PRNumber" - if ($LASTEXITCODE -ne 0) { - Write-Error "Failed to fetch PR #$PRNumber from fork" - exit 1 - } - } else { - Write-Error "Failed to fetch PR #$PRNumber" - exit 1 - } + Write-Host " ⚠️ git pull failed (non-fatal, continuing with local main): $pullOutput" -ForegroundColor Yellow } - - Write-Host " 🔀 Merging into '$currentBranch'..." -ForegroundColor Gray - git merge "temp-pr-$PRNumber" --no-edit + $baseSha = git rev-parse --short HEAD 2>$null + Write-Host " 📌 Review base: main @ $baseSha" -ForegroundColor Cyan + } else { + $currentBranch = git branch --show-current 2>$null + if (-not $currentBranch) { $currentBranch = "(detached HEAD)" } + Write-Host " 📌 Using current branch: $currentBranch" -ForegroundColor Cyan + } + + # Create review branch + Write-Host " 🔀 Creating review branch: $reviewBranch" -ForegroundColor Cyan + git checkout -b $reviewBranch 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { Write-Error "Failed to create branch '$reviewBranch'"; exit 1 } + + # Fetch PR commits + Write-Host " 📥 Fetching PR #$PRNumber..." -ForegroundColor Cyan + $tempBranch = "temp-pr-$PRNumber" + + # Clean up any leftover temp branch + git branch -D $tempBranch 2>$null | Out-Null + + # Try fetching from origin (same-repo PRs) + git fetch origin "pull/$PRNumber/head:$tempBranch" 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + # Fork PR — get fork info + Write-Host " 📥 Fetching from fork..." -ForegroundColor Cyan + $forkInfo = gh pr view $PRNumber --json headRepositoryOwner,headRefName,headRepository 2>$null | ConvertFrom-Json + if (-not $forkInfo -or -not $forkInfo.headRepositoryOwner) { + Write-Error "Failed to fetch PR #$PRNumber (not found on origin or fork)" + git checkout $originalBranch 2>$null + exit 1 + } + $forkUrl = "https://github.com/$($forkInfo.headRepositoryOwner.login)/$($forkInfo.headRepository.name).git" + $fetchOutput = git fetch $forkUrl "$($forkInfo.headRefName):$tempBranch" 2>&1 if ($LASTEXITCODE -ne 0) { - Write-Host "" - Write-Host "⚠️ Merge conflict detected!" -ForegroundColor Red - Write-Host " Please resolve conflicts manually and re-run the script with -SkipMerge" -ForegroundColor Yellow - git merge --abort 2>$null - git branch -D "temp-pr-$PRNumber" 2>$null + Write-Error "Failed to fetch from fork: $forkUrl`n$fetchOutput" + git checkout $originalBranch 2>$null exit 1 } - - # Clean up temp branch - git branch -D "temp-pr-$PRNumber" 2>$null - - Write-Host " ✅ PR #$PRNumber merged into '$currentBranch'" -ForegroundColor Green } -} else { - Write-Host "" - Write-Host "⏭️ Skipping merge (assuming PR is already merged)" -ForegroundColor Yellow -} -# Step 3: Ensure state directory exists -$stateDir = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState" -if (-not (Test-Path $stateDir)) { - New-Item -ItemType Directory -Path $stateDir -Force | Out-Null - Write-Host " 📁 Created state directory: $stateDir" -ForegroundColor Gray -} + # ── Merge PR commits (squash) ── + Write-Host " 🔀 Merging PR commits (squashed)..." -ForegroundColor Cyan + git merge --squash $tempBranch 2>&1 | Out-Null + if ($LASTEXITCODE -eq 0) { + # Check if there's anything to commit (PR might already be merged) + $staged = git diff --cached --quiet 2>$null; $hasStagedChanges = $LASTEXITCODE -ne 0 + if ($hasStagedChanges) { + git commit -m "PR #$PRNumber squashed for review" 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + git branch -D $tempBranch 2>$null + Write-Error "Failed to create squashed commit"; exit 1 + } + Write-Host " ✅ Squash-merge succeeded" -ForegroundColor Green + } else { + Write-Host " ⚠️ No changes to merge (PR may already be up to date)" -ForegroundColor Yellow + } + } else { + Write-Host " ❌ Squash-merge had conflicts." -ForegroundColor Red + git merge --abort 2>$null + git reset --hard HEAD 2>$null + + # Clean up branches + git checkout $originalBranch 2>$null + git branch -D $reviewBranch 2>$null + git branch -D $tempBranch 2>$null + + # Post a comment on the PR about merge conflicts + $conflictBody = "⚠️ **Merge Conflict Detected** — This PR has merge conflicts with its target branch. Please rebase onto the target branch and resolve the conflicts." + try { + gh pr comment $PRNumber --body $conflictBody 2>&1 | Out-Null + Write-Host " 📝 Posted merge conflict comment on PR" -ForegroundColor Cyan + } catch { + Write-Host " ⚠️ Could not post merge conflict comment (non-fatal): $_" -ForegroundColor Yellow + } -# Create PRAgent phase directories for structured output -$PRAgentDir = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent" -$phaseSubdirs = @("pre-flight", "gate", "try-fix", "report") -foreach ($subdir in $phaseSubdirs) { - $dirPath = Join-Path $PRAgentDir $subdir - if (-not (Test-Path $dirPath)) { - New-Item -ItemType Directory -Path $dirPath -Force | Out-Null + Write-Error "Merge conflicts for PR #$PRNumber. Review cannot proceed until conflicts are resolved." + exit 1 } -} -Write-Host " 📁 Created PRAgent phase directories: $PRAgentDir" -ForegroundColor Gray - -# Save the branch name and commit SHA BEFORE launching the agent. -# The Copilot agent may change HEAD (e.g., via git checkout, gh pr checkout, git reset) -# despite prompt instructions forbidding it. Using a pinned SHA for restoration -# ensures we always recover to the correct state regardless of what the agent did. -$savedBranch = git branch --show-current -$savedHead = git rev-parse HEAD -Write-Host " 📌 Pinned restore point: $savedBranch @ $($savedHead.Substring(0, 10))" -ForegroundColor Gray -# Step 4: Build the prompt for Copilot CLI -$planTemplatePath = ".github/agents/pr/PLAN-TEMPLATE.md" + # Clean up temp branch + git branch -D $tempBranch 2>$null | Out-Null -# Build platform instruction -$platformInstruction = if ($Platform) { - "**Platform for testing:** $Platform" -} else { - "**Platform for testing:** Determine the appropriate platform(s) based on the PR's affected code paths and the current host OS." + # Verify + $headCommit = git log --oneline -1 2>$null + Write-Host " ✅ Review branch ready: $reviewBranch" -ForegroundColor Green + Write-Host " 📝 HEAD: $headCommit" -ForegroundColor Gray } -$prompt = @" -Review PR #$PRNumber using the pr agent workflow. +# ─── Helper: Invoke Copilot ────────────────────────────────────────────────── +function Invoke-CopilotStep { + param([string]$StepName, [string]$Prompt) -$platformInstruction - -🚨 **CRITICAL - NEVER MODIFY GIT STATE:** -- NEVER run ``git checkout``, ``git switch``, ``git fetch``, ``git stash``, or ``git reset`` -- NEVER run ``git push`` - you do NOT have permission to push anything -- You are ALWAYS on the correct branch already - the script handles this -- If you think you need to switch branches or push changes, you are WRONG - ask the user instead - -🚨 **CRITICAL - AUTONOMOUS EXECUTION:** -- There is NO human operator to respond to questions -- NEVER stop and ask the user for input - nobody will respond -- NEVER present a list of options and wait for a choice -- When you encounter an environment blocker, use your best judgment to choose the best path forward: - - If a phase is blocked (e.g., try-fix builds fail), SKIP that phase and proceed to the next one - - If Gate passes but Fix phase is blocked, proceed directly to Report phase with Gate results only - - If a specific platform fails, try an alternative platform ONCE, then skip if still blocked - - Always prefer CONTINUING with partial results over STOPPING completely -- Document what was skipped and why in your report, but keep moving forward - -📁 **PHASE OUTPUT ARTIFACTS (MANDATORY):** -- After completing EACH phase, write a ``content.md`` file to the phase output directory -- Pre-Flight: ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pre-flight/content.md`` -- Gate: ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/gate/content.md`` -- Fix: ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/try-fix/content.md`` -- Report: ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/report/content.md`` -- These files are read by the comment scripts to build the PR comment -- See SHARED-RULES.md "Phase Output Artifacts" section for content format - -**Instructions:** -1. Read the plan template at ``$planTemplatePath`` for the 4-phase workflow -2. Read ``.github/agents/pr.md`` for Phases 1-2 instructions -3. Follow ALL critical rules, especially: - - On environment blockers: skip the blocked phase and continue autonomously (see AUTONOMOUS EXECUTION above) - - Use task agent for Gate verification - - Run multi-model try-fix in Phase 3 - -**Start with Phase 1: Pre-Flight** -- Gather context from PR #$PRNumber -- Proceed through all 4 phases - -Begin the review now. -"@ - -Write-Host "" -Write-Host "═══════════════════════════════════════════════════════════" -ForegroundColor DarkGray -Write-Host "" - -if ($DryRun) { - Write-Host "[DRY RUN] Would invoke Copilot CLI with:" -ForegroundColor Magenta - Write-Host "" - Write-Host " Agent: pr" -ForegroundColor Gray - Write-Host " Mode: $(if ($Interactive) { 'Interactive (-i)' } else { 'Non-interactive (-p)' })" -ForegroundColor Gray - Write-Host " PR: #$PRNumber" -ForegroundColor Gray - Write-Host " Platform: $(if ($Platform) { $Platform } else { '(agent will determine)' })" -ForegroundColor Gray Write-Host "" - Write-Host "Prompt:" -ForegroundColor Gray - Write-Host $prompt -ForegroundColor DarkGray - Write-Host "" - Write-Host "To run for real, remove the -DryRun flag" -ForegroundColor Yellow -} else { - Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Green - Write-Host "║ PHASE 1: PR AGENT REVIEW ║" -ForegroundColor Green - Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Green - Write-Host "" - Write-Host "PR Review Context:" -ForegroundColor Cyan - Write-Host " PR_NUMBER: $PRNumber" -ForegroundColor White - Write-Host " PLATFORM: $(if ($Platform) { $Platform } else { '(agent will determine)' })" -ForegroundColor White - Write-Host " PLAN_TEMPLATE: $planTemplatePath" -ForegroundColor White - Write-Host " CURRENT_BRANCH: $(git branch --show-current)" -ForegroundColor White - Write-Host " PR_TITLE: $($prInfo.title)" -ForegroundColor White - Write-Host " MODE: $(if ($Interactive) { 'Interactive' } else { 'Non-interactive' })" -ForegroundColor White - Write-Host "" - Write-Host "Workflow:" -ForegroundColor Cyan - Write-Host " 1. PR Agent Review (this phase)" -ForegroundColor White - if ($RunFinalize) { - Write-Host " 2. pr-finalize skill (queued)" -ForegroundColor White - } - if ($PostSummaryComment) { - $phase3Label = "3. Post comments: agent summary" - if ($RunFinalize) { - $phase3Label += " + finalize" - } - Write-Host " $phase3Label (queued)" -ForegroundColor White - } - Write-Host "" - Write-Host "─────────────────────────────────────────────────────────────" -ForegroundColor DarkGray - Write-Host "" - - # Build the copilot command arguments - $copilotArgs = @( - "--agent", "pr", - "--stream", "on" # Enable streaming for real-time output - ) - - # NOTE: --deny-tool does NOT work with --allow-all (allow-all takes precedence) - # Branch switching prevention relies on agent instructions in pr.md only - - # Create log directory for this PR - $prLogDir = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/copilot-logs" - if (-not (Test-Path $prLogDir)) { - New-Item -ItemType Directory -Path $prLogDir -Force | Out-Null - } - - # Add logging options - $copilotArgs += @("--log-dir", $prLogDir, "--log-level", "info") - - if ($Interactive) { - # Interactive mode: -i to start with prompt - $copilotArgs += @("-i", $prompt) - } else { - # Non-interactive mode (default): -p with --allow-all - # Also save session to markdown for review - $sessionFile = Join-Path $prLogDir "session-$(Get-Date -Format 'yyyyMMdd-HHmmss').md" - $copilotArgs += @("-p", $prompt, "--allow-all", "--share", $sessionFile) + Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Magenta + Write-Host "║ $($StepName.PadRight(55))║" -ForegroundColor Magenta + Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Magenta + + if ($DryRun) { + Write-Host "[DRY RUN] Prompt:" -ForegroundColor Magenta + Write-Host $Prompt -ForegroundColor Gray + return 0 } - - Write-Host "🚀 Starting Copilot CLI..." -ForegroundColor Yellow - Write-Host "" - - # Invoke Copilot CLI - & copilot @copilotArgs - - $exitCode = $LASTEXITCODE - - Write-Host "" - Write-Host "═══════════════════════════════════════════════════════════" -ForegroundColor DarkGray - if ($exitCode -eq 0) { - Write-Host "✅ Copilot CLI completed successfully" -ForegroundColor Green - } else { - Write-Host "⚠️ Copilot CLI exited with code: $exitCode" -ForegroundColor Yellow + + $stopwatch = [System.Diagnostics.Stopwatch]::StartNew() + $toolCount = 0 + $turnCount = 0 + $currentIntent = "" + $modelName = "" + $failedTools = @() + + # Tool icon mapping for common tools + $toolIcons = @{ + 'bash' = '🖥️'; 'view' = '📄'; 'edit' = '✏️'; 'create' = '📝' + 'grep' = '🔍'; 'glob' = '📂'; 'task' = '🤖'; 'skill' = '⚡' + 'sql' = '🗃️'; 'web_search' = '🌐'; 'ask_user' = '❓' + 'github-mcp-server-pull_request_read' = '🔀' + 'github-mcp-server-issue_read' = '🐛' + 'github-mcp-server-get_file_contents' = '📦' + 'github-mcp-server-search_code' = '🔎' } - - # Post-completion skills (only run if main agent completed successfully) - if ($exitCode -eq 0) { - - # Restore tracked files to clean state before running post-completion skills. - # Phase 1 (PR Agent) may have left the working tree dirty from try-fix attempts, - # or even switched branches despite instructions. We use the pinned SHA ($savedHead) - # instead of HEAD to guarantee restoration to the correct commit. - # NOTE: CustomAgentLogsTmp/ is .gitignore'd and untracked, so this won't touch them. - Write-Host "" - Write-Host "🧹 Restoring working tree to clean state between phases..." -ForegroundColor Yellow - git status --porcelain 2>$null | Set-Content "CustomAgentLogsTmp/PRState/phase1-exit-git-status.log" -ErrorAction SilentlyContinue - - # Check if the agent moved HEAD or switched branches - $postAgentBranch = git branch --show-current - $postAgentHead = git rev-parse HEAD - if ($postAgentBranch -ne $savedBranch -or $postAgentHead -ne $savedHead) { - Write-Host " ⚠️ Agent changed git state! Branch: $postAgentBranch (expected: $savedBranch), HEAD: $($postAgentHead.Substring(0, 10)) (expected: $($savedHead.Substring(0, 10)))" -ForegroundColor Red - Write-Host " 🔄 Recovering to pinned restore point..." -ForegroundColor Yellow - git checkout $savedBranch 2>&1 | Out-Null - git reset --hard $savedHead 2>&1 | Out-Null - } else { - git checkout $savedHead -- . 2>&1 | Out-Null - } - Write-Host " ✅ Working tree restored (pinned SHA: $($savedHead.Substring(0, 10)))" -ForegroundColor Green - - # Phase 2: Run pr-finalize skill if requested - if ($RunFinalize) { - Write-Host "" - Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Magenta - Write-Host "║ PHASE 2: PR-FINALIZE SKILL ║" -ForegroundColor Magenta - Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Magenta - Write-Host "" - - # Ensure output directory exists for finalize results - $finalizeDir = Join-Path $RepoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize" - if (-not (Test-Path $finalizeDir)) { - New-Item -ItemType Directory -Path $finalizeDir -Force | Out-Null - } - - $finalizePrompt = "Run the pr-finalize skill for PR #$PRNumber. Verify the PR title and description match the actual implementation. Do NOT post a comment. Write your findings to CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/pr-finalize-summary.md. If you recommend a new description, also write it to CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/recommended-description.md. If you have code review findings, also write them to CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/code-review.md." - - $finalizeArgs = @( - "-p", $finalizePrompt, - "--allow-all", - "--stream", "on" - ) - - Write-Host "🔍 Running pr-finalize..." -ForegroundColor Yellow - & copilot @finalizeArgs - - $finalizeExit = $LASTEXITCODE - if ($finalizeExit -eq 0) { - Write-Host "✅ pr-finalize completed" -ForegroundColor Green - } else { - Write-Host "⚠️ pr-finalize exited with code: $finalizeExit" -ForegroundColor Yellow - } - } - - # Phase 3: Post comments if requested - # Runs scripts directly instead of via Copilot CLI to avoid: - # - LLM creating its own broken version if skill files are missing - # - Dirty tree from Phase 2 corrupting script files - if ($PostSummaryComment) { - Write-Host "" - Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Magenta - Write-Host "║ PHASE 3: POST COMMENTS ║" -ForegroundColor Magenta - Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Magenta - Write-Host "" - - # Restore tracked files (including deleted ones) to clean state using pinned SHA. - Write-Host "🧹 Restoring working tree to clean state..." -ForegroundColor Yellow - git status --porcelain 2>$null | Set-Content "CustomAgentLogsTmp/PRState/phase2-exit-git-status.log" -ErrorAction SilentlyContinue - - # Check if Phase 2 (pr-finalize) moved HEAD or switched branches - $postPhase2Branch = git branch --show-current - $postPhase2Head = git rev-parse HEAD - if ($postPhase2Branch -ne $savedBranch -or $postPhase2Head -ne $savedHead) { - Write-Host " ⚠️ Phase 2 changed git state! Recovering to pinned restore point..." -ForegroundColor Red - git checkout $savedBranch 2>&1 | Out-Null - git reset --hard $savedHead 2>&1 | Out-Null - } else { - git checkout $savedHead -- . 2>&1 | Out-Null - } - Write-Host " ✅ Working tree restored (pinned SHA: $($savedHead.Substring(0, 10)))" -ForegroundColor Green - - # 3a: Post PR agent summary comment - $scriptPath = ".github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1" - if (-not (Test-Path $scriptPath)) { - Write-Host "⚠️ Script missing after restore, attempting targeted recovery..." -ForegroundColor Yellow - git checkout $savedHead -- $scriptPath 2>&1 | Out-Null - } - if (Test-Path $scriptPath) { - Write-Host "💬 Running post-ai-summary-comment.ps1 directly..." -ForegroundColor Yellow - & $scriptPath -PRNumber $PRNumber - - $commentExit = $LASTEXITCODE - if ($commentExit -eq 0) { - Write-Host "✅ Agent summary comment posted" -ForegroundColor Green - } else { - Write-Host "⚠️ post-ai-summary-comment.ps1 exited with code: $commentExit" -ForegroundColor Yellow - } - } else { - Write-Host "⚠️ Script not found at: $scriptPath" -ForegroundColor Yellow - Write-Host " Current directory: $(Get-Location)" -ForegroundColor Gray - Write-Host " Skipping agent summary comment." -ForegroundColor Gray - } - - # 3b: Post PR finalize comment (from Phase 2 finalize results) - if ($RunFinalize) { - $finalizeScriptPath = ".github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1" - if (-not (Test-Path $finalizeScriptPath)) { - Write-Host "⚠️ Finalize script missing, attempting targeted recovery..." -ForegroundColor Yellow - git checkout $savedHead -- $finalizeScriptPath 2>&1 | Out-Null + + # Use JSON output format to stream live progress of agent activity. + & copilot -p $Prompt --allow-all --output-format json 2>&1 | ForEach-Object { + $line = $_.ToString() + try { + $event = $line | ConvertFrom-Json -ErrorAction Stop + switch ($event.type) { + 'session.tools_updated' { + if ($event.data.model) { + $modelName = $event.data.model + Write-Host " ⚙️ Model: " -ForegroundColor DarkGray -NoNewline + Write-Host $modelName -ForegroundColor DarkCyan + } } - if (Test-Path $finalizeScriptPath) { - Write-Host "💬 Running post-pr-finalize-comment.ps1 directly (unified mode)..." -ForegroundColor Yellow - & $finalizeScriptPath -PRNumber $PRNumber -Unified - - $finalizeCommentExit = $LASTEXITCODE - if ($finalizeCommentExit -eq 0) { - Write-Host "✅ Finalize comment posted" -ForegroundColor Green + 'assistant.turn_start' { + $turnCount++ + $elapsed = $stopwatch.Elapsed.ToString("mm\:ss") + Write-Host "" + Write-Host " ┌─ Turn $turnCount " -ForegroundColor DarkGray -NoNewline + Write-Host "[$elapsed]" -ForegroundColor DarkYellow -NoNewline + if ($currentIntent) { + Write-Host " · $currentIntent" -ForegroundColor DarkCyan } else { - Write-Host "⚠️ post-pr-finalize-comment.ps1 exited with code: $finalizeCommentExit" -ForegroundColor Yellow + Write-Host "" } - } else { - Write-Host "⚠️ Script not found at: $finalizeScriptPath" -ForegroundColor Yellow - Write-Host " Skipping finalize comment." -ForegroundColor Gray } - } - } + 'assistant.turn_end' { + Write-Host " └─" -ForegroundColor DarkGray + } + 'tool.execution_start' { + $toolName = $event.data.toolName + $args_ = $event.data.arguments + + # Capture intent changes silently + if ($toolName -eq 'report_intent') { + $currentIntent = $args_.intent ?? $currentIntent + Write-Host " │ 🎯 " -ForegroundColor DarkGray -NoNewline + Write-Host $currentIntent -ForegroundColor Yellow + break + } - # Phase 4: Apply Labels - Write-Host "" - Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Blue - Write-Host "║ PHASE 4: APPLY LABELS ║" -ForegroundColor Blue - Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Blue - Write-Host "" - - $labelHelperPath = Join-Path $RepoRoot ".github/scripts/shared/Update-AgentLabels.ps1" - if (-not (Test-Path $labelHelperPath)) { - Write-Host "⚠️ Label helper missing, attempting targeted recovery..." -ForegroundColor Yellow - git checkout $savedHead -- $labelHelperPath 2>&1 | Out-Null - } + $toolCount++ + $icon = $toolIcons[$toolName] + if (-not $icon) { + # Prefix match for github-mcp-server-* and other compound names + $icon = if ($toolName -like 'github-*') { '🔀' } else { '🔧' } + } + + # Build a short display name for long tool names + $displayName = $toolName -replace '^github-mcp-server-', 'gh/' + + # Pick the most useful detail from arguments + $detail = $args_.description ?? $args_.intent ?? '' + if (-not $detail) { + # Fallback: pick first informative arg + $detail = $args_.command ?? $args_.pattern ?? $args_.query ?? $args_.path ?? $args_.prompt ?? '' + } + if ($detail) { + $detail = $detail.Substring(0, [Math]::Min($detail.Length, 90)) + # Truncate at last word boundary if we cut mid-word + if ($detail.Length -eq 90) { + $lastSpace = $detail.LastIndexOf(' ') + if ($lastSpace -gt 60) { $detail = $detail.Substring(0, $lastSpace) + "…" } + else { $detail += "…" } + } + } - if (Test-Path $labelHelperPath) { - try { - . $labelHelperPath - Apply-AgentLabels -PRNumber $PRNumber -RepoRoot $RepoRoot + Write-Host " │ $icon " -ForegroundColor DarkGray -NoNewline + Write-Host $displayName -ForegroundColor Cyan -NoNewline + if ($detail) { + Write-Host " $detail" -ForegroundColor DarkGray + } else { + Write-Host "" + } + } + 'tool.execution_complete' { + if (-not $event.data.success) { + $failedTool = $event.data.toolCallId + $failedTools += $failedTool + Write-Host " │ ❌ Tool failed" -ForegroundColor Red + } + } + 'assistant.message' { + $content = $event.data.content + # Show agent text responses (skip empty tool-request-only messages) + if ($content -and $content.Trim()) { + $preview = $content.Trim() + if ($preview.Length -gt 400) { + $preview = $preview.Substring(0, 400) + "…" + } + Write-Host " │ 💬 " -ForegroundColor DarkGray -NoNewline + Write-Host $preview -ForegroundColor White + } + } + 'result' { + # Final stats + $usage = $event.data.usage + if ($usage) { + $elapsed = $stopwatch.Elapsed.ToString("mm\:ss") + $apiMs = if ($usage.totalApiDurationMs) { [math]::Round($usage.totalApiDurationMs / 1000, 1) } else { "?" } + $changes = $usage.codeChanges + $filesChanged = if ($changes -and $changes.filesModified) { $changes.filesModified.Count } else { 0 } + $linesAdded = if ($changes) { $changes.linesAdded } else { 0 } + $linesRemoved = if ($changes) { $changes.linesRemoved } else { 0 } + + Write-Host "" + Write-Host " ╭──────────────────────────────────────────╮" -ForegroundColor DarkGray + Write-Host " │ ⏱ $elapsed elapsed ($($apiMs)s API)" -ForegroundColor DarkGray -NoNewline + Write-Host " │ 🔧 $toolCount tools" -ForegroundColor DarkGray -NoNewline + Write-Host " │ 🔄 $turnCount turns" -ForegroundColor DarkGray + if ($filesChanged -gt 0 -or $linesAdded -gt 0 -or $linesRemoved -gt 0) { + Write-Host " │ 📝 $filesChanged files " -ForegroundColor DarkGray -NoNewline + Write-Host "+$linesAdded" -ForegroundColor Green -NoNewline + Write-Host "/" -ForegroundColor DarkGray -NoNewline + Write-Host "-$linesRemoved" -ForegroundColor Red + } + Write-Host " ╰──────────────────────────────────────────╯" -ForegroundColor DarkGray + } + } } - catch { - Write-Host "⚠️ Label application failed (non-fatal): $_" -ForegroundColor Yellow + } catch { + # Non-JSON line (e.g. stats) — pass through as-is + if ($line.Trim()) { + Write-Host " $line" -ForegroundColor DarkGray } - } else { - Write-Host "⚠️ Label helper not found at: $labelHelperPath — skipping labels" -ForegroundColor Yellow } } + $exitCode = $LASTEXITCODE + $stopwatch.Stop() + + if ($exitCode -eq 0) { + Write-Host " ✅ $StepName completed" -ForegroundColor Green + } else { + Write-Host " ⚠️ $StepName exited with code: $exitCode" -ForegroundColor Yellow + } + if ($failedTools.Count -gt 0) { + Write-Host " ⚠️ $($failedTools.Count) tool(s) failed during execution" -ForegroundColor Yellow + } + return $exitCode } +# ═════════════════════════════════════════════════════════════════════════════ +# STEP 1: PR Review (4-phase skill) +# ═════════════════════════════════════════════════════════════════════════════ + +$step1Prompt = @" +Use a skill to review PR #$PRNumber. + +$platformInstruction +$autonomousRules + +📁 Write phase output to ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/{phase}/content.md`` +"@ + +Invoke-CopilotStep -StepName "STEP 1: PR REVIEW" -Prompt $step1Prompt | Out-Null + +# Restore review branch — the Copilot agent may have switched branches (e.g. via gh pr checkout) +git checkout $reviewBranch 2>$null | Out-Null + +# ═════════════════════════════════════════════════════════════════════════════ +# STEP 2: PR Finalize +# ═════════════════════════════════════════════════════════════════════════════ + +$step2Prompt = @" +Use a skill to finalize PR #$PRNumber. Write findings to ``CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/pr-finalize-summary.md``. +$autonomousRules +"@ + +Invoke-CopilotStep -StepName "STEP 2: PR FINALIZE" -Prompt $step2Prompt | Out-Null + +# Restore review branch — the Copilot agent may have switched branches (e.g. via gh pr checkout) +git checkout $reviewBranch 2>$null | Out-Null + +# ═════════════════════════════════════════════════════════════════════════════ +# STEP 3: Post AI Summary Comment (direct script invocation) +# ═════════════════════════════════════════════════════════════════════════════ + Write-Host "" -Write-Host "📋 Plan template: $planTemplatePath" -ForegroundColor Gray -if (-not $DryRun) { - Write-Host "📁 Copilot logs: CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/copilot-logs/" -ForegroundColor Gray - if (-not $Interactive) { - Write-Host "📄 Session markdown: $sessionFile" -ForegroundColor Gray +Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Magenta +Write-Host "║ STEP 3: POST AI SUMMARY ║" -ForegroundColor Magenta +Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Magenta + +$summaryScriptsDir = Join-Path $RepoRoot ".github/scripts" +$dryRunFlag = if ($DryRun) { @('-DryRun') } else { @() } + +# 3a: Post PR review phases (pre-flight, gate, try-fix, report) +$reviewScript = Join-Path $summaryScriptsDir "post-ai-summary-comment.ps1" +if (Test-Path $reviewScript) { + try { + Write-Host " 📝 Posting PR review summary..." -ForegroundColor Cyan + & $reviewScript -PRNumber $PRNumber @dryRunFlag + Write-Host " ✅ PR review summary posted" -ForegroundColor Green + } catch { + Write-Host " ⚠️ PR review summary posting failed (non-fatal): $_" -ForegroundColor Yellow } +} else { + Write-Host " ⚠️ post-ai-summary-comment.ps1 not found — skipping review summary" -ForegroundColor Yellow } + +# 3b: Post PR finalize section (title, description, code review) +$finalizeScript = Join-Path $summaryScriptsDir "post-pr-finalize-comment.ps1" +if (Test-Path $finalizeScript) { + try { + Write-Host " 📝 Posting PR finalize summary..." -ForegroundColor Cyan + & $finalizeScript -PRNumber $PRNumber @dryRunFlag + Write-Host " ✅ PR finalize summary posted" -ForegroundColor Green + } catch { + Write-Host " ⚠️ PR finalize summary posting failed (non-fatal): $_" -ForegroundColor Yellow + } +} else { + Write-Host " ⚠️ post-pr-finalize-comment.ps1 not found — skipping finalize summary" -ForegroundColor Yellow +} + +# ═════════════════════════════════════════════════════════════════════════════ +# STEP 4: Apply Labels +# ═════════════════════════════════════════════════════════════════════════════ + Write-Host "" +Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Blue +Write-Host "║ STEP 4: APPLY LABELS ║" -ForegroundColor Blue +Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Blue + +$labelHelperPath = Join-Path $RepoRoot ".github/scripts/shared/Update-AgentLabels.ps1" +if (Test-Path $labelHelperPath) { + try { + . $labelHelperPath + Apply-AgentLabels -PRNumber $PRNumber -RepoRoot $RepoRoot + Write-Host " ✅ Labels applied" -ForegroundColor Green + } catch { + Write-Host " ⚠️ Label application failed (non-fatal): $_" -ForegroundColor Yellow + } +} else { + Write-Host " ⚠️ Label helper not found — skipping" -ForegroundColor Yellow +} + +# ═════════════════════════════════════════════════════════════════════════════ +# Cleanup +# ═════════════════════════════════════════════════════════════════════════════ -# NOTE: This cleanup targets CI/ADO agent environments where only this script's -# Copilot CLI processes should exist. On developer machines, this could potentially -# affect other Copilot processes (e.g., VS Code extension). The risk is low since -# this runs at script end, but be aware if running locally. -# Clean up orphaned copilot CLI processes that may hold stdout fd open -# IMPORTANT: Only target processes whose command line contains "copilot" to avoid -# accidentally terminating the ADO agent's own node process +Write-Host "" Write-Host "🧹 Cleaning up child processes..." -ForegroundColor Gray try { - $myPid = $PID - # Find node processes running copilot CLI (not the ADO agent's node process) $orphans = Get-Process -Name "node" -ErrorAction SilentlyContinue | Where-Object { - $_.Id -ne $myPid -and - (($_.Path -and $_.Path -match "copilot") -or - ($_.CommandLine -and $_.CommandLine -match "copilot")) + ($_.Path -and $_.Path -match "copilot") -or + ($_.CommandLine -and $_.CommandLine -match "copilot") } - # Also get any process literally named "copilot" $copilotProcs = Get-Process -Name "copilot" -ErrorAction SilentlyContinue $allOrphans = @($orphans) + @($copilotProcs) | Where-Object { $_ -ne $null } | Sort-Object Id -Unique if ($allOrphans.Count -gt 0) { - Write-Host " Stopping $($allOrphans.Count) orphaned process(es): $($allOrphans | ForEach-Object { "$($_.ProcessName)($($_.Id))" } | Join-String -Separator ', ')" -ForegroundColor Gray + Write-Host " Stopping $($allOrphans.Count) orphaned process(es)" -ForegroundColor Gray $allOrphans | Stop-Process -Force -ErrorAction SilentlyContinue - } else { - Write-Host " No orphaned copilot processes found" -ForegroundColor Gray } } catch { Write-Host " ⚠️ Cleanup warning: $_" -ForegroundColor Yellow } -if ($LogFile) { - Stop-Transcript | Out-Null -} +Write-Host "" +Write-Host "✅ Review complete for PR #$PRNumber" -ForegroundColor Green +Write-Host "📁 Output: CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/" -ForegroundColor Gray +Write-Host "" + +if ($LogFile) { Stop-Transcript | Out-Null } diff --git a/.github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 b/.github/scripts/post-ai-summary-comment.ps1 similarity index 96% rename from .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 rename to .github/scripts/post-ai-summary-comment.ps1 index 9c1353794cbd..f10e6b022c57 100644 --- a/.github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 +++ b/.github/scripts/post-ai-summary-comment.ps1 @@ -1,7 +1,7 @@ #!/usr/bin/env pwsh <# .SYNOPSIS - Posts or updates the PR agent review comment on a GitHub Pull Request with validation. + Posts or updates the agent review comment on a GitHub Pull Request with validation. .DESCRIPTION Creates ONE comment for the entire PR review with all phases wrapped in an expandable section. @@ -522,7 +522,7 @@ $latestCommitTitle = ($commitJson.message -split "`n")[0] $latestCommitSha = $commitJson.sha.Substring(0, 7) $latestCommitUrl = "https://github.com/dotnet/maui/commit/$($commitJson.sha)" -# Helper function to create a NEW review session +# Helper function to create content for a review session (no wrapper
) function New-ReviewSession { param([string]$PhaseContent, [string]$CommitTitle, [string]$CommitSha, [string]$CommitUrl) @@ -530,16 +530,8 @@ function New-ReviewSession { return "" } - return @" -
-📝 Review Session — $CommitTitle · $CommitSha - ---- - -$PhaseContent - -
-"@ + # Return raw content — the commit info is shown on the top-level summary + return $PhaseContent } # Helper function to extract existing review sessions from a phase @@ -551,11 +543,25 @@ function Get-ExistingReviewSessions { } $sessions = @() + # Try old format first (wrapped in
📝 ...) $pattern = '(?s)
\s*📝.*?.*?
' $matches = [regex]::Matches($PhaseContent, $pattern) - foreach ($match in $matches) { - $sessions += $match.Value + if ($matches.Count -gt 0) { + # Old format: extract the inner content from each session wrapper + foreach ($match in $matches) { + $inner = $match.Value + if ($inner -match '(?s)📝.*?\s*---\s*(.*?)\s*
') { + $sessions += ($Matches[1].Trim() -replace '(?m)^---\s*$', '').Trim() + } else { + $sessions += $inner + } + } + } else { + # New format: content is directly in the phase section (no wrapper) + # Strip leading/trailing --- separators that may remain from old format + $cleaned = ($PhaseContent.Trim() -replace '(?m)^---\s*$', '').Trim() + $sessions += $cleaned } return $sessions @@ -728,7 +734,7 @@ $SECTION_END = "" $prReviewSection = @" $SECTION_START
-📊 Expand Full Review +📊 Expand Full Review — $latestCommitSha · $latestCommitTitle --- diff --git a/.github/scripts/post-pr-finalize-comment.ps1 b/.github/scripts/post-pr-finalize-comment.ps1 new file mode 100644 index 000000000000..7e80d48a6b27 --- /dev/null +++ b/.github/scripts/post-pr-finalize-comment.ps1 @@ -0,0 +1,199 @@ +#!/usr/bin/env pwsh +<# +.SYNOPSIS + Posts the PR finalize summary into the unified AI Summary comment on a GitHub PR. + +.DESCRIPTION + Reads the pr-finalize-summary.md file and injects its content directly into the + block of the unified AI Summary comment. + + No parsing or re-formatting — the summary file content is posted as-is inside + a collapsible details block. + + Auto-discovers the summary file from: + CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/pr-finalize/pr-finalize-summary.md + +.PARAMETER PRNumber + The PR number to post comment on (required) + +.PARAMETER SummaryFile + Path to pr-finalize-summary.md file. Auto-discovered from PRNumber if not provided. + +.PARAMETER DryRun + Print comment instead of posting + +.PARAMETER PreviewFile + Custom path for dry-run preview output + +.EXAMPLE + ./post-pr-finalize-comment.ps1 -PRNumber 34427 + +.EXAMPLE + ./post-pr-finalize-comment.ps1 -PRNumber 34427 -DryRun +#> + +param( + [Parameter(Mandatory=$false)] + [int]$PRNumber, + + [Parameter(Mandatory=$false)] + [string]$SummaryFile, + + [Parameter(Mandatory=$false)] + [switch]$DryRun, + + [Parameter(Mandatory=$false)] + [string]$PreviewFile +) + +$ErrorActionPreference = "Stop" + +Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ PR Finalize Comment (Post/Update) ║" -ForegroundColor Cyan +Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan + +# ============================================================================ +# AUTO-DISCOVERY +# ============================================================================ + +$repoRoot = git rev-parse --show-toplevel 2>$null + +if ($PRNumber -gt 0 -and [string]::IsNullOrWhiteSpace($SummaryFile)) { + $candidates = @( + "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/pr-finalize-summary.md" + ) + if ($repoRoot) { + $candidates += Join-Path $repoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/pr-finalize-summary.md" + } + foreach ($candidate in $candidates) { + if (Test-Path $candidate) { + $SummaryFile = $candidate + Write-Host "ℹ️ Auto-discovered summary file: $SummaryFile" -ForegroundColor Cyan + break + } + } +} + +# Extract PRNumber from path if not provided +if ($PRNumber -eq 0 -and -not [string]::IsNullOrWhiteSpace($SummaryFile) -and $SummaryFile -match '[/\\](\d+)[/\\]') { + $PRNumber = [int]$Matches[1] + Write-Host "ℹ️ Auto-detected PRNumber: $PRNumber from path" -ForegroundColor Cyan +} + +if ($PRNumber -eq 0) { + throw "PRNumber is required. Provide via -PRNumber or use -SummaryFile with path containing PR number." +} + +# ============================================================================ +# LOAD SUMMARY CONTENT +# ============================================================================ + +if ([string]::IsNullOrWhiteSpace($SummaryFile) -or -not (Test-Path $SummaryFile)) { + Write-Host "⚠️ No pr-finalize summary file found for PR #$PRNumber — nothing to post" -ForegroundColor Yellow + exit 0 +} + +$summaryContent = (Get-Content $SummaryFile -Raw -Encoding UTF8).Trim() +if ([string]::IsNullOrWhiteSpace($summaryContent)) { + Write-Host "⚠️ Summary file is empty — nothing to post" -ForegroundColor Yellow + exit 0 +} + +Write-Host " ✅ Loaded summary ($($summaryContent.Length) chars) from: $SummaryFile" -ForegroundColor Green + +# ============================================================================ +# BUILD SECTION +# ============================================================================ + +$MAIN_MARKER = "" +$SECTION_START = "" +$SECTION_END = "" + +$finalizeSection = @" +$SECTION_START +
+📋 Expand PR Finalization Review + +$summaryContent + +
+$SECTION_END +"@ + +# ============================================================================ +# POST / UPDATE +# ============================================================================ + +Write-Host "`nInjecting into AI Summary comment on #$PRNumber..." -ForegroundColor Yellow + +# Find existing unified comment +$existingUnifiedComment = $null +try { + $commentsJson = gh api "repos/dotnet/maui/issues/$PRNumber/comments?per_page=100" 2>$null + $comments = $commentsJson | ConvertFrom-Json + foreach ($comment in $comments) { + if ($comment.body -match [regex]::Escape($MAIN_MARKER)) { + $existingUnifiedComment = $comment + Write-Host "✓ Found unified AI Summary comment (ID: $($comment.id))" -ForegroundColor Green + break + } + } + if (-not $existingUnifiedComment) { + Write-Host "✓ No existing AI Summary comment found — will create new" -ForegroundColor Yellow + } +} catch { + Write-Host "⚠️ Could not fetch comments: $_" -ForegroundColor Yellow +} + +# Build the final comment body +if ($existingUnifiedComment) { + $body = $existingUnifiedComment.body + if ($body -match [regex]::Escape($SECTION_START)) { + $pattern = [regex]::Escape($SECTION_START) + "[\s\S]*?" + [regex]::Escape($SECTION_END) + $finalComment = $body -replace $pattern, $finalizeSection + } else { + $finalComment = $body.TrimEnd() + "`n`n" + $finalizeSection + } + $finalComment = $finalComment -replace "`n{4,}", "`n`n`n" +} else { + $finalComment = @" +$MAIN_MARKER + +## 🤖 AI Summary + +$finalizeSection +"@ +} + +# DryRun: preview to file +if ($DryRun) { + if ([string]::IsNullOrWhiteSpace($PreviewFile)) { + $PreviewFile = "CustomAgentLogsTmp/PRState/$PRNumber/ai-summary-comment-preview.md" + } + $previewDir = Split-Path $PreviewFile -Parent + if ($previewDir -and -not (Test-Path $previewDir)) { + New-Item -ItemType Directory -Path $previewDir -Force | Out-Null + } + Set-Content -Path $PreviewFile -Value "$($finalComment.TrimEnd())`n" -Encoding UTF8 -NoNewline + Write-Host "`n=== COMMENT PREVIEW ===" -ForegroundColor Yellow + Write-Host $finalComment + Write-Host "`n=== END PREVIEW ===" -ForegroundColor Yellow + Write-Host "`n✅ Preview saved to: $PreviewFile" -ForegroundColor Green + exit 0 +} + +# Post to GitHub +$tempFile = [System.IO.Path]::GetTempFileName() +@{ body = $finalComment } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 + +if ($existingUnifiedComment) { + Write-Host "Updating unified comment ID $($existingUnifiedComment.id)..." -ForegroundColor Yellow + $result = gh api --method PATCH "repos/dotnet/maui/issues/comments/$($existingUnifiedComment.id)" --input $tempFile --jq '.html_url' + Write-Host "✅ PR finalize section updated: $result" -ForegroundColor Green +} else { + Write-Host "Creating new unified comment on PR #$PRNumber..." -ForegroundColor Yellow + $result = gh api --method POST "repos/dotnet/maui/issues/$PRNumber/comments" --input $tempFile --jq '.html_url' + Write-Host "✅ Unified comment posted: $result" -ForegroundColor Green +} + +Remove-Item $tempFile diff --git a/.github/scripts/shared/Start-Emulator.ps1 b/.github/scripts/shared/Start-Emulator.ps1 index b1d7875326ba..33f9c8687a54 100644 --- a/.github/scripts/shared/Start-Emulator.ps1 +++ b/.github/scripts/shared/Start-Emulator.ps1 @@ -32,7 +32,10 @@ param( [string]$Platform, [Parameter(Mandatory=$false)] - [string]$DeviceUdid + [string]$DeviceUdid, + + [Parameter(Mandatory=$false)] + [switch]$Headless ) # Import shared utilities @@ -194,17 +197,26 @@ if ($Platform -eq "android") { # Redirect output to a log file for debugging $emulatorLog = Join-Path ([System.IO.Path]::GetTempPath()) "emulator-$selectedAvd.log" + # Use -no-window only when explicitly headless or running in CI + $useHeadless = $Headless -or $env:CI -or $env:TF_BUILD -or $env:GITHUB_ACTIONS + if ($IsWindows) { - Start-Process $emulatorBin -ArgumentList "-avd", $selectedAvd, "-no-snapshot-load", "-no-boot-anim", "-gpu", "swiftshader_indirect" -WindowStyle Hidden + $windowStyle = if ($useHeadless) { "Hidden" } else { "Normal" } + Start-Process $emulatorBin -ArgumentList "-avd", $selectedAvd, "-no-snapshot-load", "-no-boot-anim", "-gpu", "swiftshader_indirect" -WindowStyle $windowStyle } else { # macOS/Linux: Use nohup to detach from terminal # Use -no-snapshot (not -no-snapshot-load) to ensure clean emulator state for CI/testing. # This disables both snapshot load and save, so each boot is a cold boot. # Trade-off: slower boots, but guarantees no stale state between test runs. - $startScript = "nohup '$emulatorBin' -avd '$selectedAvd' -no-window -no-snapshot -no-audio -no-boot-anim -gpu swiftshader_indirect > '$emulatorLog' 2>&1 &" + $windowFlag = if ($useHeadless) { "-no-window" } else { "" } + $startScript = "nohup '$emulatorBin' -avd '$selectedAvd' $windowFlag -no-snapshot -no-audio -no-boot-anim -gpu swiftshader_indirect > '$emulatorLog' 2>&1 &" bash -c $startScript - Write-Info "Emulator started in background. Log file: $emulatorLog" + if ($useHeadless) { + Write-Info "Emulator started headless (no window). Log file: $emulatorLog" + } else { + Write-Info "Emulator started with window. Log file: $emulatorLog" + } } # Give the emulator process time to start diff --git a/.github/scripts/shared/Update-AgentLabels.ps1 b/.github/scripts/shared/Update-AgentLabels.ps1 index 818992fc00ff..7c2cd59c1d15 100644 --- a/.github/scripts/shared/Update-AgentLabels.ps1 +++ b/.github/scripts/shared/Update-AgentLabels.ps1 @@ -4,7 +4,7 @@ Shared functions for managing agent workflow labels on GitHub PRs. .DESCRIPTION - Provides idempotent label management for the PR agent review workflow. + Provides idempotent label management for the pr-review skill review workflow. Labels use the 's/agent-*' prefix convention for easy querying. Label categories: diff --git a/.github/skills/ai-summary-comment/IMPROVEMENTS.md b/.github/skills/ai-summary-comment/IMPROVEMENTS.md deleted file mode 100644 index 4c50133edaf9..000000000000 --- a/.github/skills/ai-summary-comment/IMPROVEMENTS.md +++ /dev/null @@ -1,640 +0,0 @@ -# PR Comment Script Improvements - -## Summary of Changes - -The `post-ai-summary-comment.ps1` script has been significantly improved to make posting PR review comments easier and more flexible using **dynamic section extraction**. - -## Key Improvements - -### 1. **Dynamic Section Extraction** (NEW!) - -**Before:** Script used hardcoded pattern matching with predefined title variations - -**After:** Script **automatically discovers ALL sections** from the provided content and extracts them dynamically - -```powershell -# Extracts ALL
TITLE sections -$allSections = Extract-AllSections -StateContent $Content - -# Then maps them to phases using flexible regex patterns -$preFlightContent = Get-SectionByPattern -Sections $allSections -Patterns @( - '📋.*Issue Summary', - '📋.*Pre-Flight' -) -``` - -**Benefits:** -- ✅ **No hardcoded titles** - works with ANY section header you use -- ✅ Automatically adapts - add new sections without modifying the script -- ✅ Better debugging - shows exactly which sections were found in the content -- ✅ More maintainable - less code, more flexible - -**Example debug output:** -``` -[DEBUG] Found 6 section(s) in content -[DEBUG] Section: '📋 Issue Summary' (803 chars) -[DEBUG] Section: '🚦 Gate - Test Verification' (488 chars) -[DEBUG] Section: '🔧 Fix Candidates' (868 chars) -[DEBUG] Section: '📋 Final Report' (2351 chars) -[DEBUG] Matched '📋 Final Report' with pattern '📋.*Report' -``` - ---- - -### 2. **Flexible Pattern Matching** - -**Before:** Exact string matching required - -**After:** Uses **regex patterns** to match section titles flexibly - -```powershell -# Matches any of these (and more!): -- "📋 Final Report" ✅ -- "📋 Phase 4: Final Report" ✅ -- "📋 Report - Final Recommendation" ✅ -- Any title containing "📋" and "Report" ✅ -``` - -**Pattern examples:** -- `'📋.*Issue Summary'` matches "📋 Issue Summary", "📋 Pre-Flight Issue Summary", etc. -- `'🚦.*Gate'` matches "🚦 Gate", "🚦 Phase 2: Gate", etc. -- `'📋.*Report'` matches any title with 📋 and Report in it - ---- - -### 3. **Errors vs Warnings** - -**Validation levels:** -- **Errors** (❌) - Block posting (missing content, PENDING markers) -- **Warnings** (⚠️) - Suggestions only (missing optional sections) - -**Example:** -``` -✅ All validation checks passed! - -⚠️ VALIDATION WARNINGS -Found 2 warning(s) (non-critical): - - Fix: Fix phase missing 'Exhausted' field (non-critical) - -💡 These are suggestions but won't block posting. -``` - ---- - -### 4. **Debug Mode** - -Enable detailed extraction information: - -```powershell -$DebugPreference = 'Continue' -./post-ai-summary-comment.ps1 -PRNumber 12345 -``` - -**Shows:** -- Which sections were found in the content -- How many characters each section contains -- Which patterns matched which sections -- Why validation passed or failed - ---- - -### 5. **Better Error Messages** - -**Comprehensive guidance when validation fails:** -``` -⛔ VALIDATION FAILED - -💡 Fix these issues in the content before posting. - Or use -SkipValidation to bypass these checks. - -🐛 Debug tip: Run with $DebugPreference = 'Continue' for details -``` - ---- - -## How Dynamic Extraction Works - -### Step 1: Extract ALL Sections - -```powershell -function Extract-AllSections { - # Pattern matches:
TITLE...content...
- $pattern = '(?s)
\s*([^<]+)(.*?)
' - $matches = [regex]::Matches($StateContent, $pattern) - - # Returns hashtable: @{ "Title" = "content", ... } -} -``` - -**Result:** Hashtable with ALL sections from your content - -### Step 2: Map to Phases - -```powershell -function Get-SectionByPattern { - # Try each pattern until one matches - foreach ($pattern in $Patterns) { - foreach ($key in $Sections.Keys) { - if ($key -match $pattern) { - return $Sections[$key] # Found it! - } - } - } -} -``` - -**Result:** Phase content matched by flexible regex patterns - ---- - -## Usage Examples - -### Basic Usage (unchanged) -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -``` - -### With Debug Mode (recommended when troubleshooting) -```powershell -pwsh -Command '$DebugPreference = "Continue"; ./.github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340' -``` - -### Skip Validation -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -SkipValidation -``` - -### Dry Run -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -DryRun -``` - ---- - -## What Section Headers Work Now? - -**The script uses regex patterns, so it's VERY flexible:** - -### Pre-Flight Phase -Any title matching `'📋.*Issue Summary'` or `'📋.*Pre-Flight'`: -- ✅ "📋 Issue Summary" (preferred) -- ✅ "📋 Pre-Flight Analysis" -- ✅ "📋 Context and Issue Summary" - -### Gate Phase -Any title matching `'🚦.*Gate'`: -- ✅ "🚦 Gate - Test Verification" -- ✅ "🚦 Gate" -- ✅ "🚦 Phase 2: Gate" - -### Fix Phase -Any title matching `'🔧.*Fix'`: -- ✅ "🔧 Fix Candidates" -- ✅ "🔧 Fix Analysis" -- ✅ "🔧 Fix" - -### Report Phase -Any title matching `'📋.*Report'` or `'Final Report'`: -- ✅ "📋 Final Report" -- ✅ "📋 Phase 4: Report" -- ✅ "📋 Report - Final Recommendation" -- ✅ "Final Report" - -**The beauty:** You don't need to remember exact titles anymore! - ---- - -## Migration Guide - -**No changes needed!** The script is backward compatible. - -**Old content** with exact headers like: -```markdown -📋 Phase 4: Report — Final Recommendation -``` - -**New content** with simpler headers like: -```markdown -📋 Final Report -``` - -**Both work!** The dynamic extraction finds them automatically. - ---- - -## Advantages Over Old Approach - -| Aspect | Old (Pattern Matching) | New (Dynamic Extraction) | -|--------|------------------------|--------------------------| -| **Flexibility** | ❌ Hardcoded titles | ✅ Any title works | -| **Maintenance** | ❌ Update code for new headers | ✅ No code changes needed | -| **Debugging** | ⚠️ Limited visibility | ✅ Full extraction visibility | -| **Speed** | ⚠️ Tries multiple patterns | ✅ Single pass extraction | -| **Reliability** | ⚠️ Can miss variations | ✅ Finds everything | - ---- - -## Common Issues & Solutions - -### Issue: "Phase X has NO content" - -**Step 1:** Enable debug mode to see what was found -```powershell -pwsh -Command '$DebugPreference = "Continue"; ./post-ai-summary-comment.ps1 -PRNumber XXXXX' -``` - -**Look for:** -``` -[DEBUG] Found 7 section(s) in content -[DEBUG] Section: 'Your Section Title' (XXX chars) -``` - -**Step 2:** Check if your section title matches the patterns - -Report phase patterns: `'📋.*Report'`, `'Final Report'` - -If your title is `"📋 Final Analysis"`, it won't match! - -**Solution:** Either: -- Rename section to include "Report": `"📋 Final Report"` ✅ -- Or use `-SkipValidation` if content is there - ---- - -### Issue: Section extracted but content is empty - -**Cause:** Content structure issue (missing content between tags) - -**Check your markdown:** -```markdown -
-📋 Final Report - - -Your report content... - -
-``` - -**Not this:** -```markdown -
-📋 Final Report -
❌ No content! -``` - ---- - -## Developer Notes - -### How to Add Support for New Phase Patterns - -Just add a regex pattern to the mapping: - -```powershell -$reportContent = Get-SectionByPattern -Sections $allSections -Patterns @( - '📋.*Report', - 'Final Report', - 'Your New Pattern Here' # Add here -) -Debug:$debugMode -``` - -**Example:** Support "Summary" as alias for "Report": -```powershell -$reportContent = Get-SectionByPattern -Sections $allSections -Patterns @( - '📋.*Report', - '📋.*Summary', # New pattern - 'Final Report' -) -``` - ---- - -### Regex Pattern Tips - -- `.*` matches any characters -- `^` matches start of string -- `$` matches end of string -- Use `[regex]::Escape()` if you need literal special chars - -**Examples:** -- `'🚦.*Gate'` - Title must contain both 🚦 and Gate -- `'^📋 Report'` - Title must START with "📋 Report" -- `'Report$'` - Title must END with "Report" - ---- - -## Testing - -Tested with: -- ✅ PR #27340 (6 sections extracted successfully) -- ✅ Debug mode showing section discovery -- ✅ Various header formats -- ✅ Dry run mode -- ✅ Skip validation mode -- ✅ Empty sections (proper error handling) - -**Debug output example:** -``` -[DEBUG] Found 6 section(s) in content -[DEBUG] Section: '📋 Issue Summary' (803 chars) -[DEBUG] Section: '📁 Files Changed' (0 chars) -[DEBUG] Section: '💬 PR Discussion Summary' (0 chars) -[DEBUG] Section: '🚦 Gate - Test Verification' (488 chars) -[DEBUG] Section: '🔧 Fix Candidates' (868 chars) -[DEBUG] Section: '📋 Final Report' (2351 chars) -[DEBUG] Matched '📋 Issue Summary' with pattern '📋.*Issue Summary' -[DEBUG] Matched '🚦 Gate - Test Verification' with pattern '🚦.*Gate' -[DEBUG] Matched '🔧 Fix Candidates' with pattern '🔧.*Fix' -[DEBUG] Matched '📋 Final Report' with pattern '📋.*Report' -``` - ---- - -## Future Improvements - -Potential enhancements: -- [ ] Auto-generate comment structure from discovered sections -- [ ] Support markdown headings (`##`/`###`) as alternative to `
` -- [ ] Validate section content structure (required fields) -- [ ] Suggest section renaming for better patterns -- [ ] Export sections as separate files for versioning - ---- - -## Feedback - -The dynamic extraction makes the script much more maintainable and flexible! - -If you find sections that aren't being extracted: -1. Run with `$DebugPreference = 'Continue'` to see what was found -2. Check which patterns are being used -3. Add a new pattern if needed (or rename your section) - ---- - -### 2. **Errors vs Warnings** - -**Before:** Everything was treated as a blocking error - -**After:** Two levels of feedback: -- **Errors** (❌) - Block posting (e.g., missing content, PENDING markers) -- **Warnings** (⚠️) - Suggestions only (e.g., missing optional sections) - -**Example output:** -``` -✅ All validation checks passed! - -⚠️ VALIDATION WARNINGS -Found 2 warning(s) (non-critical): - - Report: Report phase missing root cause analysis (non-critical) - - Fix: Fix phase missing 'Exhausted' field (non-critical) - -💡 These are suggestions for improvement but won't block posting. -``` - ---- - -### 3. **Debug Mode** - -**New feature:** Set `$DebugPreference = 'Continue'` to see detailed extraction information - -```powershell -$DebugPreference = 'Continue' -./post-ai-summary-comment.ps1 -PRNumber 12345 -``` - -**Debug output shows:** -``` -[DEBUG] Matched pattern for: 📋 Final Report -[DEBUG] Content length: 2355 chars -[DEBUG] First 100 chars: --- - -### Summary - -PR #27340 provides a **correct and well-tested fix**... -``` - -**Benefit:** Easy troubleshooting when validation fails - ---- - -### 4. **Better Error Messages** - -**Before:** -``` -⛔ VALIDATION FAILED -Found 1 validation error(s): - - Report: Phase Report is marked as '✅ COMPLETE' but has NO content -``` - -**After:** -``` -⛔ VALIDATION FAILED -Found 1 validation error(s): - - Report: Phase Report is marked as '✅ COMPLETE' but has NO content - -💡 Fix these issues in the content before posting the review comment. - Or use -SkipValidation to bypass these checks (not recommended). - -🐛 Debug tip: Run with $DebugPreference = 'Continue' for detailed extraction info -``` - ---- - -### 5. **Relaxed Phase 4 Validation** - -**Before:** Report phase required: -- Exact "Final Recommendation" text -- "Root Cause" section -- "Key Findings" section -- "Solution Analysis" section -- Minimum 500 characters - -**After:** Report phase only requires: -- Any form of recommendation (APPROVE, REQUEST CHANGES, etc.) -- Any form of analysis (Summary, Fix Quality, etc.) -- Minimum 200 characters (reduced from 500) - -**Benefit:** More flexibility in how you structure the final report - ---- - -## Usage Examples - -### Basic Usage (unchanged) -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -``` - -### With Debug Mode -```powershell -$DebugPreference = 'Continue' -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -``` - -### Skip Validation (when needed) -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -SkipValidation -``` - -### Dry Run (preview only) -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27340 -DryRun -``` - ---- - -## What You Need to Know - -### Headers That Work Now - -Any of these variations will be recognized: - -**Pre-Flight:** -- `📋 Issue Summary` ✅ (preferred) -- `📋 Pre-Flight` ✅ -- `🔍 Pre-Flight` ✅ - -**Gate:** -- `🚦 Gate - Test Verification` ✅ (preferred) -- `🚦 Gate` ✅ -- `📋 Gate` ✅ - -**Fix:** -- `🔧 Fix Candidates` ✅ (preferred) -- `🔧 Fix` ✅ -- `📋 Fix` ✅ - -**Report:** -- `📋 Final Report` ✅ -- `📋 Phase 4: Final Report` ✅ -- `📋 Report` ✅ -- `Phase 4: Report` ✅ -- `Final Report` ✅ - ---- - -## Migration Guide - -**No changes needed!** The script is backward compatible. If you have existing content with the old header format, it will continue to work. - -If you want to use the new flexibility: -- Just use simpler headers like `📋 Final Report` instead of `📋 Phase 4: Report — Final Recommendation` -- The script will find it either way - ---- - -## Common Issues & Solutions - -### Issue: "Phase Report has NO content" - -**Solution 1:** Check your content structure -```bash -grep -A 5 "📋.*Report" your-content-file.md -``` - -Make sure you have: -```markdown -
-📋 Final Report - -Your report content here... - -
-``` - -**Solution 2:** Use debug mode to see what's happening -```powershell -$DebugPreference = 'Continue' -./post-ai-summary-comment.ps1 -PRNumber XXXXX -``` - -**Solution 3:** Use `-SkipValidation` if content is definitely there -```powershell -./post-ai-summary-comment.ps1 -PRNumber XXXXX -SkipValidation -``` - ---- - -### Issue: Validation warnings about missing sections - -**These are just suggestions!** Warnings won't block posting. You can: -- Ignore them (the comment will post anyway) -- Address them if you want a more complete review -- Use `-SkipValidation` to hide all validation output - ---- - -## Developer Notes - -### How Pattern Matching Works - -```powershell -function Extract-PhaseContent { - param( - [string]$StateContent, - [string[]]$PhaseTitles, # Array of possible titles - [switch]$Debug - ) - - foreach ($title in $PhaseTitles) { - $pattern = "(?s)
\s*$([regex]::Escape($title))(.*?)
" - if ($StateContent -match $pattern) { - return $Matches[1].Trim() - } - } - return $null # No match found -} -``` - -The function tries each pattern in order until one matches. - -### Adding New Pattern Variations - -To support a new header variation, just add it to the array: - -```powershell -$reportContent = Extract-PhaseContent -StateContent $Content -PhaseTitles @( - "📋 Phase 4: Report — Final Recommendation", - "📋 Phase 4: Final Report", - "📋 Phase 4: Report", - "📋 Final Report", - "📋 Report", - "Phase 4: Report", - "Final Report", - "Your New Pattern Here" # <-- Add here -) -Debug:$debugMode -``` - ---- - -## Future Improvements - -Potential enhancements: -- [ ] Auto-detect phase titles from content (no hardcoded patterns) -- [ ] Support markdown headings (`##` / `###`) in addition to `
` -- [ ] Validate links and references work -- [ ] Check that commit SHAs are valid -- [ ] Suggest fixes for common issues (auto-fix mode) - ---- - -## Testing - -The improvements have been tested with: -- ✅ PR #27340 (Entry/Editor keyboard issue) -- ✅ Content with various header formats -- ✅ Dry run mode -- ✅ Debug mode -- ✅ Skip validation mode -- ✅ Multiple phase title variations - ---- - -## Feedback - -If you encounter issues or have suggestions, please: -1. Try debug mode first: `$DebugPreference = 'Continue'` -2. Check the content structure -3. Report the issue with debug output included diff --git a/.github/skills/ai-summary-comment/NO-EXTERNAL-REFERENCES-RULE.md b/.github/skills/ai-summary-comment/NO-EXTERNAL-REFERENCES-RULE.md deleted file mode 100644 index 70223059df16..000000000000 --- a/.github/skills/ai-summary-comment/NO-EXTERNAL-REFERENCES-RULE.md +++ /dev/null @@ -1,186 +0,0 @@ -# Critical Rule: No External File References in PR Comments - -## The Problem - -When the PR agent posts review comments to GitHub, those comments are viewed by: -- PR authors -- Other reviewers -- Future contributors searching issue history -- Community members watching the PR - -**None of these people have access to your local files!** - -## The Rule - -### ❌ NEVER Do This - -```markdown -### Title & Description: ⚠️ Minor Updates Needed - -**Issues to fix:** -1. Missing required NOTE block -2. Technical inaccuracy in description - -**See:** `CustomAgentLogsTmp/PRState/27340/pr-finalize/pr-finalize-summary.md` for recommended updates -``` - -**Why this is bad:** -- GitHub users can't access `CustomAgentLogsTmp/` -- Makes the review useless - "see file I can't access" -- Author can't act on recommendations -- Future agents can't learn from the review - ---- - -### ✅ ALWAYS Do This - -```markdown -### Title & Description: ⚠️ Minor Updates Needed - -**Current description is HIGH QUALITY:** -- ✅ Clear root cause for both platforms -- ✅ Before/after videos -- ✅ Well-structured sections - -**Issues to fix:** - -**Issue 1: Missing required NOTE block** - -Add this at the top of the description: -```markdown -> [!NOTE] -> Are you waiting for the changes in this PR to be merged? -> It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! -``` - -**Issue 2: Technical inaccuracy in "Description of Change"** - -Current text says: -> "Added platform-specific handling to dismiss the soft keyboard **and remove focus**..." - -Should say: -> "Added platform-specific handling to dismiss the soft keyboard when the Entry or Editor visibility is set to False." - -**Reason:** The code only calls `HideSoftInputAsync()` to dismiss the keyboard. It does NOT call `Unfocus()` to remove focus. Focus state remains unchanged. - -**Recommended additions:** - -Add an **Implementation** subsection: -```markdown -**Implementation:** -- Added `MapIsVisible` handler in `InputView.Platform.cs` (iOS/Android only) -- Calls `HideSoftInputAsync()` when control becomes invisible and keyboard is showing -- Registered handler in `Entry.Mapper.cs` and `Editor.Mapper.cs` -``` -``` - -**Why this is good:** -- ✅ Self-contained - everything needed is in the comment -- ✅ Actionable - author can copy/paste the fixes -- ✅ Clear - shows exact before/after text -- ✅ Educational - explains the reasoning -- ✅ Accessible - works on GitHub - ---- - -## Where This Applies - -### pr-finalize Skill - -When running `pr-finalize` skill, you create output: - -1. **Summary file** (local reference) - - Location: `CustomAgentLogsTmp/PRState/XXXXX/pr-finalize/pr-finalize-summary.md` - - Purpose: Your detailed analysis and working notes - - Audience: You and local CLI users - - **MUST be self-contained** - no external references - -### PR Agent Phase 4 (Report) - -When completing Phase 4: -- Include ALL pr-finalize findings inline -- Show exact code blocks for NOTE block -- Show exact before/after text for corrections -- Explain reasoning for each recommendation -- Never reference local files - ---- - -## Examples from Real Usage - -### ❌ Bad Example (PR #27340 - first attempt) - -```markdown -**Issues to fix:** -1. **Missing required NOTE block** (mandatory for all PRs) -2. **Minor technical inaccuracy:** Description says "remove focus" but code only dismisses keyboard - -**See:** `CustomAgentLogsTmp/PRState/27340/pr-finalize/pr-finalize-summary.md` for recommended updates -``` - -**Result:** PR author sees the issues but has no idea how to fix them without accessing local files. - ---- - -### ✅ Good Example (PR #27340 - corrected) - -```markdown -**Issues to fix:** - -**Issue 1: Missing required NOTE block** - -Add this at the top of the description: -```markdown -> [!NOTE] -> Are you waiting for the changes in this PR to be merged? -> It would be very helpful if you could [test the resulting artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from this PR and let us know in a comment if this change resolves your issue. Thank you! -``` - -**Issue 2: Technical inaccuracy in "Description of Change"** - -Current text says: -> "Added platform-specific handling to dismiss the soft keyboard **and remove focus**..." - -Should say: -> "Added platform-specific handling to dismiss the soft keyboard when the Entry or Editor visibility is set to False." - -**Reason:** The code only calls `HideSoftInputAsync()` to dismiss the keyboard. It does NOT call `Unfocus()` to remove focus. - -**Recommended additions to description:** - -Add an **Implementation** subsection after "Description of Change": -```markdown -**Implementation:** -- Added `MapIsVisible` handler in `InputView.Platform.cs` (iOS/Android only) -- Calls `HideSoftInputAsync()` when control becomes invisible and keyboard is showing -- Registered handler in `Entry.Mapper.cs` and `Editor.Mapper.cs` -``` -``` - -**Result:** PR author can immediately act on every recommendation with clear guidance. - ---- - -## Checklist for Report Phase - -When completing Phase 4, verify: - -- [ ] All recommendations are inline (no file references) -- [ ] Code blocks show exact text to add -- [ ] Before/after comparisons for corrections -- [ ] Reasoning explained for each issue -- [ ] Examples are copy/paste ready -- [ ] No references to `CustomAgentLogsTmp/` - ---- - -## Quick Reference - -| What | Where | Audience | Self-Contained? | -|------|-------|----------|-----------------| -| Summary file | `CustomAgentLogsTmp/.../summary.md` | Local CLI | N/A (local only) | -| PR comment | GitHub PR page | Public | ✅ YES - REQUIRED | - -**Remember:** Anything posted to GitHub must be self-contained. Never reference local files. - --- diff --git a/.github/skills/ai-summary-comment/SKILL.md b/.github/skills/ai-summary-comment/SKILL.md deleted file mode 100644 index 073fea7448fb..000000000000 --- a/.github/skills/ai-summary-comment/SKILL.md +++ /dev/null @@ -1,574 +0,0 @@ ---- -name: ai-summary-comment -description: Posts or updates automated progress comments on GitHub PRs. Use after completing any PR agent phase (pre-flight, tests, gate, fix, report). Triggers on 'post comment to PR', 'update PR progress', 'comment on PR with results', 'post pre-flight comment'. Creates single aggregated review comment with collapsible sections per commit. -metadata: - author: dotnet-maui - version: "5.0" -compatibility: Requires GitHub CLI (gh) authenticated with access to dotnet/maui repository. ---- - -# PR Comment Skill - -This skill posts automated progress comments to GitHub Pull Requests during the PR review workflow. Comments are **self-contained** with collapsible Review Session details, providing rich context to maintainers and contributors. - -**⚠️ Self-Contained Rule**: All content in PR comments must be self-contained. Never reference local files like `CustomAgentLogsTmp/` - GitHub users cannot access your local filesystem. - -**✨ Key Features**: -- **Single Unified Comment**: ONE comment per PR/Issue containing ALL sections (PR Review, Try-Fix, Write-Tests, Verify-Tests) -- **Section-Based Updates**: Each script updates only its section, preserving others -- **Duplicate Prevention**: Finds existing `` comment and updates it -- **File-Based DryRun Preview**: Use `-DryRun` to preview changes in a local file before posting -- **Simple Interface**: Just provide PR number - script handles everything else - -## Comment Architecture - -### Unified AI Summary Comment - -Most scripts post to the **same single comment** identified by ``. Each script updates its own section: - -```markdown - - -## 🤖 AI Summary - - -... PR review phases ... - - - -... try-fix attempts ... - - - -... write-tests attempts ... - - - -... test verification results ... - -``` - -**Behavior:** -- First script to run creates the comment -- Subsequent scripts find the existing comment and update/add their section -- Sections are independent - updating one preserves others - -### Separate PR Finalization Comment - -The `post-pr-finalize-comment.ps1` script posts a **separate comment** identified by ``. This comment contains three sections: -- **Title**: Shows the current vs recommended PR title -- **Description**: Shows description assessment, missing elements, and **recommended description** -- **Code Review**: Shows code review findings (critical issues, suggestions, positive observations) - -If an existing finalize comment exists, it will be replaced with the updated sections. This keeps finalization reviews distinct from automated analysis. - -**⚠️ Important Requirements for PR Finalize Comments:** -- When `TitleStatus` is `NeedsUpdate`, **always provide** `-RecommendedTitle` -- When `DescriptionStatus` is `NeedsUpdate` or `NeedsRewrite`, **always provide** `-RecommendedDescription` with the full suggested description text -- The script will warn if these are missing but won't fail - -## Section Scripts - -### AI Summary Sections (Unified Comment) - -| Section | Script | Location | -|---------|--------|----------| -| `PR-REVIEW` | `post-ai-summary-comment.ps1` | `.github/skills/ai-summary-comment/scripts/` | -| `TRY-FIX` | `post-try-fix-comment.ps1` | `.github/skills/ai-summary-comment/scripts/` | -| `WRITE-TESTS` | `post-write-tests-comment.ps1` | `.github/skills/ai-summary-comment/scripts/` | -| `VERIFY-TESTS` | `post-verify-tests-comment.ps1` | `.github/skills/ai-summary-comment/scripts/` | - -### Separate Comments - -| Comment | Script | Marker | -|---------|--------|--------| -| PR Finalization | `post-pr-finalize-comment.ps1` | `` | - -## Supported Phases - -| Phase | Description | When to Post | What This Enables Next | -|-------|-------------|--------------|------------------------| -| `pre-flight` | Context gathering complete | After documenting issue, files, and discussion | **Tests Phase**: Agent can now verify/create test files that reproduce the bug | -| `tests` | Test analysis complete | After identifying test files and coverage | **Gate Phase**: Agent can run tests to verify they catch the bug | -| `gate` | Test validation complete | After running tests and verifying bug reproduction | **Fix Phase**: Agent can explore alternative fixes (tests proven to catch bug) | -| `fix` | Solution comparison complete | After comparing PR fix with alternatives | **Report Phase**: Agent can finalize recommendation based on fix comparison | -| `report` | Final analysis complete | After generating comprehensive review | **PR Decision**: Maintainers can approve/merge or request changes based on full analysis | - -## Usage - -### Simplest: Just provide PR number (auto-loads from phase files) - -```bash -# Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/*/content.md -pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27246 -``` - -### Parameters - -| Parameter | Required | Description | Example | -|-----------|----------|-------------|---------| -| `PRNumber` | Yes | Pull request number | `12345` | -| `DryRun` | No | Preview changes in local file instead of posting to GitHub | `-DryRun` | -| `PreviewFile` | No | Path to local preview file for DryRun mode (default: `CustomAgentLogsTmp/PRState/{PRNumber}/ai-summary-comment-preview.md`) | `-PreviewFile ./preview.md` | -| `SkipValidation` | No | Skip validation checks (not recommended) | `-SkipValidation` | - -## DryRun Preview Workflow - -Use `-DryRun` to preview the combined comment before posting to GitHub. Each script updates the same preview file, mirroring how the actual GitHub comment is updated. - -```bash -# Step 1: Run verify-tests script (creates preview file) -pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 -PRNumber 32891 -DryRun - -# Step 2: Run try-fix script (updates same preview file) -pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber 32891 -DryRun - -# Step 3: Review the combined preview -open CustomAgentLogsTmp/PRState/32891/ai-summary-comment-preview.md - -# Step 4: Post for real (remove -DryRun) -pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 -PRNumber 32891 -pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber 32891 -``` - -**Key behavior:** The preview file exactly matches what will be posted to GitHub. Multiple scripts accumulate their sections in the same file. - -### Section Ordering - -Sections appear in the unified comment in this order (based on which scripts run first): -1. **VERIFY-TESTS** - Test verification results -2. **TRY-FIX** - Alternative fix exploration attempts -3. **WRITE-TESTS** - Test writing attempts -4. **PR-REVIEW** - PR review phases - -Each section is wrapped with markers like `` and ``. - -### Cleanup - -To reset the preview file for a fresh start: -```bash -rm CustomAgentLogsTmp/PRState/{PRNumber}/ai-summary-comment-preview.md -``` - -### Prerequisites - -Scripts require GitHub CLI authentication: -```bash -gh auth status # Verify authentication before running -``` - -## Comment Format - -Comments are formatted with: -- **Phase badge** (🔍 Pre-Flight, 🚦 Gate, 🔧 Fix, 📋 Report) -- **Status indicator** (✅ Completed, ⚠️ Issues Found) -- **Expandable review sessions** (each session is a collapsible section) -- **What's Next** (what phase happens next) - -### Review Session Tracking - -When the same PR is reviewed multiple times (e.g., after new commits), the script **updates the single aggregated review comment** and adds a new expandable section for each commit-based review session. - -### Example Output - -```markdown -## 🔍 Pre-Flight: Context Gathering Complete - -✅ **Status**: Phase completed successfully - -### Summary -- **Issue**: #33356 - CollectionView crash on iOS -- **Platforms Affected**: iOS, MacCatalyst -- **Files Changed**: 2 implementation files, 1 test file -- **Discussion**: 3 key reviewer comments identified - -### Key Findings -- Crash occurs when scrolling rapidly with large datasets -- Existing PR adds null check in ItemsViewController -- Test coverage includes iOS device test - -### Next Steps -→ **Phase 2: Gate** - Verifying tests catch the bug - ---- -*Posted by PR Agent @ 2026-01-17 14:23:45 UTC* -``` - -## Script Files - -- [`post-ai-summary-comment.ps1`](scripts/post-ai-summary-comment.ps1) - Posts or updates the aggregated PR agent review comment -- [`post-try-fix-comment.ps1`](scripts/post-try-fix-comment.ps1) - Posts or updates try-fix attempts comment - -## Try-Fix Comment Script - -The `post-try-fix-comment.ps1` script updates the `` section of the unified AI Summary comment. It aggregates all try-fix attempts into collapsible sections. Works for both issues and PRs (GitHub treats PR comments as issue comments). - -**✨ Auto-Loading from `CustomAgentLogsTmp`**: The script automatically discovers and aggregates ALL attempt directories from `CustomAgentLogsTmp/PRState/{IssueNumber}/PRAgent/try-fix/`. - -### Usage - -#### Simplest: Provide attempt directory - -```powershell -# All parameters auto-loaded from directory structure -pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 ` - -TryFixDir CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/attempt-1 -``` - -#### Or just provide issue number - -```powershell -# Auto-discovers and posts latest attempt from CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/ -pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber 27246 -``` - -#### Manual parameters - -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 ` - -IssueNumber 19806 ` - -AttemptNumber 1 ` - -Approach "LayoutExtensions Width Constraint" ` - -RootCause "ComputeFrame only constrains width for Fill alignment" ` - -FilesChanged "| File | Changes |`n|------|---------|`n| LayoutExtensions.cs | +17/-3 |" ` - -Status "Compiles" ` - -CodeSnippet "else if (!hasExplicitWidth) { ... }" ` - -Analysis "Core project compiles successfully" -``` - -### Parameters - -| Parameter | Required | Description | -|-----------|----------|-------------| -| `TryFixDir` | No* | Path to try-fix attempt directory (auto-loads all parameters) | -| `IssueNumber` | No* | Issue or PR number to post comment on | -| `AttemptNumber` | No* | Attempt number (1, 2, 3, etc.) - auto-detected from TryFixDir | -| `Approach` | No* | Brief description of fix approach | -| `RootCause` | No | Description of root cause identified | -| `FilesChanged` | No* | Markdown table of files changed - auto-generated from diff | -| `Status` | No* | "Compiles", "Pass", or "Fail" - loaded from result.txt | -| `CodeSnippet` | No | Code snippet showing the fix - loaded from fix.diff | -| `Analysis` | No | Analysis of why it worked/failed - loaded from analysis.md | -| `DryRun` | No | Print comment instead of posting | - -*When using `-TryFixDir`, all marked parameters are auto-loaded from files in the directory. - -### Expected Directory Structure - -``` -CustomAgentLogsTmp/PRState/{IssueNumber}/PRAgent/try-fix/ -├── attempt-1/ -│ ├── approach.md # Brief description of the approach (required) -│ ├── result.txt # "Pass", "Fail", or "Compiles" (required) -│ ├── fix.diff # Git diff of the fix (optional) -│ └── analysis.md # Detailed analysis (optional) -├── attempt-2/ -│ └── ... -└── attempt-3/ - └── ... -``` - -### Comment Format - -```markdown -## 🔧 Try-Fix Attempts for Issue #XXXXX - - - -
-📊 Expand Full Details - -**Issue:** [#XXXXX](link) - ---- - -
-🔧 Attempt #1: Approach Name ✅ Status -... attempt details ... -
- ---- - -*This fix was developed independently.* - -
-``` - -### Key Behaviors - -- First attempt creates new comment with `` marker -- Subsequent attempts **edit the same comment** (no new comments) -- Outer wrapper shows "📊 Expand Full Details" - keeps PR page clean -- Each attempt is a nested collapsible section inside the wrapper - ---- - -## Verify-Tests Comment Script - -The `post-verify-tests-comment.ps1` script updates the `` section of the unified AI Summary comment. It documents test verification results (whether tests fail without fix and pass with fix). - -**✨ Auto-Loading from `CustomAgentLogsTmp`**: The script automatically loads verification results from `CustomAgentLogsTmp/PRState/{PRNumber}/verify-tests-fail/verification-report.md`. - -### Usage - -#### Simplest: Provide PR number - -```powershell -# Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/verify-tests-fail/ -pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 -PRNumber 32891 -``` - -#### With explicit report file - -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 ` - -PRNumber 32891 ` - -ReportFile CustomAgentLogsTmp/PRState/32891/PRAgent/gate/verify-tests-fail/verification-report.md -``` - -### Parameters - -| Parameter | Required | Description | -|-----------|----------|-------------| -| `PRNumber` | Yes | Pull request number | -| `ReportFile` | No | Path to verification report (auto-discovered if not provided) | -| `Status` | No | "Passed" or "Failed" - auto-detected from report | -| `Platform` | No | Platform tested (ios, android, etc.) - auto-detected from report | -| `Mode` | No | "FailureOnly" or "FullVerification" - auto-detected from report | -| `DryRun` | No | Preview changes in local file instead of posting | -| `PreviewFile` | No | Path to local preview file for DryRun mode | - -### Expected Directory Structure - -``` -CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/gate/verify-tests-fail/ -├── verification-report.md # Full verification report (required) -├── verification-log.txt # Detailed log (optional) -├── test-without-fix.log # Test output without fix (optional) -└── test-with-fix.log # Test output with fix (optional) -``` - ---- - -## Write-Tests Comment Script - -The `post-write-tests-comment.ps1` script updates the `` section of the unified AI Summary comment. It documents test writing attempts for an issue. - -**✨ Auto-Loading from `CustomAgentLogsTmp`**: The script can automatically load test details from the write-tests output directory structure. - -### Usage - -#### Simplest: Provide test directory - -```powershell -# All parameters auto-loaded from directory structure -pwsh .github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 ` - -TestDir CustomAgentLogsTmp/PRState/27246/write-tests/attempt-1 -``` - -#### Or just provide issue number - -```powershell -# Auto-discovers and posts latest attempt from CustomAgentLogsTmp/PRState/27246/write-tests/ -pwsh .github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 -IssueNumber 27246 -``` - -#### Manual parameters - -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 ` - -IssueNumber 33331 ` - -AttemptNumber 1 ` - -TestDescription "Verifies Picker.IsOpen property changes correctly" ` - -HostAppFile "src/Controls/tests/TestCases.HostApp/Issues/Issue33331.cs" ` - -TestFile "src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue33331.cs" ` - -TestMethod "PickerIsOpenPropertyChanges" ` - -Category "Picker" ` - -VerificationStatus "Verified" -``` - -### Parameters - -| Parameter | Required | Description | -|-----------|----------|-------------| -| `TestDir` | No* | Path to write-tests attempt directory (auto-loads all parameters) | -| `IssueNumber` | No* | Issue or PR number to post comment on | -| `AttemptNumber` | No* | Attempt number (1, 2, 3, etc.) - auto-detected from TestDir | -| `TestDescription` | No* | Brief description of what the test verifies | -| `HostAppFile` | No* | Path to the HostApp test page file | -| `TestFile` | No* | Path to the NUnit test file | -| `TestMethod` | No* | Name of the test method | -| `Category` | No* | UITestCategories category used | -| `VerificationStatus` | No* | "Verified", "Failed", or "Unverified" - loaded from result.txt | -| `Platforms` | No | Platforms the test runs on (default: "All") | -| `Notes` | No | Additional notes - loaded from notes.md | -| `DryRun` | No | Print comment instead of posting | - -*When using `-TestDir`, all marked parameters are auto-loaded from files in the directory. - -### Expected Directory Structure - -``` -CustomAgentLogsTmp/PRState/{IssueNumber}/write-tests/ -├── attempt-1/ -│ ├── description.md # Brief test description (required) -│ ├── test-info.json # {HostAppFile, TestFile, TestMethod, Category} (required) -│ ├── result.txt # "Verified", "Pass", "Failed", or "Unverified" (required) -│ └── notes.md # Additional notes (optional) -├── attempt-2/ -│ └── ... -└── attempt-3/ - └── ... -``` - -### test-info.json Format - -```json -{ - "HostAppFile": "src/Controls/tests/TestCases.HostApp/Issues/Issue27246.cs", - "TestFile": "src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue27246.cs", - "TestMethod": "ScrollToFirstItemWithHeader", - "Category": "CollectionView" -} -``` - ---- - -## PR Finalize Comment Script - -The `post-pr-finalize-comment.ps1` script posts a **separate comment** (not part of the unified AI Summary) specifically for PR finalization reviews. It provides structured feedback on the PR title, description, and code review findings. - -### Usage - -#### Simplest: Just provide PR number (auto-loads from summary file) - -```powershell -# Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/pr-finalize/pr-finalize-summary.md -pwsh .github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 -PRNumber 33892 -``` - -#### Full manual parameters (recommended for best results) - -```powershell -pwsh .github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 ` - -PRNumber 33892 ` - -TitleStatus "NeedsUpdate" ` - -CurrentTitle "Fix 32650 Image Orientation" ` - -RecommendedTitle "[iOS][Android] MediaPicker: Fix image orientation when RotateImage=true" ` - -TitleIssues "- Missing platform tags -- Doesn't describe the behavior fix" ` - -DescriptionStatus "NeedsUpdate" ` - -DescriptionAssessment "The current description is minimal and missing: -- ❌ Missing NOTE block for testing artifacts -- ❌ No root cause analysis -- ❌ No technical details" ` - -MissingElements "Add the NOTE block, root cause, and technical details." ` - -RecommendedDescription "> [!NOTE] -> Are you waiting for this PR? Test it: [Testing PR Builds](link) - -### Root Cause -...description... - -### Description of Change -...details..." ` - -CodeReviewStatus "IssuesFound" ` - -CodeReviewFindings "### 🔴 Critical Issues -**1. Broken indentation** -- File: \`src/file.cs\` -- Problem: Inconsistent tabs/spaces - -### 🟡 Suggestions -1. Consider disposing Matrix object - -### ✅ Looks Good -- Proper cleanup in finally block" -``` - -### Parameters - -| Parameter | Required | Description | -|-----------|----------|-------------| -| `PRNumber` | Yes* | Pull request number | -| `SummaryFile` | No | Path to pr-finalize-summary.md (auto-discovered) | -| `TitleStatus` | No* | `Good` or `NeedsUpdate` | -| `CurrentTitle` | No* | Current PR title (fetched from GitHub if not provided) | -| `RecommendedTitle` | No | **Required if TitleStatus is NeedsUpdate** | -| `TitleIssues` | No | List of issues with current title | -| `DescriptionStatus` | No* | `Excellent`, `Good`, `NeedsUpdate`, or `NeedsRewrite` | -| `DescriptionAssessment` | Yes | Assessment of description quality | -| `MissingElements` | No | What's missing from the description | -| `RecommendedDescription` | No | **Required if DescriptionStatus is NeedsUpdate/NeedsRewrite** | -| `CodeReviewStatus` | No | `Passed`, `IssuesFound`, or `Skipped` | -| `CodeReviewFindings` | No | Markdown content for code review section | -| `DryRun` | No | Preview instead of posting | - -*At least PRNumber or SummaryFile required. Script auto-detects values when possible. - -### ⚠️ Common Mistakes to Avoid - -1. **Missing RecommendedTitle when TitleStatus is NeedsUpdate** - - The script will warn but still post - always provide a recommended title - -2. **Missing RecommendedDescription when DescriptionStatus is NeedsUpdate** - - Users need to see what the description SHOULD look like - -3. **Code review findings not starting with proper headers** - - Always structure with `### 🔴 Critical Issues`, `### 🟡 Suggestions`, `### ✅ Looks Good` - -4. **Auto-parsing from summary file getting confused** - - When in doubt, provide explicit parameters instead of relying on auto-parsing - -### Expected Directory Structure - -The PR agent writes phase output files that comment scripts auto-load: - -``` -CustomAgentLogsTmp/PRState/{PRNumber}/ -├── PRAgent/ -│ ├── pre-flight/ -│ │ └── content.md # Phase 1 output (auto-loaded by post-ai-summary-comment.ps1) -│ ├── gate/ -│ │ ├── content.md # Phase 2 output (auto-loaded by post-ai-summary-comment.ps1) -│ │ └── verify-tests-fail/ # Script output from verify-tests-fail.ps1 -│ │ ├── verification-report.md -│ │ ├── verification-log.txt -│ │ ├── test-without-fix.log -│ │ └── test-with-fix.log -│ ├── try-fix/ -│ │ ├── content.md # Phase 3 summary (auto-loaded by post-ai-summary-comment.ps1) -│ │ ├── attempt-1/ # Individual attempt outputs -│ │ │ ├── approach.md -│ │ │ ├── result.txt -│ │ │ ├── fix.diff -│ │ │ └── analysis.md -│ │ └── attempt-2/ -│ │ └── ... -│ └── report/ -│ └── content.md # Phase 4 output (auto-loaded by post-ai-summary-comment.ps1) -├── pr-finalize/ -│ ├── pr-finalize-summary.md -│ ├── recommended-description.md -│ └── code-review.md -└── copilot-logs/ - ├── process-*.log - └── session-*.md -``` - -### Auto-Loading Behavior - -When `post-ai-summary-comment.ps1` is called, it auto-discovers phase files: -1. Checks `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/*/content.md` -2. Loads all available phase content files -3. Builds the comment structure from the loaded files -4. Posts/updates the unified AI Summary comment - -This eliminates the need to pass large content strings as parameters. - ---- - -## Technical Details - -- Comments identified by HTML marker `` -- Existing comments are updated (not duplicated) when posting again -- Review sessions grouped by commit SHA -- Uses `gh api` for create/update operations diff --git a/.github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 b/.github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 deleted file mode 100644 index e32b5f9d1698..000000000000 --- a/.github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 +++ /dev/null @@ -1,808 +0,0 @@ -#!/usr/bin/env pwsh -<# -.SYNOPSIS - Posts or updates a PR finalize comment on a GitHub Pull Request. - -.DESCRIPTION - Creates ONE comment for PR finalization with three collapsible sections: Title, Description, and Code Review. - Uses HTML marker for identification. - - **Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/pr-finalize/** - - If an existing finalize comment exists, it will be REPLACED with the new content. - Otherwise, a new comment will be created. - - Format: - ## 📋 PR Finalization Review - - -
- Title: ✅ Good - ... title details ... -
- -
- Description: ⚠️ Needs Update - ... description details ... -
- -
- Code Review: ✅ Passed - ... code review findings ... -
- -.PARAMETER PRNumber - The PR number to post comment on (required unless SummaryFile provided) - -.PARAMETER SummaryFile - Path to pr-finalize-summary.md file. If provided, auto-loads review data from this file. - -.PARAMETER TitleStatus - Title assessment: "Good", "NeedsUpdate" (required unless loading from SummaryFile) - -.PARAMETER CurrentTitle - Current PR title (required unless loading from SummaryFile) - -.PARAMETER RecommendedTitle - Recommended PR title (optional - only if TitleStatus is NeedsUpdate) - -.PARAMETER TitleIssues - List of issues with the current title (optional - only if TitleStatus is NeedsUpdate) - -.PARAMETER DescriptionStatus - Description assessment: "Excellent", "Good", "NeedsUpdate", "NeedsRewrite" (required unless loading from SummaryFile) - -.PARAMETER DescriptionAssessment - Quality assessment of the description (required unless loading from SummaryFile) - -.PARAMETER MissingElements - List of missing elements (optional) - -.PARAMETER RecommendedDescription - Full recommended description (optional - only if needs rewrite) - -.PARAMETER CodeReviewStatus - Code review assessment: "Passed", "IssuesFound", "Skipped" (optional - defaults to "Skipped" if not provided) - -.PARAMETER CodeReviewFindings - Code review findings content (optional - markdown with critical issues, suggestions, and positive observations) - -.PARAMETER DryRun - Print comment instead of posting - -.EXAMPLE - # Simplest: Just provide PR number (auto-loads from CustomAgentLogsTmp) - ./post-pr-finalize-comment.ps1 -PRNumber 27246 - -.EXAMPLE - # Or provide summary file path - ./post-pr-finalize-comment.ps1 -SummaryFile CustomAgentLogsTmp/PRState/27246/pr-finalize/pr-finalize-summary.md - -.EXAMPLE - # Manual parameters - ./post-pr-finalize-comment.ps1 -PRNumber 25748 ` - -TitleStatus "Good" ` - -CurrentTitle "[iOS] Fix SafeArea padding calculation" ` - -DescriptionStatus "Good" ` - -DescriptionAssessment "Clear structure, accurate technical details, matches implementation" ` - -CodeReviewStatus "Passed" ` - -CodeReviewFindings "No critical issues found. Code follows best practices." -#> - -param( - [Parameter(Mandatory=$false)] - [int]$PRNumber, - - [Parameter(Mandatory=$false)] - [string]$SummaryFile, - - [Parameter(Mandatory=$false)] - [ValidateSet("Good", "NeedsUpdate", "")] - [string]$TitleStatus, - - [Parameter(Mandatory=$false)] - [string]$CurrentTitle, - - [Parameter(Mandatory=$false)] - [string]$RecommendedTitle, - - [Parameter(Mandatory=$false)] - [string]$TitleIssues, - - [Parameter(Mandatory=$false)] - [ValidateSet("Excellent", "Good", "NeedsUpdate", "NeedsRewrite", "")] - [string]$DescriptionStatus, - - [Parameter(Mandatory=$false)] - [string]$DescriptionAssessment, - - [Parameter(Mandatory=$false)] - [string]$MissingElements, - - [Parameter(Mandatory=$false)] - [string]$RecommendedDescription, - - [Parameter(Mandatory=$false)] - [ValidateSet("Passed", "IssuesFound", "Skipped", "")] - [string]$CodeReviewStatus, - - [Parameter(Mandatory=$false)] - [string]$CodeReviewFindings, - - [Parameter(Mandatory=$false)] - [switch]$DryRun, - - [Parameter(Mandatory=$false)] - [switch]$Unified, - - [Parameter(Mandatory=$false)] - [string]$PreviewFile -) - -$ErrorActionPreference = "Stop" - -Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan -Write-Host "║ PR Finalize Comment (Post/Update) ║" -ForegroundColor Cyan -Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan - -# ============================================================================ -# AUTO-DISCOVERY FROM SUMMARY FILE -# ============================================================================ - -# If PRNumber provided but no SummaryFile, try to find it -if ($PRNumber -gt 0 -and [string]::IsNullOrWhiteSpace($SummaryFile)) { - $summaryPath = "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/pr-finalize-summary.md" - if (-not (Test-Path $summaryPath)) { - $repoRoot = git rev-parse --show-toplevel 2>$null - if ($repoRoot) { - $summaryPath = Join-Path $repoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize/pr-finalize-summary.md" - } - } - - if (Test-Path $summaryPath) { - $SummaryFile = $summaryPath - Write-Host "ℹ️ Auto-discovered summary file: $SummaryFile" -ForegroundColor Cyan - } -} - -# If SummaryFile provided, parse it -if (-not [string]::IsNullOrWhiteSpace($SummaryFile)) { - if (-not (Test-Path $SummaryFile)) { - throw "Summary file not found: $SummaryFile" - } - - $content = Get-Content $SummaryFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Loading from summary file: $SummaryFile" -ForegroundColor Cyan - - # Extract PRNumber from path if not provided - if ($PRNumber -eq 0 -and $SummaryFile -match '[/\\](\d+)[/\\]pr-finalize') { - $PRNumber = [int]$Matches[1] - Write-Host "ℹ️ Auto-detected PRNumber: $PRNumber from path" -ForegroundColor Cyan - } - - # Extract Recommended Title FIRST (needed for TitleStatus detection) - if ([string]::IsNullOrWhiteSpace($RecommendedTitle)) { - # Try different patterns - if ($content -match '\*\*Recommended.*?Title.*?\*\*:?\s*[`"]?([^`"\n]+)[`"]?') { - $RecommendedTitle = $Matches[1].Trim() - } elseif ($content -match 'Recommended:\s*`([^`]+)`') { - $RecommendedTitle = $Matches[1].Trim() - } elseif ($content -match '(?s)\*\*Recommended:\*\*\s*```\s*([^\n]+)') { - # Code fence format - $RecommendedTitle = $Matches[1].Trim() - } elseif ($content -match '(?s)### 📋 Title Assessment.+?\*\*Recommended:\*\*\s*```\s*([^\n]+)') { - $RecommendedTitle = $Matches[1].Trim() - } - if ($RecommendedTitle) { - Write-Host "ℹ️ Extracted RecommendedTitle: $RecommendedTitle" -ForegroundColor Cyan - } - } - - # Extract Title assessment - if RecommendedTitle exists, title needs update - if ([string]::IsNullOrWhiteSpace($TitleStatus)) { - # If we have a recommended title, the title needs update - if (-not [string]::IsNullOrWhiteSpace($RecommendedTitle)) { - $TitleStatus = "NeedsUpdate" - } - # Look for explicit status in Title Assessment section - elseif ($content -match '(?s)### 📋 Title Assessment.+?\*\*Status:\*\*\s*(✅|❌|⚠️)?\s*(Good|NeedsUpdate|Needs Update)') { - $statusMatch = $Matches[2] -replace '\s+', '' - if ($statusMatch -eq "Good") { - $TitleStatus = "Good" - } else { - $TitleStatus = "NeedsUpdate" - } - } - # Fallback: check for recommended title specifically in title section - elseif ($content -match '(?s)### 📋 Title Assessment.+?\*\*Recommended.*?Title') { - $TitleStatus = "NeedsUpdate" - } else { - $TitleStatus = "Good" - } - Write-Host "ℹ️ Detected TitleStatus: $TitleStatus" -ForegroundColor Cyan - } - - # Extract Current Title - if ([string]::IsNullOrWhiteSpace($CurrentTitle)) { - if ($content -match '\*\*Current.*?Title.*?\*\*:?\s*[`"]?([^`"\n]+)[`"]?') { - $CurrentTitle = $Matches[1].Trim() - } elseif ($content -match 'Current:\s*`([^`]+)`') { - $CurrentTitle = $Matches[1].Trim() - } - if ($CurrentTitle) { - Write-Host "ℹ️ Extracted CurrentTitle: $CurrentTitle" -ForegroundColor Cyan - } - } - - # Extract Description assessment - if ([string]::IsNullOrWhiteSpace($DescriptionStatus)) { - if ($content -match 'Description.*?Excellent|Excellent.*?description') { - $DescriptionStatus = "Excellent" - } elseif ($content -match 'Description.*?Good|Good.*?description') { - $DescriptionStatus = "Good" - } elseif ($content -match 'Needs\s*Rewrite|NeedsRewrite') { - $DescriptionStatus = "NeedsRewrite" - } elseif ($content -match 'Needs\s*Update|NeedsUpdate') { - $DescriptionStatus = "NeedsUpdate" - } else { - $DescriptionStatus = "Good" - } - Write-Host "ℹ️ Detected DescriptionStatus: $DescriptionStatus" -ForegroundColor Cyan - } - - # Extract Title Issues/Reasoning - $titleIssues = "" - if ([string]::IsNullOrWhiteSpace($RecommendedTitle)) { - # No recommended title means title is good - $titleIssues = "Title is clear and follows conventions." - } else { - # Extract the issues list - if ($content -match '(?s)\*\*Issues:\*\*(.+?)(?=\*\*Recommended|\*\*Reasoning|###|$)') { - $titleIssues = $Matches[1].Trim() - } elseif ($content -match '(?s)### 📋 Title Assessment.+?\*\*Issues:\*\*(.+?)(?=\*\*Recommended|###|$)') { - $titleIssues = $Matches[1].Trim() - } - } - - # Extract Description Assessment text - if ([string]::IsNullOrWhiteSpace($DescriptionAssessment)) { - # Try to extract detailed issues from the summary - $issuesSection = "" - if ($content -match '(?s)### 📝 Description Quality Assessment(.+?)(?=###|---|\z)') { - $issuesSection = $Matches[1].Trim() - # Remove the Status line since we already show it in the header - $issuesSection = $issuesSection -replace '(?m)^\*\*Status:\*\*.*$\n?', '' - $issuesSection = $issuesSection.Trim() - } elseif ($content -match '(?s)\*\*Issues:\*\*(.+?)(?=\*\*Recommended|\*\*Action|###|---|\z)') { - $issuesSection = $Matches[1].Trim() - } - - # Try to extract what works - $worksSection = "" - if ($content -match '(?s)### ❌ Issues Found(.+?)(?=###|---|\z)') { - $worksSection = $Matches[1].Trim() - } elseif ($content -match '(?s)\| Issue \| Severity \| Details \|(.+?)(?=###|---|\z)') { - # Extract table content - $worksSection = "**Issues:**`n" + $Matches[1].Trim() - } - - # Combine into assessment - if (-not [string]::IsNullOrWhiteSpace($issuesSection) -or -not [string]::IsNullOrWhiteSpace($worksSection)) { - $DescriptionAssessment = "" - if ($worksSection) { $DescriptionAssessment += $worksSection + "`n`n" } - if ($issuesSection) { $DescriptionAssessment += $issuesSection } - } else { - # Fallback - try to get the verdict section - if ($content -match '(?s)## ⚠️ Verdict:(.+?)(?=###|$)') { - $DescriptionAssessment = $Matches[1].Trim() - } else { - $DescriptionAssessment = "Description needs updates. See details below." - } - } - } - - # Extract Missing Elements - if ([string]::IsNullOrWhiteSpace($MissingElements)) { - if ($content -match '(?s)Missing.*?elements?:(.+?)(?=###|$)') { - $MissingElements = $Matches[1].Trim() - } elseif ($content -match '(?s)Only addition needed:(.+?)(?=###|\*\*Action|$)') { - $MissingElements = $Matches[1].Trim() - } - } - - # Extract Recommended Description - if ([string]::IsNullOrWhiteSpace($RecommendedDescription)) { - # First, try to find a separate recommended-description.md file in same directory - $summaryDir = Split-Path $SummaryFile -Parent - $recommendedDescFile = Join-Path $summaryDir "recommended-description.md" - - if (Test-Path $recommendedDescFile) { - Write-Host "ℹ️ Found recommended-description.md file, loading full content..." -ForegroundColor Cyan - $RecommendedDescription = Get-Content $recommendedDescFile -Raw -Encoding UTF8 - $RecommendedDescription = $RecommendedDescription.Trim() - } - # Try to extract from
section in summary file - elseif ($content -match '(?s)
\s*Click to see proposed description\s*```markdown\s*(.+?)```\s*
') { - Write-Host "ℹ️ Extracted recommended description from expandable section..." -ForegroundColor Cyan - $RecommendedDescription = $Matches[1].Trim() - } - # Otherwise, try to extract from header in summary file - elseif ($content -match '(?s)### Recommended Description(.+?)(?=### [A-Z]|---|\z)') { - $RecommendedDescription = $Matches[1].Trim() - } - # If still empty, check for recommended description block in summary - elseif ($content -match '(?s)```markdown\s* comment - # ======================================================================== - - $MAIN_MARKER = "" - $SECTION_START = "" - $SECTION_END = "" - $LEGACY_MARKER = "" - - # Build the finalize section wrapped in expandable details - $finalizeSection = @" -$SECTION_START -
-📋 Expand PR Finalization Review - ---- - -$titleSection - -$descSection -$codeReviewSection - ---- - -
-$SECTION_END -"@ - - Write-Host "`nUnified mode: injecting into AI Summary comment on #$PRNumber..." -ForegroundColor Yellow - - $existingUnifiedComment = $null - $existingLegacyComment = $null - - try { - $commentsJson = gh api "repos/dotnet/maui/issues/$PRNumber/comments?per_page=100" 2>$null - $comments = $commentsJson | ConvertFrom-Json - - foreach ($comment in $comments) { - if ($comment.body -match [regex]::Escape($MAIN_MARKER)) { - $existingUnifiedComment = $comment - Write-Host "✓ Found unified AI Summary comment (ID: $($comment.id))" -ForegroundColor Green - } - if ($comment.body -match [regex]::Escape($LEGACY_MARKER)) { - $existingLegacyComment = $comment - } - } - } catch { - Write-Host "⚠️ Could not fetch comments: $_" -ForegroundColor Yellow - } - - if ($DryRun) { - if ([string]::IsNullOrWhiteSpace($PreviewFile)) { - $PreviewFile = "CustomAgentLogsTmp/PRState/$PRNumber/ai-summary-comment-preview.md" - } - - $previewDir = Split-Path $PreviewFile -Parent - if (-not (Test-Path $previewDir)) { - New-Item -ItemType Directory -Path $previewDir -Force | Out-Null - } - - $existingPreview = "" - if (Test-Path $PreviewFile) { - $existingPreview = Get-Content $PreviewFile -Raw -Encoding UTF8 - } - - if ($existingPreview -match [regex]::Escape($SECTION_START)) { - $pattern = [regex]::Escape($SECTION_START) + "[\s\S]*?" + [regex]::Escape($SECTION_END) - $finalComment = $existingPreview -replace $pattern, $finalizeSection - } elseif (-not [string]::IsNullOrWhiteSpace($existingPreview)) { - $finalComment = $existingPreview.TrimEnd() + "`n`n" + $finalizeSection - } else { - $finalComment = @" -$MAIN_MARKER - -## 🤖 AI Summary - -$finalizeSection -"@ - } - - Set-Content -Path $PreviewFile -Value "$($finalComment.TrimEnd())`n" -Encoding UTF8 -NoNewline - - Write-Host "`n=== COMMENT PREVIEW ===" -ForegroundColor Yellow - Write-Host $finalComment - Write-Host "`n=== END PREVIEW ===" -ForegroundColor Yellow - Write-Host "`n✅ Preview saved to: $PreviewFile" -ForegroundColor Green - exit 0 - } - - if ($existingUnifiedComment) { - $body = $existingUnifiedComment.body - - if ($body -match [regex]::Escape($SECTION_START)) { - $pattern = [regex]::Escape($SECTION_START) + "[\s\S]*?" + [regex]::Escape($SECTION_END) - $newBody = $body -replace $pattern, $finalizeSection - } else { - $newBody = $body.TrimEnd() + "`n`n" + $finalizeSection - } - - $newBody = $newBody -replace "`n{4,}", "`n`n`n" - - Write-Host "Updating unified comment ID $($existingUnifiedComment.id) with PR finalize section..." -ForegroundColor Yellow - $tempFile = [System.IO.Path]::GetTempFileName() - @{ body = $newBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - - $result = gh api --method PATCH "repos/dotnet/maui/issues/comments/$($existingUnifiedComment.id)" --input $tempFile --jq '.html_url' - Remove-Item $tempFile - Write-Host "✅ PR finalize section added to unified comment: $result" -ForegroundColor Green - } else { - $commentBody = @" -$MAIN_MARKER - -## 🤖 AI Summary - -$finalizeSection -"@ - - Write-Host "Creating new unified comment with PR finalize section on PR #$PRNumber..." -ForegroundColor Yellow - $tempFile = [System.IO.Path]::GetTempFileName() - @{ body = $commentBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - - $result = gh api --method POST "repos/dotnet/maui/issues/$PRNumber/comments" --input $tempFile --jq '.html_url' - Remove-Item $tempFile - Write-Host "✅ Unified comment posted: $result" -ForegroundColor Green - } - - # Clean up legacy standalone finalize comment if it exists - if ($existingLegacyComment) { - Write-Host "🧹 Removing legacy standalone PR Finalization comment (ID: $($existingLegacyComment.id))..." -ForegroundColor Yellow - gh api --method DELETE "repos/dotnet/maui/issues/comments/$($existingLegacyComment.id)" 2>$null | Out-Null - if ($LASTEXITCODE -eq 0) { - Write-Host " ✅ Legacy comment removed" -ForegroundColor Green - } else { - Write-Host " ⚠️ Could not remove legacy comment (non-fatal)" -ForegroundColor Yellow - } - } - -} else { - # ======================================================================== - # STANDALONE MODE (default): Post as separate PR Finalization comment - # ======================================================================== - -$FINALIZE_MARKER = "" - -Write-Host "`nChecking for existing PR Finalization comment on #$PRNumber..." -ForegroundColor Yellow -$existingComment = $null - -try { - $commentsJson = gh api "repos/dotnet/maui/issues/$PRNumber/comments?per_page=100" 2>$null - $comments = $commentsJson | ConvertFrom-Json - - foreach ($comment in $comments) { - if ($comment.body -match [regex]::Escape($FINALIZE_MARKER)) { - $existingComment = $comment - Write-Host "✓ Found existing PR Finalization comment (ID: $($comment.id))" -ForegroundColor Green - break - } - } - - if (-not $existingComment) { - Write-Host "✓ No existing PR Finalization comment found - will create new" -ForegroundColor Yellow - } -} catch { - Write-Host "✓ No existing PR Finalization comment found - will create new" -ForegroundColor Yellow -} - -# Build the full comment body (always replaces existing comment entirely) -$commentBody = @" -## 📋 PR Finalization Review -$FINALIZE_MARKER - -$titleSection - -$descSection -$codeReviewSection -"@ - -if ($DryRun) { - # File-based DryRun: uses separate preview file for finalize (separate comment from unified) - if ([string]::IsNullOrWhiteSpace($PreviewFile)) { - $PreviewFile = "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/pr-finalize-preview.md" - } - - # Ensure directory exists - $previewDir = Split-Path $PreviewFile -Parent - if (-not (Test-Path $previewDir)) { - New-Item -ItemType Directory -Path $previewDir -Force | Out-Null - } - - # For finalize, we replace the entire file (it's a separate comment) - Write-Host "ℹ️ Saving finalize preview to: $PreviewFile" -ForegroundColor Cyan - Set-Content -Path $PreviewFile -Value "$($commentBody.TrimEnd())`n" -Encoding UTF8 -NoNewline - - Write-Host "`n=== COMMENT PREVIEW ===" -ForegroundColor Yellow - Write-Host $commentBody - Write-Host "`n=== END PREVIEW ===" -ForegroundColor Yellow - Write-Host "`n✅ Preview saved to: $PreviewFile" -ForegroundColor Green - Write-Host " Run 'open $PreviewFile' to view in editor" -ForegroundColor Gray - exit 0 -} - -# Write to temp file to avoid shell escaping issues -$tempFile = [System.IO.Path]::GetTempFileName() -@{ body = $commentBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - -if ($existingComment) { - Write-Host "Updating comment ID $($existingComment.id)..." -ForegroundColor Yellow - $result = gh api --method PATCH "repos/dotnet/maui/issues/comments/$($existingComment.id)" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment updated: $result" -ForegroundColor Green -} else { - Write-Host "Posting new comment to PR #$PRNumber..." -ForegroundColor Yellow - $result = gh api --method POST "repos/dotnet/maui/issues/$PRNumber/comments" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment posted: $result" -ForegroundColor Green -} - -Remove-Item $tempFile - -} # end standalone mode diff --git a/.github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 b/.github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 deleted file mode 100644 index 27bd4566bcb0..000000000000 --- a/.github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 +++ /dev/null @@ -1,515 +0,0 @@ -#!/usr/bin/env pwsh -<# -.SYNOPSIS - Posts or updates a try-fix attempts comment on a GitHub Issue or Pull Request. - -.DESCRIPTION - Creates ONE comment for all try-fix attempts with each attempt in a collapsible section. - Uses HTML marker for identification. - - If an existing try-fix comment exists, it will be EDITED with the new attempt added. - Otherwise, a new comment will be created. - - **NEW: Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/try-fix/** - - Format: - ## 🔧 Try-Fix Analysis for Issue #XXXXX - - -
- Attempt 1: Approach Name ✅ PASS - - ... attempt details ... -
- -
- Attempt 2: Different Approach ❌ FAIL - - ... attempt details ... -
- -.PARAMETER IssueNumber - The issue number to post comment on (required unless -TryFixDir provided) - -.PARAMETER AttemptNumber - The attempt number (1, 2, 3, etc.) - auto-detected from TryFixDir if not specified - -.PARAMETER TryFixDir - Path to try-fix attempt directory (e.g., CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/attempt-1) - If provided, all parameters are auto-loaded from files in this directory - -.PARAMETER Approach - Brief description of the fix approach (required unless loading from TryFixDir) - -.PARAMETER RootCause - Description of the root cause identified (optional for failed attempts) - -.PARAMETER FilesChanged - Markdown table or list of files changed (required unless loading from TryFixDir) - -.PARAMETER Status - Status of the attempt: "Compiles", "Pass", "Fail" (required unless loading from TryFixDir) - -.PARAMETER Analysis - Analysis of why it worked or failed (optional) - -.PARAMETER CodeSnippet - Code snippet showing the fix (optional) - -.PARAMETER DryRun - Print comment instead of posting - -.EXAMPLE - # Simplest: Just provide attempt directory (all info auto-loaded) - ./post-try-fix-comment.ps1 -TryFixDir CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/attempt-1 - -.EXAMPLE - # Post all attempts for an issue - ./post-try-fix-comment.ps1 -IssueNumber 27246 - -.EXAMPLE - # Manual parameters - ./post-try-fix-comment.ps1 -IssueNumber 19560 -AttemptNumber 1 ` - -Approach "Change Shadow base class to StyleableElement" ` - -RootCause "Shadow inherits from Element which lacks styling support" ` - -FilesChanged "| File | Changes |`n|------|---------|`n| Shadow.cs | +1/-1 |" ` - -Status "Pass" -#> - -param( - [Parameter(Mandatory=$false)] - [int]$IssueNumber, - - [Parameter(Mandatory=$false)] - [int]$AttemptNumber, - - [Parameter(Mandatory=$false)] - [string]$TryFixDir, - - [Parameter(Mandatory=$false)] - [string]$Approach, - - [Parameter(Mandatory=$false)] - [string]$RootCause, - - [Parameter(Mandatory=$false)] - [string]$FilesChanged, - - [Parameter(Mandatory=$false)] - [ValidateSet("Compiles", "Pass", "Fail", "")] - [string]$Status, - - [Parameter(Mandatory=$false)] - [string]$Analysis, - - [Parameter(Mandatory=$false)] - [string]$CodeSnippet, - - [Parameter(Mandatory=$false)] - [switch]$DryRun, - - [Parameter(Mandatory=$false)] - [string]$PreviewFile -) - -$ErrorActionPreference = "Stop" - -Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan -Write-Host "║ Try-Fix Comment (Post/Update) ║" -ForegroundColor Cyan -Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan - -# ============================================================================ -# AUTO-DISCOVERY FROM DIRECTORIES -# ============================================================================ - -# If TryFixDir provided, load everything from there -if (-not [string]::IsNullOrWhiteSpace($TryFixDir)) { - if (-not (Test-Path $TryFixDir)) { - throw "Try-fix directory not found: $TryFixDir" - } - - # Extract IssueNumber from path (e.g., CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/attempt-1) - if ($TryFixDir -match '[/\\](\d+)[/\\]try-fix') { - if ($IssueNumber -eq 0) { - $IssueNumber = [int]$Matches[1] - Write-Host "ℹ️ Auto-detected IssueNumber: $IssueNumber from path" -ForegroundColor Cyan - } - } - - # Extract AttemptNumber from path (e.g., attempt-1) - if ($TryFixDir -match 'attempt-(\d+)$') { - if ($AttemptNumber -eq 0) { - $AttemptNumber = [int]$Matches[1] - Write-Host "ℹ️ Auto-detected AttemptNumber: $AttemptNumber from path" -ForegroundColor Cyan - } - } - - # Load approach from approach.md or approach.txt - if ([string]::IsNullOrWhiteSpace($Approach)) { - $approachFile = Join-Path $TryFixDir "approach.md" - if (-not (Test-Path $approachFile)) { - $approachFile = Join-Path $TryFixDir "approach.txt" - } - if (Test-Path $approachFile) { - $Approach = Get-Content $approachFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Loaded approach from: $approachFile" -ForegroundColor Cyan - } - } - - # Load result from result.txt - if ([string]::IsNullOrWhiteSpace($Status)) { - $resultFile = Join-Path $TryFixDir "result.txt" - if (Test-Path $resultFile) { - $resultContent = (Get-Content $resultFile -Raw -Encoding UTF8).Trim().ToUpper() - $Status = switch -Regex ($resultContent) { - 'PASS' { "Pass" } - 'FAIL' { "Fail" } - 'COMPILES' { "Compiles" } - default { "Fail" } - } - Write-Host "ℹ️ Loaded status: $Status from result.txt" -ForegroundColor Cyan - } - } - - # Load analysis from analysis.md - if ([string]::IsNullOrWhiteSpace($Analysis)) { - $analysisFile = Join-Path $TryFixDir "analysis.md" - if (Test-Path $analysisFile) { - $Analysis = Get-Content $analysisFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Loaded analysis from: $analysisFile" -ForegroundColor Cyan - } - } - - # Load diff from fix.diff - if ([string]::IsNullOrWhiteSpace($CodeSnippet)) { - $diffFile = Join-Path $TryFixDir "fix.diff" - if (Test-Path $diffFile) { - $CodeSnippet = Get-Content $diffFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Loaded code diff from: $diffFile" -ForegroundColor Cyan - } - } - - # Generate FilesChanged from diff if not provided - if ([string]::IsNullOrWhiteSpace($FilesChanged) -and -not [string]::IsNullOrWhiteSpace($CodeSnippet)) { - $files = $CodeSnippet | Select-String -Pattern "^\+\+\+ b/(.+)$" -AllMatches | - ForEach-Object { $_.Matches.Groups[1].Value } | - Sort-Object -Unique - if ($files) { - $FilesChanged = "| File | Type |`n|------|------|`n" - foreach ($file in $files) { - $FilesChanged += "| ``$file`` | Modified |`n" - } - } else { - $FilesChanged = "_See diff above_" - } - } -} - -# If IssueNumber provided but no TryFixDir, try to find all attempts -if ($IssueNumber -gt 0 -and [string]::IsNullOrWhiteSpace($TryFixDir) -and [string]::IsNullOrWhiteSpace($Approach)) { - $tryFixBase = "CustomAgentLogsTmp/PRState/$IssueNumber/PRAgent/try-fix" - if (-not (Test-Path $tryFixBase)) { - $repoRoot = git rev-parse --show-toplevel 2>$null - if ($repoRoot) { - $tryFixBase = Join-Path $repoRoot "CustomAgentLogsTmp/PRState/$IssueNumber/PRAgent/try-fix" - } - } - - if (Test-Path $tryFixBase) { - $attemptDirs = Get-ChildItem -Path $tryFixBase -Directory | Where-Object { $_.Name -match '^attempt-\d+$' } | Sort-Object { [int]($_.Name -replace 'attempt-', '') } - if ($attemptDirs.Count -gt 0) { - Write-Host "ℹ️ Found $($attemptDirs.Count) attempt(s) in $tryFixBase" -ForegroundColor Cyan - Write-Host " Posting ALL attempts..." -ForegroundColor Cyan - - # Loop through ALL attempts and recurse for each - foreach ($attemptDir in $attemptDirs) { - Write-Host " Processing: $($attemptDir.Name)" -ForegroundColor Gray - & $PSCommandPath -TryFixDir $attemptDir.FullName -DryRun:$DryRun -PreviewFile:$PreviewFile - } - exit 0 - } - } -} - -# Validate required parameters -if ($IssueNumber -eq 0) { - throw "IssueNumber is required. Provide via -IssueNumber or use -TryFixDir with path containing issue number" -} - -if ($AttemptNumber -eq 0) { - throw "AttemptNumber is required. Provide via -AttemptNumber or use -TryFixDir with path like attempt-N" -} - -if ([string]::IsNullOrWhiteSpace($Approach)) { - throw "Approach is required. Provide via -Approach or create approach.md in TryFixDir" -} - -if ([string]::IsNullOrWhiteSpace($Status)) { - throw "Status is required. Provide via -Status or create result.txt in TryFixDir" -} - -if ([string]::IsNullOrWhiteSpace($FilesChanged)) { - $FilesChanged = "_No files changed information available_" -} - -# Status emoji mapping -$statusEmoji = switch ($Status) { - "Pass" { "✅" } - "Fail" { "❌" } - "Compiles" { "🔨" } - default { "⚪" } -} - -# Build the new attempt section - compact format -$attemptSection = @" -
-$statusEmoji Fix $AttemptNumber - -"@ - -# Show brief approach description -if (-not [string]::IsNullOrWhiteSpace($Approach)) { - $attemptSection += "`n$Approach`n`n" -} - -# Only show diff if available -if (-not [string]::IsNullOrWhiteSpace($CodeSnippet)) { - $attemptSection += @" -``````diff -$CodeSnippet -`````` - -"@ -} - -# Show analysis if available (explains why it passed/failed) -if (-not [string]::IsNullOrWhiteSpace($Analysis)) { - $attemptSection += "$Analysis`n" -} - -$attemptSection += @" -
-"@ - -# ============================================================================ -# UNIFIED COMMENT HANDLING -# Uses single comment with section markers -# ============================================================================ - -$MAIN_MARKER = "" -$SECTION_START = "" -$SECTION_END = "" - -Write-Host "`nChecking for existing AI Summary comment on #$IssueNumber..." -ForegroundColor Yellow -$existingComment = $null -$existingBody = "" - -try { - $commentsJson = gh api "repos/dotnet/maui/issues/$IssueNumber/comments" 2>$null - $comments = $commentsJson | ConvertFrom-Json - - foreach ($comment in $comments) { - if ($comment.body -match [regex]::Escape($MAIN_MARKER)) { - $existingComment = $comment - $existingBody = $comment.body - Write-Host "✓ Found existing AI Summary comment (ID: $($comment.id))" -ForegroundColor Green - break - } - } - - if (-not $existingComment) { - Write-Host "✓ No existing AI Summary comment found - will create new" -ForegroundColor Yellow - } -} catch { - Write-Host "✓ No existing AI Summary comment found - will create new" -ForegroundColor Yellow -} - -# Build the try-fix section content -# Count existing attempts to show in summary -$existingAttemptCount = 0 -$passCount = 0 -$failCount = 0 - -# Extract existing try-fix section to preserve previous attempts -$existingTryFixContent = "" -$startPattern = [regex]::Escape($SECTION_START) -$endPattern = [regex]::Escape($SECTION_END) -if ($existingBody -match "(?s)$startPattern(.*?)$endPattern") { - $existingTryFixContent = $Matches[1].Trim() -} - -# Extract just the inner attempt details (strip outer wrapper and headers) -$innerAttempts = "" -if ($existingTryFixContent -match '(?s)
\s*🔧 Try-Fix Analysis.*?\s*(.*?)\s*
\s*$') { - # New format - extract content inside the outer details - $innerAttempts = $Matches[1].Trim() -} elseif ($existingTryFixContent -match '(?s)### 🔧.*?`n`n(.*)') { - # Old header format - $innerAttempts = $Matches[1].Trim() -} else { - # Just use as-is but strip any stray headers - $innerAttempts = $existingTryFixContent -replace "(?s)^### 🔧[^\n]*\n+", "" -} - -# Strip any leading horizontal rules,
tags, or whitespace before the first
-$innerAttempts = $innerAttempts -replace "(?s)^\s*---\s*\n+", "" -$innerAttempts = $innerAttempts -replace "(?s)^(
\s*)+", "" -$innerAttempts = $innerAttempts.TrimStart() - -# Count existing attempts (only count inner
that are Fix attempts) -$existingAttemptCount = ([regex]::Matches($innerAttempts, '
\s*[✅❌🔨⚪]')).Count -$passCount = ([regex]::Matches($innerAttempts, '
\s*✅')).Count -$failCount = ([regex]::Matches($innerAttempts, '
\s*❌')).Count - -# Check if this attempt number already exists and replace it, or add new -$attemptPattern = "(?s)
\s*[✅❌🔨⚪]\s*Fix $AttemptNumber.*?
" -if ($innerAttempts -match $attemptPattern) { - Write-Host "Replacing existing Fix $AttemptNumber..." -ForegroundColor Yellow - $tryFixInnerContent = $innerAttempts -replace $attemptPattern, $attemptSection -} elseif (-not [string]::IsNullOrWhiteSpace($innerAttempts)) { - Write-Host "Adding new Fix $AttemptNumber..." -ForegroundColor Yellow - $tryFixInnerContent = $innerAttempts.TrimEnd() + "`n`n" + $attemptSection -} else { - Write-Host "Creating first fix..." -ForegroundColor Yellow - $tryFixInnerContent = $attemptSection -} - -# Recalculate attempt statistics from the final content to ensure consistency -$totalAttemptCount = ([regex]::Matches($tryFixInnerContent, '
\s*[✅❌🔨⚪]')).Count -$passCount = ([regex]::Matches($tryFixInnerContent, '
\s*✅')).Count -$failCount = ([regex]::Matches($tryFixInnerContent, '
\s*❌')).Count - -# Build summary line with counts -$summaryStatus = if ($passCount -gt 0) { "✅ $passCount passed" } else { "" } -if ($failCount -gt 0) { - if ($summaryStatus -ne "") { $summaryStatus += ", " } - $summaryStatus += "❌ $failCount failed" -} -if ($summaryStatus -eq "") { $summaryStatus = "$totalAttemptCount attempt(s)" } - -# Wrap everything in a single collapsible section -$tryFixContent = @" -
-🔧 Try-Fix Analysis: $summaryStatus
-$tryFixInnerContent - -
-"@ - -# Build the section with markers -$tryFixSection = @" -$SECTION_START -$tryFixContent -$SECTION_END -"@ - -if ($existingComment) { - # Update existing comment - replace or add try-fix section - if ($existingBody -match "(?s)$startPattern.*?$endPattern") { - # Replace existing try-fix section - $commentBody = $existingBody -replace "(?s)$startPattern.*?$endPattern", $tryFixSection - } else { - # Add try-fix section before footer - $footerPattern = "(?s)(---\s*\n+.*?\s*)$" - if ($existingBody -match $footerPattern) { - $commentBody = $existingBody -replace $footerPattern, "`n$tryFixSection`n`n`$1" - } else { - $commentBody = $existingBody.TrimEnd() + "`n`n$tryFixSection" - } - } -} else { - # Create new unified comment - $commentBody = @" -$MAIN_MARKER - -## 🤖 AI Summary - -$tryFixSection -"@ -} - -if ($DryRun) { - # File-based DryRun: mirrors GitHub comment behavior using a local file - if ([string]::IsNullOrWhiteSpace($PreviewFile)) { - $PreviewFile = "CustomAgentLogsTmp/PRState/$IssueNumber/ai-summary-comment-preview.md" - } - - # Ensure directory exists - $previewDir = Split-Path $PreviewFile -Parent - if (-not (Test-Path $previewDir)) { - New-Item -ItemType Directory -Path $previewDir -Force | Out-Null - } - - # Read existing preview file (mimics reading existing GitHub comment) - $existingPreview = "" - if (Test-Path $PreviewFile) { - $existingPreview = Get-Content $PreviewFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Updating existing preview file: $PreviewFile" -ForegroundColor Cyan - } else { - Write-Host "ℹ️ Creating new preview file: $PreviewFile" -ForegroundColor Cyan - } - - # Update or insert the TRY-FIX section - $TRY_FIX_MARKER = "" - $TRY_FIX_END_MARKER = "" - - if ($existingPreview -match [regex]::Escape($TRY_FIX_MARKER)) { - # Extract existing TRY-FIX content to preserve previous attempts (same logic as GitHub comment path) - $startPattern = [regex]::Escape($TRY_FIX_MARKER) - $endPattern = [regex]::Escape($TRY_FIX_END_MARKER) - $existingTryFixPreview = "" - if ($existingPreview -match "(?s)$startPattern(.*?)$endPattern") { - $existingTryFixPreview = $Matches[1].Trim() - } - - # Check if this attempt already exists - replace it, otherwise append - $attemptPatternPreview = "(?s)
\s*.*?(Attempt $AttemptNumber`:|Fix $AttemptNumber).*?
" - if ($existingTryFixPreview -match $attemptPatternPreview) { - Write-Host "Replacing existing Fix $AttemptNumber in preview..." -ForegroundColor Yellow - $updatedTryFixContent = $existingTryFixPreview -replace $attemptPatternPreview, $attemptSection - $tryFixSectionUpdated = "$SECTION_START`n$tryFixHeader$updatedTryFixContent`n$SECTION_END" - } else { - Write-Host "Adding Fix $AttemptNumber to preview..." -ForegroundColor Yellow - # Remove header if present to avoid duplication - $existingTryFixPreview = $existingTryFixPreview -replace "^### � (Try-Fix Analysis|Fix Attempts)\s*`n*", "" - $updatedTryFixContent = $tryFixHeader + $existingTryFixPreview.TrimEnd() + "`n`n" + $attemptSection - $tryFixSectionUpdated = "$SECTION_START`n$updatedTryFixContent`n$SECTION_END" - } - - # Replace the section in the preview - $pattern = [regex]::Escape($TRY_FIX_MARKER) + "[\s\S]*?" + [regex]::Escape($TRY_FIX_END_MARKER) - $finalComment = $existingPreview -replace $pattern, $tryFixSectionUpdated - } elseif (-not [string]::IsNullOrWhiteSpace($existingPreview)) { - # Append TRY-FIX section to existing content - $finalComment = $existingPreview.TrimEnd() + "`n`n" + $tryFixSection - } else { - # New file - use full comment body - $finalComment = $commentBody - } - - # Write to preview file - Set-Content -Path $PreviewFile -Value "$($finalComment.TrimEnd())`n" -Encoding UTF8 -NoNewline - - Write-Host "`n=== COMMENT PREVIEW ===" -ForegroundColor Yellow - Write-Host $finalComment - Write-Host "`n=== END PREVIEW ===" -ForegroundColor Yellow - Write-Host "`n✅ Preview saved to: $PreviewFile" -ForegroundColor Green - Write-Host " Run 'open $PreviewFile' to view in editor" -ForegroundColor Gray - exit 0 -} - -# Write to temp file to avoid shell escaping issues -$tempFile = [System.IO.Path]::GetTempFileName() -@{ body = $commentBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - -if ($existingComment) { - Write-Host "Updating comment ID $($existingComment.id)..." -ForegroundColor Yellow - $result = gh api --method PATCH "repos/dotnet/maui/issues/comments/$($existingComment.id)" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment updated: $result" -ForegroundColor Green -} else { - Write-Host "Posting new comment to issue #$IssueNumber..." -ForegroundColor Yellow - $result = gh api --method POST "repos/dotnet/maui/issues/$IssueNumber/comments" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment posted: $result" -ForegroundColor Green -} - -Remove-Item $tempFile diff --git a/.github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 b/.github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 deleted file mode 100644 index cc78274e74a2..000000000000 --- a/.github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 +++ /dev/null @@ -1,331 +0,0 @@ -#!/usr/bin/env pwsh -<# -.SYNOPSIS - Posts or updates the verification test results in the unified AI Summary comment. - -.DESCRIPTION - Reads verification report from CustomAgentLogsTmp and posts/updates - the VERIFY-TESTS section in the unified AI Summary comment. - - Uses the same marker and - section markers as other comment scripts. - -.PARAMETER PRNumber - The PR number to post comment on (required unless ReportFile provided) - -.PARAMETER ReportFile - Path to verification-report.md file. If not provided, auto-discovers from - CustomAgentLogsTmp/PRState/{PRNumber}/verify-tests-fail/verification-report.md - -.PARAMETER Status - Overall verification status: "Passed", "Failed" (auto-detected from report if not provided) - -.PARAMETER Platform - Platform tested (auto-detected from report if not provided) - -.PARAMETER Mode - Verification mode: "FailureOnly", "FullVerification" (auto-detected from report if not provided) - -.PARAMETER Summary - Brief summary of results (auto-generated if not provided) - -.PARAMETER DryRun - Print comment instead of posting - -.EXAMPLE - # Simplest: Just provide PR number (auto-loads from CustomAgentLogsTmp) - ./post-verify-tests-comment.ps1 -PRNumber 27246 - -.EXAMPLE - # Or provide report file path - ./post-verify-tests-comment.ps1 -ReportFile CustomAgentLogsTmp/PRState/27246/verify-tests-fail/verification-report.md - -.EXAMPLE - # Manual parameters - ./post-verify-tests-comment.ps1 -PRNumber 27246 -Status "Passed" -Platform "android" -Mode "FullVerification" -#> - -param( - [Parameter(Mandatory=$false)] - [int]$PRNumber, - - [Parameter(Mandatory=$false)] - [string]$ReportFile, - - [Parameter(Mandatory=$false)] - [ValidateSet("Passed", "Failed", "")] - [string]$Status, - - [Parameter(Mandatory=$false)] - [string]$Platform, - - [Parameter(Mandatory=$false)] - [ValidateSet("FailureOnly", "FullVerification", "")] - [string]$Mode, - - [Parameter(Mandatory=$false)] - [string]$Summary, - - [Parameter(Mandatory=$false)] - [switch]$DryRun, - - [Parameter(Mandatory=$false)] - [string]$PreviewFile -) - -$ErrorActionPreference = "Stop" - -Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan -Write-Host "║ Verify-Tests Comment (Post/Update) ║" -ForegroundColor Cyan -Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan - -# ============================================================================ -# AUTO-DISCOVERY FROM REPORT FILE -# ============================================================================ - -# If PRNumber provided but no ReportFile, try to find it -if ($PRNumber -gt 0 -and [string]::IsNullOrWhiteSpace($ReportFile)) { - $reportPath = "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/gate/verify-tests-fail/verification-report.md" - if (-not (Test-Path $reportPath)) { - $repoRoot = git rev-parse --show-toplevel 2>$null - if ($repoRoot) { - $reportPath = Join-Path $repoRoot "CustomAgentLogsTmp/PRState/$PRNumber/PRAgent/gate/verify-tests-fail/verification-report.md" - } - } - - if (Test-Path $reportPath) { - $ReportFile = $reportPath - Write-Host "ℹ️ Auto-discovered report file: $ReportFile" -ForegroundColor Cyan - } -} - -# If ReportFile provided, parse it -if (-not [string]::IsNullOrWhiteSpace($ReportFile)) { - if (-not (Test-Path $ReportFile)) { - throw "Report file not found: $ReportFile" - } - - $reportContent = Get-Content $ReportFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Loading from report file: $ReportFile" -ForegroundColor Cyan - - # Extract PRNumber from path if not provided - if ($PRNumber -eq 0 -and $ReportFile -match '[/\\](\d+)[/\\]verify-tests-fail') { - $PRNumber = [int]$Matches[1] - Write-Host "ℹ️ Auto-detected PRNumber: $PRNumber from path" -ForegroundColor Cyan - } - - # Extract Status from report - if ([string]::IsNullOrWhiteSpace($Status)) { - if ($reportContent -match 'VERIFICATION PASSED|✅\s*PASSED') { - $Status = "Passed" - } elseif ($reportContent -match 'VERIFICATION FAILED|❌\s*FAILED') { - $Status = "Failed" - } else { - $Status = "Unknown" - } - Write-Host "ℹ️ Detected Status: $Status" -ForegroundColor Cyan - } - - # Extract Platform from report - if ([string]::IsNullOrWhiteSpace($Platform)) { - if ($reportContent -match 'Platform[:\s]+(\w+)') { - $Platform = $Matches[1] - } elseif ($reportContent -match '(android|ios|catalyst|windows)' ) { - $Platform = $Matches[1] - } - if ($Platform) { - Write-Host "ℹ️ Detected Platform: $Platform" -ForegroundColor Cyan - } - } - - # Extract Mode from report - if ([string]::IsNullOrWhiteSpace($Mode)) { - if ($reportContent -match 'Full Verification|FAIL without fix.*PASS with fix') { - $Mode = "FullVerification" - } elseif ($reportContent -match 'Verify Failure Only|tests.*FAILED as expected') { - $Mode = "FailureOnly" - } - if ($Mode) { - Write-Host "ℹ️ Detected Mode: $Mode" -ForegroundColor Cyan - } - } - - # Use report content as summary if not provided - if ([string]::IsNullOrWhiteSpace($Summary)) { - # Extract key results from report, excluding verbose "Test Results Details" section - if ($reportContent -match '(?s)^(.*?)(?=####\s*Test Results Details)') { - $Summary = $Matches[1].TrimEnd() - } else { - $Summary = $reportContent - } - } -} - -# Validate required parameters -if ($PRNumber -eq 0) { - throw "PRNumber is required. Provide via -PRNumber or use -ReportFile with path containing PR number" -} - -if ([string]::IsNullOrWhiteSpace($Status)) { - throw "Status is required. Provide via -Status or use -ReportFile with verification results" -} - -# Generate summary if not provided -if ([string]::IsNullOrWhiteSpace($Summary)) { - $statusEmoji = if ($Status -eq "Passed") { "✅" } else { "❌" } - $modeDesc = if ($Mode -eq "FullVerification") { "Full verification (FAIL without fix, PASS with fix)" } else { "Failure only (tests FAIL as expected)" } - $Summary = @" -**Status**: $statusEmoji $Status - -**Mode**: $modeDesc -**Platform**: $Platform - -_Run `verify-tests-fail.ps1` for full details._ -"@ -} - -# Status emoji -$statusEmoji = if ($Status -eq "Passed") { "✅ passed" } else { "❌ failed" } -$modeDesc = if ($Mode -eq "FullVerification") { "Full Verification" } else { "Failure Only" } - -# Build verification section content - wrapped in collapsible with status in summary -$verifyContent = @" -
-🚦 Test Verification: $statusEmoji - -$Summary - -
-"@ - -# ============================================================================ -# UNIFIED COMMENT HANDLING -# Uses single comment with section markers -# ============================================================================ - -$MAIN_MARKER = "" -$SECTION_START = "" -$SECTION_END = "" - -Write-Host "`nChecking for existing AI Summary comment on #$PRNumber..." -ForegroundColor Yellow -$existingComment = $null -$existingBody = "" - -try { - $commentsJson = gh api "repos/dotnet/maui/issues/$PRNumber/comments" 2>$null - $comments = $commentsJson | ConvertFrom-Json - - foreach ($comment in $comments) { - if ($comment.body -match [regex]::Escape($MAIN_MARKER)) { - $existingComment = $comment - $existingBody = $comment.body - Write-Host "✓ Found existing AI Summary comment (ID: $($comment.id))" -ForegroundColor Green - break - } - } - - if (-not $existingComment) { - Write-Host "✓ No existing AI Summary comment found - will create new" -ForegroundColor Yellow - } -} catch { - Write-Host "✓ No existing AI Summary comment found - will create new" -ForegroundColor Yellow -} - -# Build the section with markers -$verifySection = @" -$SECTION_START -$verifyContent -$SECTION_END -"@ - -$startPattern = [regex]::Escape($SECTION_START) -$endPattern = [regex]::Escape($SECTION_END) - -if ($existingComment) { - # Update existing comment - replace or add verify-tests section - if ($existingBody -match "(?s)$startPattern.*?$endPattern") { - # Replace existing verify-tests section - $commentBody = $existingBody -replace "(?s)$startPattern.*?$endPattern", $verifySection - } else { - # Add verify-tests section before footer - $footerPattern = "(?s)(---\s*\n+.*?\s*)$" - if ($existingBody -match $footerPattern) { - $commentBody = $existingBody -replace $footerPattern, "`n$verifySection`n`n`$1" - } else { - $commentBody = $existingBody.TrimEnd() + "`n`n$verifySection" - } - } -} else { - # Create new unified comment - $commentBody = @" -$MAIN_MARKER - -## 🤖 AI Summary - -$verifySection -"@ -} - -if ($DryRun) { - # File-based DryRun: mirrors GitHub comment behavior using a local file - if ([string]::IsNullOrWhiteSpace($PreviewFile)) { - $PreviewFile = "CustomAgentLogsTmp/PRState/$PRNumber/ai-summary-comment-preview.md" - } - - # Ensure directory exists - $previewDir = Split-Path $PreviewFile -Parent - if (-not (Test-Path $previewDir)) { - New-Item -ItemType Directory -Path $previewDir -Force | Out-Null - } - - # Read existing preview file (mimics reading existing GitHub comment) - $existingPreview = "" - if (Test-Path $PreviewFile) { - $existingPreview = Get-Content $PreviewFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Updating existing preview file: $PreviewFile" -ForegroundColor Cyan - } else { - Write-Host "ℹ️ Creating new preview file: $PreviewFile" -ForegroundColor Cyan - } - - # Update or insert the VERIFY-TESTS section - $VERIFY_MARKER = "" - $VERIFY_END_MARKER = "" - - if ($existingPreview -match [regex]::Escape($VERIFY_MARKER)) { - # Replace existing VERIFY-TESTS section - $pattern = [regex]::Escape($VERIFY_MARKER) + "[\s\S]*?" + [regex]::Escape($VERIFY_END_MARKER) - $finalComment = $existingPreview -replace $pattern, $verifySection - } elseif (-not [string]::IsNullOrWhiteSpace($existingPreview)) { - # Append VERIFY-TESTS section to existing content - $finalComment = $existingPreview.TrimEnd() + "`n`n" + $verifySection - } else { - # New file - use full comment body - $finalComment = $commentBody - } - - # Write to preview file - Set-Content -Path $PreviewFile -Value "$($finalComment.TrimEnd())`n" -Encoding UTF8 -NoNewline - - Write-Host "`n=== COMMENT PREVIEW ===" -ForegroundColor Yellow - Write-Host $finalComment - Write-Host "`n=== END PREVIEW ===" -ForegroundColor Yellow - Write-Host "`n✅ Preview saved to: $PreviewFile" -ForegroundColor Green - Write-Host " Run 'open $PreviewFile' to view in editor" -ForegroundColor Gray - exit 0 -} - -# Write to temp file to avoid shell escaping issues -$tempFile = [System.IO.Path]::GetTempFileName() -@{ body = $commentBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - -if ($existingComment) { - Write-Host "Updating comment ID $($existingComment.id)..." -ForegroundColor Yellow - $result = gh api --method PATCH "repos/dotnet/maui/issues/comments/$($existingComment.id)" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment updated: $result" -ForegroundColor Green -} else { - Write-Host "Posting new comment to PR #$PRNumber..." -ForegroundColor Yellow - $result = gh api --method POST "repos/dotnet/maui/issues/$PRNumber/comments" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment posted: $result" -ForegroundColor Green -} - -Remove-Item $tempFile diff --git a/.github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 b/.github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 deleted file mode 100644 index da3972cd92e4..000000000000 --- a/.github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 +++ /dev/null @@ -1,487 +0,0 @@ -#!/usr/bin/env pwsh -<# -.SYNOPSIS - Posts or updates a write-tests comment on a GitHub Issue or Pull Request. - -.DESCRIPTION - Creates ONE comment for all test-writing attempts with each attempt in a collapsible section. - Uses HTML marker for identification. - - **NEW: Auto-loads from CustomAgentLogsTmp/PRState/{IssueNumber}/write-tests/** - - If an existing write-tests comment exists, it will be EDITED with the new attempt added. - Otherwise, a new comment will be created. - - Format: - ## 🧪 Test Writing for Issue #XXXXX - - -
- Attempt 1: Test description ✅ Verified - - ... test details ... -
- -
- Attempt 2: Different approach ❌ Failed - - ... test details ... -
- -.PARAMETER IssueNumber - The issue number to post comment on (required unless TestDir provided) - -.PARAMETER AttemptNumber - The attempt number (1, 2, 3, etc.) - auto-detected from TestDir if not specified - -.PARAMETER TestDir - Path to test attempt directory (e.g., CustomAgentLogsTmp/PRState/27246/write-tests/attempt-1) - If provided, all parameters are auto-loaded from files in this directory - -.PARAMETER TestDescription - Brief description of what the test verifies (required unless loading from TestDir) - -.PARAMETER HostAppFile - Path to the HostApp test page file (required unless loading from TestDir) - -.PARAMETER TestFile - Path to the NUnit test file (required unless loading from TestDir) - -.PARAMETER TestMethod - Name of the test method (required unless loading from TestDir) - -.PARAMETER Category - UITestCategories category used (required unless loading from TestDir) - -.PARAMETER VerificationStatus - Status: "Verified" (tests fail without fix), "Failed" (tests don't catch bug), "Unverified" (not yet run) (required unless loading from TestDir) - -.PARAMETER Platforms - Platforms the test runs on (e.g., "All", "iOS, Android") (optional) - -.PARAMETER Notes - Additional notes about the test (optional) - -.PARAMETER DryRun - Print comment instead of posting - -.EXAMPLE - # Simplest: Just provide test directory (all info auto-loaded) - ./post-write-tests-comment.ps1 -TestDir CustomAgentLogsTmp/PRState/27246/write-tests/attempt-1 - -.EXAMPLE - # Or just provide issue number (auto-discovers latest attempt) - ./post-write-tests-comment.ps1 -IssueNumber 27246 - -.EXAMPLE - # Manual parameters - ./post-write-tests-comment.ps1 -IssueNumber 33331 -AttemptNumber 1 ` - -TestDescription "Verifies Picker.IsOpen property changes correctly" ` - -HostAppFile "src/Controls/tests/TestCases.HostApp/Issues/Issue33331.cs" ` - -TestFile "src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue33331.cs" ` - -TestMethod "PickerIsOpenPropertyChanges" ` - -Category "Picker" ` - -VerificationStatus "Verified" -#> - -param( - [Parameter(Mandatory=$false)] - [int]$IssueNumber, - - [Parameter(Mandatory=$false)] - [int]$AttemptNumber, - - [Parameter(Mandatory=$false)] - [string]$TestDir, - - [Parameter(Mandatory=$false)] - [string]$TestDescription, - - [Parameter(Mandatory=$false)] - [string]$HostAppFile, - - [Parameter(Mandatory=$false)] - [string]$TestFile, - - [Parameter(Mandatory=$false)] - [string]$TestMethod, - - [Parameter(Mandatory=$false)] - [string]$Category, - - [Parameter(Mandatory=$false)] - [ValidateSet("Verified", "Failed", "Unverified", "")] - [string]$VerificationStatus, - - [Parameter(Mandatory=$false)] - [string]$Platforms = "All", - - [Parameter(Mandatory=$false)] - [string]$Notes, - - [Parameter(Mandatory=$false)] - [switch]$DryRun, - - [Parameter(Mandatory=$false)] - [string]$PreviewFile -) - -$ErrorActionPreference = "Stop" - -Write-Host "╔═══════════════════════════════════════════════════════════╗" -ForegroundColor Cyan -Write-Host "║ Write-Tests Comment (Post/Update) ║" -ForegroundColor Cyan -Write-Host "╚═══════════════════════════════════════════════════════════╝" -ForegroundColor Cyan - -# ============================================================================ -# AUTO-DISCOVERY FROM DIRECTORIES -# ============================================================================ - -# If TestDir provided, load everything from there -if (-not [string]::IsNullOrWhiteSpace($TestDir)) { - if (-not (Test-Path $TestDir)) { - throw "Test directory not found: $TestDir" - } - - # Extract IssueNumber from path (e.g., CustomAgentLogsTmp/PRState/27246/write-tests/attempt-1) - if ($TestDir -match '[/\\](\d+)[/\\]write-tests') { - if ($IssueNumber -eq 0) { - $IssueNumber = [int]$Matches[1] - Write-Host "ℹ️ Auto-detected IssueNumber: $IssueNumber from path" -ForegroundColor Cyan - } - } - - # Extract AttemptNumber from path (e.g., attempt-1) - if ($TestDir -match 'attempt-(\d+)$') { - if ($AttemptNumber -eq 0) { - $AttemptNumber = [int]$Matches[1] - Write-Host "ℹ️ Auto-detected AttemptNumber: $AttemptNumber from path" -ForegroundColor Cyan - } - } - - # Load test description from description.md or description.txt - if ([string]::IsNullOrWhiteSpace($TestDescription)) { - $descFile = Join-Path $TestDir "description.md" - if (-not (Test-Path $descFile)) { - $descFile = Join-Path $TestDir "description.txt" - } - if (Test-Path $descFile) { - $TestDescription = (Get-Content $descFile -Raw -Encoding UTF8).Trim() - Write-Host "ℹ️ Loaded description from: $descFile" -ForegroundColor Cyan - } - } - - # Load test info from test-info.json or test-info.txt - $infoFile = Join-Path $TestDir "test-info.json" - if (Test-Path $infoFile) { - $info = Get-Content $infoFile -Raw | ConvertFrom-Json - if ([string]::IsNullOrWhiteSpace($HostAppFile) -and $info.HostAppFile) { $HostAppFile = $info.HostAppFile } - if ([string]::IsNullOrWhiteSpace($TestFile) -and $info.TestFile) { $TestFile = $info.TestFile } - if ([string]::IsNullOrWhiteSpace($TestMethod) -and $info.TestMethod) { $TestMethod = $info.TestMethod } - if ([string]::IsNullOrWhiteSpace($Category) -and $info.Category) { $Category = $info.Category } - Write-Host "ℹ️ Loaded test info from: $infoFile" -ForegroundColor Cyan - } - - # Load verification status from result.txt - if ([string]::IsNullOrWhiteSpace($VerificationStatus)) { - $resultFile = Join-Path $TestDir "result.txt" - if (Test-Path $resultFile) { - $resultContent = (Get-Content $resultFile -Raw -Encoding UTF8).Trim().ToUpper() - $VerificationStatus = switch -Regex ($resultContent) { - 'VERIFIED' { "Verified" } - 'PASS' { "Verified" } - 'FAILED' { "Failed" } - 'FAIL' { "Failed" } - default { "Unverified" } - } - Write-Host "ℹ️ Loaded status: $VerificationStatus from result.txt" -ForegroundColor Cyan - } - } - - # Load notes from notes.md - if ([string]::IsNullOrWhiteSpace($Notes)) { - $notesFile = Join-Path $TestDir "notes.md" - if (Test-Path $notesFile) { - $Notes = Get-Content $notesFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Loaded notes from: $notesFile" -ForegroundColor Cyan - } - } -} - -# If IssueNumber provided but no TestDir, try to find all attempts -if ($IssueNumber -gt 0 -and [string]::IsNullOrWhiteSpace($TestDir) -and [string]::IsNullOrWhiteSpace($TestDescription)) { - $testBase = "CustomAgentLogsTmp/PRState/$IssueNumber/write-tests" - if (-not (Test-Path $testBase)) { - $repoRoot = git rev-parse --show-toplevel 2>$null - if ($repoRoot) { - $testBase = Join-Path $repoRoot "CustomAgentLogsTmp/PRState/$IssueNumber/write-tests" - } - } - - if (Test-Path $testBase) { - $attemptDirs = Get-ChildItem -Path $testBase -Directory | Where-Object { $_.Name -match '^attempt-\d+$' } | Sort-Object Name - if ($attemptDirs.Count -gt 0) { - Write-Host "ℹ️ Found $($attemptDirs.Count) attempt(s) in $testBase" -ForegroundColor Cyan - Write-Host " Use -TestDir to post a specific attempt, or posting latest..." -ForegroundColor Cyan - - # Post the latest attempt - $latestAttempt = $attemptDirs | Sort-Object { [int]($_.Name -replace 'attempt-', '') } | Select-Object -Last 1 - $TestDir = $latestAttempt.FullName - - # Recurse with the discovered directory - & $PSCommandPath -TestDir $TestDir -DryRun:$DryRun - exit 0 - } - } -} - -# Validate required parameters -if ($IssueNumber -eq 0) { - throw "IssueNumber is required. Provide via -IssueNumber or use -TestDir with path containing issue number" -} - -if ($AttemptNumber -eq 0) { - throw "AttemptNumber is required. Provide via -AttemptNumber or use -TestDir with path like attempt-N" -} - -if ([string]::IsNullOrWhiteSpace($TestDescription)) { - throw "TestDescription is required. Provide via -TestDescription or create description.md in TestDir" -} - -if ([string]::IsNullOrWhiteSpace($HostAppFile)) { - throw "HostAppFile is required. Provide via -HostAppFile or add to test-info.json in TestDir" -} - -if ([string]::IsNullOrWhiteSpace($TestFile)) { - throw "TestFile is required. Provide via -TestFile or add to test-info.json in TestDir" -} - -if ([string]::IsNullOrWhiteSpace($TestMethod)) { - throw "TestMethod is required. Provide via -TestMethod or add to test-info.json in TestDir" -} - -if ([string]::IsNullOrWhiteSpace($Category)) { - throw "Category is required. Provide via -Category or add to test-info.json in TestDir" -} - -if ([string]::IsNullOrWhiteSpace($VerificationStatus)) { - $VerificationStatus = "Unverified" -} - -# Status emoji mapping -$statusEmoji = switch ($VerificationStatus) { - "Verified" { "✅ Verified" } - "Failed" { "❌ Failed" } - "Unverified" { "⏳ Unverified" } - default { $VerificationStatus } -} - -# Build the new attempt section (collapsible) -$attemptSection = @" -
-Attempt $AttemptNumber`: $TestDescription $statusEmoji - - -### Test Details - -| Property | Value | -|----------|-------| -| **Test Method** | ``$TestMethod`` | -| **Category** | ``UITestCategories.$Category`` | -| **Platforms** | $Platforms | -| **Status** | $statusEmoji | - -### Files Created - -
-📄 HostApp Test Page - Click to expand code - -**File:** ``$HostAppFile`` - -``````csharp -$(if (Test-Path $HostAppFile) { Get-Content $HostAppFile -Raw } else { "File not found: $HostAppFile" }) -`````` - -
- -
-🧪 NUnit Test - Click to expand code - -**File:** ``$TestFile`` - -``````csharp -$(if (Test-Path $TestFile) { Get-Content $TestFile -Raw } else { "File not found: $TestFile" }) -`````` - -
- -"@ - -if (-not [string]::IsNullOrWhiteSpace($Notes)) { - $attemptSection += @" - -### Notes - -$Notes - -"@ -} - -$attemptSection += @" -
-"@ - -# ============================================================================ -# UNIFIED COMMENT HANDLING -# Uses single comment with section markers -# ============================================================================ - -$MAIN_MARKER = "" -$SECTION_START = "" -$SECTION_END = "" - -Write-Host "`nChecking for existing AI Summary comment on #$IssueNumber..." -ForegroundColor Yellow -$existingComment = $null -$existingBody = "" - -try { - $commentsJson = gh api "repos/dotnet/maui/issues/$IssueNumber/comments" 2>$null - $comments = $commentsJson | ConvertFrom-Json - - foreach ($comment in $comments) { - if ($comment.body -match [regex]::Escape($MAIN_MARKER)) { - $existingComment = $comment - $existingBody = $comment.body - Write-Host "✓ Found existing AI Summary comment (ID: $($comment.id))" -ForegroundColor Green - break - } - } - - if (-not $existingComment) { - Write-Host "✓ No existing AI Summary comment found - will create new" -ForegroundColor Yellow - } -} catch { - Write-Host "✓ No existing AI Summary comment found - will create new" -ForegroundColor Yellow -} - -# Build the write-tests section content -$writeTestsHeader = "### 🧪 Test Writing`n`n" - -# Extract existing write-tests section to preserve previous attempts -$existingWriteTestsContent = "" -$startPattern = [regex]::Escape($SECTION_START) -$endPattern = [regex]::Escape($SECTION_END) -if ($existingBody -match "(?s)$startPattern(.*?)$endPattern") { - $existingWriteTestsContent = $Matches[1].Trim() -} - -# Check if this attempt number already exists and replace it, or add new -$attemptPattern = "(?s)
\s*Attempt $AttemptNumber`:.*?
" -if ($existingWriteTestsContent -match $attemptPattern) { - Write-Host "Replacing existing Attempt $AttemptNumber..." -ForegroundColor Yellow - $tryFixContent = $existingWriteTestsContent -replace $attemptPattern, $attemptSection -} elseif (-not [string]::IsNullOrWhiteSpace($existingWriteTestsContent)) { - Write-Host "Adding new Attempt $AttemptNumber..." -ForegroundColor Yellow - # Remove header if present to avoid duplication - $existingWriteTestsContent = $existingWriteTestsContent -replace "^### 🧪 Test Writing\s*`n*", "" - $writeTestsContent = $writeTestsHeader + $existingWriteTestsContent.TrimEnd() + "`n`n" + $attemptSection -} else { - Write-Host "Creating first attempt..." -ForegroundColor Yellow - $writeTestsContent = $writeTestsHeader + $attemptSection -} - -# Build the section with markers -$writeTestsSection = @" -$SECTION_START -$writeTestsContent -$SECTION_END -"@ - -if ($existingComment) { - # Update existing comment - replace or add write-tests section - if ($existingBody -match "(?s)$startPattern.*?$endPattern") { - # Replace existing write-tests section - $commentBody = $existingBody -replace "(?s)$startPattern.*?$endPattern", $writeTestsSection - } else { - # Add write-tests section before footer - $footerPattern = "(?s)(---\s*\n+.*?\s*)$" - if ($existingBody -match $footerPattern) { - $commentBody = $existingBody -replace $footerPattern, "`n$writeTestsSection`n`n`$1" - } else { - $commentBody = $existingBody.TrimEnd() + "`n`n$writeTestsSection" - } - } -} else { - # Create new unified comment - $commentBody = @" -$MAIN_MARKER - -## 🤖 AI Summary - -$writeTestsSection -"@ -} - -if ($DryRun) { - # File-based DryRun: mirrors GitHub comment behavior using a local file - if ([string]::IsNullOrWhiteSpace($PreviewFile)) { - $PreviewFile = "CustomAgentLogsTmp/PRState/$IssueNumber/ai-summary-comment-preview.md" - } - - # Ensure directory exists - $previewDir = Split-Path $PreviewFile -Parent - if (-not (Test-Path $previewDir)) { - New-Item -ItemType Directory -Path $previewDir -Force | Out-Null - } - - # Read existing preview file - $existingPreview = "" - if (Test-Path $PreviewFile) { - $existingPreview = Get-Content $PreviewFile -Raw -Encoding UTF8 - Write-Host "ℹ️ Updating existing preview file: $PreviewFile" -ForegroundColor Cyan - } else { - Write-Host "ℹ️ Creating new preview file: $PreviewFile" -ForegroundColor Cyan - } - - # Update or insert the WRITE-TESTS section - $sectionMarker = "" - $sectionEndMarker = "" - $wrappedSection = "$sectionMarker`n$writeTestsSection`n$sectionEndMarker" - - if ($existingPreview -match [regex]::Escape($sectionMarker)) { - # Replace existing WRITE-TESTS section - $pattern = [regex]::Escape($sectionMarker) + "[\s\S]*?" + [regex]::Escape($sectionEndMarker) - $finalComment = $existingPreview -replace $pattern, $wrappedSection - } elseif (-not [string]::IsNullOrWhiteSpace($existingPreview)) { - # Append WRITE-TESTS section to existing content - $finalComment = $existingPreview.TrimEnd() + "`n`n" + $wrappedSection - } else { - # New file - use full comment body with section markers - $finalComment = $commentBody.Replace($writeTestsSection, $wrappedSection) - } - - # Write to preview file - Set-Content -Path $PreviewFile -Value "$($finalComment.TrimEnd())`n" -Encoding UTF8 -NoNewline - - Write-Host "`n=== COMMENT PREVIEW ===" -ForegroundColor Yellow - Write-Host $finalComment - Write-Host "`n=== END PREVIEW ===" -ForegroundColor Yellow - Write-Host "`n✅ Preview saved to: $PreviewFile" -ForegroundColor Green - Write-Host " Run 'open $PreviewFile' to view in editor" -ForegroundColor Gray - exit 0 -} - -# Write to temp file to avoid shell escaping issues -$tempFile = [System.IO.Path]::GetTempFileName() -@{ body = $commentBody } | ConvertTo-Json -Depth 10 | Set-Content -Path $tempFile -Encoding UTF8 - -if ($existingComment) { - Write-Host "Updating comment ID $($existingComment.id)..." -ForegroundColor Yellow - $result = gh api --method PATCH "repos/dotnet/maui/issues/comments/$($existingComment.id)" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment updated: $result" -ForegroundColor Green -} else { - Write-Host "Posting new comment to issue #$IssueNumber..." -ForegroundColor Yellow - $result = gh api --method POST "repos/dotnet/maui/issues/$IssueNumber/comments" --input $tempFile --jq '.html_url' - Write-Host "✅ Comment posted: $result" -ForegroundColor Green -} - -Remove-Item $tempFile diff --git a/.github/skills/pr-finalize/SKILL.md b/.github/skills/pr-finalize/SKILL.md index d2efd2d25ebf..28ba6916dd8b 100644 --- a/.github/skills/pr-finalize/SKILL.md +++ b/.github/skills/pr-finalize/SKILL.md @@ -7,7 +7,7 @@ description: Finalizes any PR for merge by verifying title/description match imp Ensures PR title and description accurately reflect the implementation, and performs a **code review** for best practices before merge. -**Standalone skill** - Can be used on any PR, not just PRs created by the pr agent. +**Standalone skill** - Can be used on any PR, not just PRs reviewed by the pr-review skill. ## Two-Phase Workflow @@ -33,13 +33,13 @@ Ensures PR title and description accurately reflect the implementation, and perf | Action | Allowed? | Why | |--------|----------|-----| -| `gh pr review --comment` | ❌ **NEVER** | Use ai-summary-comment skill instead | -| `gh pr comment` | ❌ **NEVER** | Use ai-summary-comment skill instead | +| `gh pr review --comment` | ❌ **NEVER** | Review-PR.ps1 handles posting via scripts | +| `gh pr comment` | ❌ **NEVER** | Review-PR.ps1 handles posting via scripts | | Analyze and report findings | ✅ **YES** | This is the skill's purpose | **Correct workflow:** -1. **This skill**: Analyze PR, produce findings in your response to the user -2. **User explicitly asks to post comment**: Then invoke `ai-summary-comment` skill +1. **This skill**: Analyze PR, produce findings and write to `pr-finalize-summary.md` +2. **Review-PR.ps1** calls `post-pr-finalize-comment.ps1` to post the summary **Only humans control when comments are posted.** Your job is to analyze and present findings. @@ -371,13 +371,13 @@ gh pr diff XXXXX -- path/to/file.cs | Action | Allowed? | Why | |--------|----------|-----| -| `gh pr review --comment` | ❌ **NEVER** | Use ai-summary-comment skill instead | -| `gh pr comment` | ❌ **NEVER** | Use ai-summary-comment skill instead | +| `gh pr review --comment` | ❌ **NEVER** | Review-PR.ps1 handles posting via scripts | +| `gh pr comment` | ❌ **NEVER** | Review-PR.ps1 handles posting via scripts | | Analyze and report findings | ✅ **YES** | This is the skill's purpose | **Workflow:** -1. **This skill**: Analyze PR, produce findings in your response -2. **User asks to post**: Then invoke `ai-summary-comment` skill to post +1. **This skill**: Analyze PR, produce findings and write to `pr-finalize-summary.md` +2. **Review-PR.ps1** calls `post-pr-finalize-comment.ps1` to post the summary The user controls when comments are posted. Your job is to analyze and present findings. diff --git a/.github/skills/pr-review/SKILL.md b/.github/skills/pr-review/SKILL.md new file mode 100644 index 000000000000..a3772b0dbd8c --- /dev/null +++ b/.github/skills/pr-review/SKILL.md @@ -0,0 +1,255 @@ +--- +name: pr-review +description: "End-to-end PR reviewer for dotnet/maui. Orchestrates 4 phases — Pre-Flight, Gate, Try-Fix, Report. Use when asked to 'review PR #XXXXX', 'work on PR #XXXXX', or 'fix issue #XXXXX'." +--- + +# PR Review — 4-Phase Orchestrator + +End-to-end PR review workflow that orchestrates phases to verify tests, explore independent fix alternatives, and produce a recommendation. + +**Trigger phrases:** "review PR #XXXXX", "work on PR #XXXXX", "fix issue #XXXXX" + +> 🚨 **NEVER** use `gh pr review --approve` or `--request-changes`. AI agents must NEVER post review comments. +> 🚨 **DO NOT post any comments to the PR.** This skill only produces output files in `CustomAgentLogsTmp/PRState/`. + +--- + +## Overview + +``` +Phase 1: Pre-Flight → Gather context, classify files → .github/pr-review/pr-preflight.md +Phase 2: Gate → ⛔ MUST PASS — verify tests FAIL/PASS → .github/pr-review/pr-gate.md +Phase 3: Try-Fix → ⚠️ MANDATORY multi-model exploration → invoke try-fix skill (×4 models) +Phase 4: Report → Write review recommendation → .github/pr-review/pr-report.md +``` + +> **Branch setup** is handled by `Review-PR.ps1` before this skill is invoked. By the time this skill runs, the review branch already exists with the PR commits cherry-picked and squashed. + +**All phases write output to:** `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/{phase}/content.md` + +--- + +## Critical Rules + +- ❌ Never run `git checkout` or `git switch` to change branches — stay on the review branch set up by the caller +- ❌ Never stop and ask the user — use best judgment to skip blocked phases and continue +- ❌ Never mark a phase complete with pending fields +- ❌ **Never skip Phase 3 multi-model exploration — it is MANDATORY for every review, no exceptions** +- ❌ Never run git commands that change branch state during Phases 2-3 (scripts handle file manipulation) +- ✅ Always create `CustomAgentLogsTmp/` output files for every phase +- ✅ Always include `Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>` in any commits +- ✅ Always use skills' scripts — don't bypass with manual commands + +### Multi-Model Configuration + +Phase 3 uses these 4 AI models (run SEQUENTIALLY — they modify the same files): + +| Order | Model | +|-------|-------| +| 1 | `claude-opus-4.6` | +| 2 | `claude-sonnet-4.6` | +| 3 | `gpt-5.3-codex` | +| 4 | `gemini-3-pro-preview` | + +**🚨 MANDATORY: Use `mode: "sync"` for ALL try-fix task invocations.** Never use `mode: "background"`. Background mode causes the orchestrator to move on before the attempt finishes, which means `try-fix/content.md` is never written and try-fix results are lost from the PR comment. Each try-fix task MUST complete and return its result before you proceed to the next attempt or to the Phase 3 completion checklist. + +### Environment Blockers + +| Blocker Type | Max Retries | Then Do | +|--------------|-------------|---------| +| Missing tool/driver | 1 install attempt | Skip phase, continue | +| Server errors (500, timeout) | 1 retry | Skip phase, continue | +| Port conflicts | 1 (kill process) | Skip phase, continue | +| Build failures in try-fix | 2 attempts | Skip remaining models, proceed to Report | +| Configuration issues | 1 fix attempt | Skip phase, continue | + +--- + +## Phase 1: Pre-Flight + +> Read and follow `.github/pr-review/pr-preflight.md` + +Gather context from the issue, PR, comments, and classify changed files. + +**Gate:** None — always runs. + +--- + +## Phase 2: Gate + +> Read and follow `.github/pr-review/pr-gate.md` + +Verify that the PR's tests actually catch the bug (FAIL without fix, PASS with fix). + +**Gate:** Pre-Flight must be ✅ COMPLETE. + +**If Gate fails:** +- Tests PASS without fix → Tests don't catch the bug. Proceed to Try-Fix anyway. +- Tests FAIL with fix → PR's fix doesn't work. Skip Try-Fix, proceed to Report. + +--- + +## Phase 3: Try-Fix → Invoke `try-fix` Skill (×4 Models) + +> Read and follow `.github/skills/try-fix/SKILL.md` + +> **⚠️ THIS PHASE IS MANDATORY. YOU MUST NEVER SKIP IT. NO EXCEPTIONS.** + +Even if the PR's fix looks correct and Gate passed, you MUST still run all 4 models to explore alternative approaches. The purpose is to find the BEST fix, not just validate one. + +### 🚨 CRITICAL: try-fix is Independent of PR's Fix + +The purpose is NOT to re-test the PR's fix, but to: +1. **Generate independent fix ideas** — What would YOU do to fix this bug? +2. **Test those ideas empirically** — Actually implement and run tests +3. **Compare with PR's fix** — Is there a simpler/better alternative? +4. **Learn from failures** — Record WHY failed attempts didn't work + +### Checklist (you MUST complete ALL of these) + +- [ ] Attempt 1 launched with claude-opus-4.6 +- [ ] `try-fix/content.md` updated with attempt 1 result +- [ ] Attempt 2 launched with claude-sonnet-4.6 +- [ ] `try-fix/content.md` updated with attempt 2 result +- [ ] Attempt 3 launched with gpt-5.3-codex +- [ ] `try-fix/content.md` updated with attempt 3 result +- [ ] Attempt 4 launched with gemini-3-pro-preview +- [ ] `try-fix/content.md` updated with attempt 4 result +- [ ] Cross-pollination round completed (all models queried) +- [ ] Best fix selected with comparison table + +### Round 1: Independent Exploration + +For each model, invoke `try-fix` skill via a `general-purpose` task agent with that model: + +``` +prompt: | + Invoke the try-fix skill for PR #XXXXX: + - problem: {bug description from Pre-Flight} + - platform: {platform from Platform Selection} + - test_command: pwsh .github/scripts/BuildAndRunHostApp.ps1 -Platform {platform} -TestFilter "IssueXXXXX" + - target_files: + - src/{area}/{file1}.cs + - src/{area}/{file2}.cs + + Generate ONE independent fix idea. Review the PR's fix first to ensure your approach is DIFFERENT. +``` + +**Wait for each to complete before starting the next.** + +**🧹 MANDATORY: Clean up between attempts:** + +```bash +# Restore baseline from previous attempt — this is the ONLY way to restore. +# Do NOT use manual git checkout/restore/reset commands. +pwsh .github/scripts/EstablishBrokenBaseline.ps1 -Restore +``` + +**📝 MANDATORY: Update `try-fix/content.md` after EVERY attempt.** Do not wait until all attempts are done. After each try-fix attempt completes (pass or fail), immediately write/update `CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/try-fix/content.md` with all results so far. This ensures the PR comment always reflects the latest try-fix state, even if a later attempt times out or the agent is interrupted. + +### Round 2+: Cross-Pollination (MANDATORY) + +After Round 1, invoke EACH model via task agent: +``` +"Review PR #XXXXX fix attempts: + - Attempt 1: {approach} - ✅/❌ + - Attempt 2: {approach} - ✅/❌ + ... + Do you have any NEW fix ideas? Reply: 'NEW IDEA: {desc}' or 'NO NEW IDEAS'" +``` + +Run any new ideas as additional try-fix attempts. Repeat until all say "NO NEW IDEAS" (max 3 rounds). + +### Selecting the Best Fix + +Compare all passing candidates on: +1. **Must pass tests** — Only consider ✅ PASS candidates +2. **Simplest solution** — Fewer files, fewer lines +3. **Most robust** — Handles edge cases +4. **Matches codebase style** — Consistent with existing patterns + +### Output File + +```bash +mkdir -p CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/try-fix +``` + +Write `content.md`: +```markdown +### Fix Candidates +| # | Source | Approach | Test Result | Files Changed | Notes | +|---|--------|----------|-------------|---------------|-------| +| 1 | try-fix | {approach} | ✅/❌ | 1 file | {insight} | +| ... | ... | ... | ... | ... | ... | +| PR | PR #XXXXX | {approach} | ✅ PASSED (Gate) | 2 files | Original PR | + +### Cross-Pollination +| Model | Round | New Ideas? | Details | +|-------|-------|------------|---------| +| ... | 2 | Yes/No | {idea or "NO NEW IDEAS"} | + +**Exhausted:** {Yes/No} +**Selected Fix:** {PR's fix / Candidate #N} — {Reason} +``` + +### Common Mistakes + +- ❌ Looking at PR's fix before generating ideas — generate independently first +- ❌ Running try-fix in parallel — SEQUENTIAL ONLY, always `mode: "sync"` +- ❌ Using `mode: "background"` for try-fix tasks — results will be lost +- ❌ Skipping cleanup between attempts — ALWAYS run cleanup commands +- ❌ Declaring exhaustion without querying all 4 models + +--- + +## Phase 4: Report + +> Read and follow `.github/pr-review/pr-report.md` + +Deliver the final review recommendation. + +> 🚨 **DO NOT post any comments.** All output goes to `CustomAgentLogsTmp/PRState/`. + +**Gate:** Phases 1-3 must be complete. + +--- + +## Output Directory Structure (MANDATORY) + +``` +CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/ +├── pre-flight/ +│ └── content.md # Phase 1 output (pr-preflight) +├── gate/ +│ └── content.md # Phase 2 output (pr-gate) +├── try-fix/ +│ ├── content.md # Phase 3 summary +│ └── attempt-{N}/ # Per-model attempt +│ ├── approach.md # What was tried +│ ├── result.txt # Pass / Fail / Blocked +│ ├── fix.diff # git diff of changes +│ └── analysis.md # Why it worked/failed +└── report/ + └── content.md # Phase 4 output (pr-report) +``` + +--- + +## Quick Reference + +| Phase | Instructions | Key Action | If Blocked | +|-------|--------------|------------|------------| +| 1. Pre-Flight | `pr-preflight.md` | Read issue + PR context | Skip missing info, continue | +| 2. Gate | `pr-gate.md` | Verify tests via task agent | Document, continue to Try-Fix | +| 3. Try-Fix | `try-fix` skill (×4) | **4-model exploration (MANDATORY)** | Skip failing models, continue | +| 4. Report | `pr-report.md` | Write review recommendation | Never skip | + +--- + +## Common Errors and Recovery + +| Error | Cause | Fix | +|-------|-------|-----| +| `ENOENT: no such file` on skill | Dirty working tree from prior attempt | Run cleanup: `-Restore` + `git checkout HEAD -- .` + `git clean -fd --exclude=CustomAgentLogsTmp/` | +| Dirty working tree before attempt | Prior attempt didn't restore | Same cleanup as above | +| Build errors in unmodified files | Stale state | Cleanup + retry; if still fails, treat as environment blocker | diff --git a/.github/skills/try-fix/SKILL.md b/.github/skills/try-fix/SKILL.md index d13690296403..03ed7e57644e 100644 --- a/.github/skills/try-fix/SKILL.md +++ b/.github/skills/try-fix/SKILL.md @@ -186,22 +186,21 @@ The skill is complete when: ### Step 2: Establish Baseline (MANDATORY) -🚨 **ALWAYS use EstablishBrokenBaseline.ps1 - NEVER manually revert files.** +🚨 **ONLY use EstablishBrokenBaseline.ps1 — NEVER use `git checkout`, `git restore`, or `git reset` to revert fix files.** + +The script auto-restores any previous baseline, tracks state, and prevents loops. +Manual git commands bypass all of this and WILL cause infinite loops in CI. ```powershell -# Capture baseline output as proof it was run pwsh .github/scripts/EstablishBrokenBaseline.ps1 *>&1 | Tee-Object -FilePath "$OUTPUT_DIR/baseline.log" ``` -The script auto-detects and reverts fix files to merge-base state while preserving test files. **Will fail fast if no fix files detected** - the PR's changes must be present in the current branch (the `Review-PR.ps1` script handles this by merging the PR before the agent runs). Optional flags: `-BaseBranch main`, `-DryRun`. - **Verify baseline was established:** ```powershell -# baseline.log should contain "Baseline established" and list of reverted files Select-String -Path "$OUTPUT_DIR/baseline.log" -Pattern "Baseline established" ``` -**If the script fails with "No fix files detected":** The PR changes are not present on the current branch. Report this as `Blocked` — do NOT switch branches. +**If the script fails with "No fix files detected":** Report as `Blocked` — do NOT switch branches. **If something fails mid-attempt:** `pwsh .github/scripts/EstablishBrokenBaseline.ps1 -Restore` @@ -316,9 +315,10 @@ git diff | Set-Content "$OUTPUT_DIR/fix.diff" ```bash pwsh .github/scripts/EstablishBrokenBaseline.ps1 -Restore -git checkout HEAD -- . ``` +🚨 Do NOT use `git checkout HEAD -- .` or `git clean` to restore — use the script. + ### Step 9: Report Results Provide structured output to the invoker: diff --git a/.github/skills/verify-tests-fail-without-fix/SKILL.md b/.github/skills/verify-tests-fail-without-fix/SKILL.md index ba3df1d1aaa6..201258220d4d 100644 --- a/.github/skills/verify-tests-fail-without-fix/SKILL.md +++ b/.github/skills/verify-tests-fail-without-fix/SKILL.md @@ -94,7 +94,7 @@ The script auto-detects which mode to use based on whether fix files are present 7. Runs tests (should PASS with fix) 8. **Generates markdown reports**: - `CustomAgentLogsTmp/TestValidation/verification-report.md` - Full detailed report - - `CustomAgentLogsTmp/PRState/verification-report.md` - Gate section for PR agent + - `CustomAgentLogsTmp/PRState/verification-report.md` - Validate section for agent 9. **Updates PR labels** based on result 10. Reports result diff --git a/eng/pipelines/ci-copilot.yml b/eng/pipelines/ci-copilot.yml index b8c8c4d4399f..bbd7fdad5c29 100644 --- a/eng/pipelines/ci-copilot.yml +++ b/eng/pipelines/ci-copilot.yml @@ -524,7 +524,7 @@ stages: # The script will merge the PR into the current branch # -PostSummaryComment and -RunFinalize handle posting comments set +e - pwsh -NoProfile .github/scripts/Review-PR.ps1 -PRNumber ${{ parameters.PRNumber }} -Platform ${{ parameters.Platform }} -RunFinalize -PostSummaryComment -LogFile "$(Build.ArtifactStagingDirectory)/copilot-logs/copilot_review_output.md" + pwsh -NoProfile .github/scripts/Review-PR.ps1 -PRNumber ${{ parameters.PRNumber }} -Platform ${{ parameters.Platform }} -LogFile "$(Build.ArtifactStagingDirectory)/copilot-logs/copilot_review_output.md" COPILOT_EXIT_CODE=$? set -e diff --git a/eng/pipelines/common/provision.yml b/eng/pipelines/common/provision.yml index 1f00c37d748e..d0b4050173fd 100644 --- a/eng/pipelines/common/provision.yml +++ b/eng/pipelines/common/provision.yml @@ -143,9 +143,19 @@ steps: done if [[ -z "$XCODE_PATH" ]]; then - echo "ERROR: No suitable Xcode version found for requested version ${ORIGINAL_VERSION}" + echo "WARNING: No exact match for requested Xcode version ${ORIGINAL_VERSION}" echo "Tried: ${VERSIONS_TO_TRY[*]}" - exit 1 + echo "Falling back to latest available Xcode on this agent..." + # Find the latest Xcode by sorting version numbers + LATEST_XCODE=$(ls -1d /Applications/Xcode_*.app 2>/dev/null | sed 's|/Applications/Xcode_||;s|\.app||' | sort -t. -k1,1n -k2,2n -k3,3n | tail -1) + if [[ -n "$LATEST_XCODE" ]]; then + XCODE_VERSION="$LATEST_XCODE" + XCODE_PATH="/Applications/Xcode_${LATEST_XCODE}.app" + echo "Using latest available Xcode: ${XCODE_VERSION} at ${XCODE_PATH}" + else + echo "ERROR: No Xcode installations found in /Applications" + exit 1 + fi fi sudo xcode-select -s "$XCODE_PATH"