Allow fork PRs to auto-trigger evaluate-pr-tests workflow#34655
Merged
Allow fork PRs to auto-trigger evaluate-pr-tests workflow#34655
Conversation
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34655Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34655" |
Add forks: ["*"] to pull_request trigger. Safe because: - Agent runs in sandboxed container with scrubbed credentials - Checkout-GhAwPr.ps1 restores trusted skills from base branch - safe-outputs limits to 1 sanitized comment - All filenames escaped via Escape-ForCodeSpan Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
7100785 to
c7d545e
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Evaluate PR Tests gh-aw workflow to broaden when it can run (intended to include fork PRs) and hardens the supporting scripts around PR checkout and report generation.
Changes:
- Adds
forks: ["*"]to the workflow’spull_requesttrigger and updates the compiled.lock.yml. - Introduces a shared
.github/scripts/Checkout-GhAwPr.ps1to checkout PRs and restore trusted agent infrastructure. - Enhances
Gather-TestContext.ps1with-PrNumbersupport and additional markdown escaping for PR-sourced filenames.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/copilot-evaluate-tests.md |
Adjusts workflow trigger configuration to (intended) allow fork PRs. |
.github/workflows/copilot-evaluate-tests.lock.yml |
Regenerated compiled workflow reflecting the trigger and execution model changes. |
.github/skills/evaluate-pr-tests/scripts/Gather-TestContext.ps1 |
Adds PR-number-based diffing/download and improves output escaping. |
.github/scripts/Checkout-GhAwPr.ps1 |
Adds a shared pre-agent checkout + base restore mechanism for gh-aw workflows. |
.github/instructions/gh-aw-workflows.instructions.md |
Adds repo guidance for authoring gh-aw workflows and fork handling patterns. |
.github/aw/actions-lock.json |
Adds pinned entries for github/gh-aw-actions/setup versions used by the workflow compilation. |
Comments suppressed due to low confidence (3)
.github/workflows/copilot-evaluate-tests.lock.yml:112
- On
pull_requestevents from forks, GitHub Actions does not provide repository secrets to the workflow run. This workflow depends onsecrets.COPILOT_GITHUB_TOKEN(validated here and used later to run Copilot), so fork-triggered runs would fail even if fork gating is removed. If the intent is truly “auto-trigger on fork PRs”, this likely needs a different trigger strategy (e.g.,pull_request_targetwith a hardened checkout/restore model) or an explicit maintainer-approved dispatch/comment flow instead.
- name: Validate COPILOT_GITHUB_TOKEN secret
id: validate-secret
run: ${RUNNER_TEMP}/gh-aw/actions/validate_multi_secret.sh COPILOT_GITHUB_TOKEN 'GitHub Copilot CLI' https://github.github.com/gh-aw/reference/engines/#github-copilot-default
env:
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
.github/workflows/copilot-evaluate-tests.lock.yml:1008
- The workflow still has a
pre_activationmembership gate (activated: ...is_team_member == 'true'). For fork PRs opened by external contributors, this will evaluate to false, so the workflow won’t activate even withforks: ["*"]. Either adjust the activation gating to allowpull_requestevents from forks (while keeping stricter gating for/evaluate-tests), or update the PR title/description because this change alone doesn’t enable auto-triggering for fork PRs.
pre_activation:
if: >
(github.event_name == 'pull_request' && github.event.pull_request.draft == false) || github.event_name == 'workflow_dispatch' || (github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
startsWith(github.event.comment.body, '/evaluate-tests'))
runs-on: ubuntu-slim
outputs:
activated: ${{ steps.check_membership.outputs.is_team_member == 'true' }}
matched_command: ''
steps:
- name: Setup Scripts
uses: github/gh-aw-actions/setup@20045bbd5ad2632b9809856c389708eab1bd16ef # v0.62.2
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check team membership for workflow
id: check_membership
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
env:
GH_AW_REQUIRED_ROLES: admin,maintainer,write
with:
.github/workflows/copilot-evaluate-tests.lock.yml:121
- Security: the activation job checks out
.githubviaactions/checkoutwithout pinningrefto the base branch SHA. Onpull_requestevents this normally checks outrefs/pull/<n>/merge, meaning a fork PR can influence the working tree used forruntime-importprompt rendering (and any other files read in activation). If fork PRs are allowed to trigger this workflow, consider pinning checkout to the base SHA/ref for pull_request events or keep the fork guard so untrusted forks can’t affect the prompt/activation context.
- name: Checkout .github and .agents folders
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github
.agents
sparse-checkout-cone-mode: true
fetch-depth: 1
The agent runs sandboxed with no credentials and can only post 1 comment. For issue_comment on forks, the platform may overwrite restored skills, but the agent pre-flight catches missing SKILL.md gracefully, and even a manipulated SKILL.md can only produce a misleading comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
For pull_request and issue_comment triggers, the gh-aw platform's checkout_pr_branch.cjs already handles PR checkout. Our script was redundant for those paths (and its restore was overwritten by the platform re-checkout). Now only runs for workflow_dispatch where the platform checkout is skipped entirely. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
d777812 to
b12fdee
Compare
- Update gh-aw-workflows.instructions.md: remove all references to the deleted fork guard, document forks: ["*"] opt-in, clarify residual risk for fork PRs on all triggers - Fix Checkout-GhAwPr.ps1 comment: platform checkout runs AFTER user steps (not before) for pull_request triggers, matching the documented gh-aw execution model Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
b12fdee to
4328beb
Compare
kubaflo
approved these changes
Mar 25, 2026
This was referenced Mar 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables the copilot-evaluate-tests gh-aw workflow to run on fork PRs by adding
forks: ["*"]to thepull_requesttrigger and removing the fork guard fromCheckout-GhAwPr.ps1.Changes
copilot-evaluate-tests.md: Added
forks: ["*"]to opt out of gh-aw auto-injected fork activation guard. ScopedCheckout-GhAwPr.ps1step toworkflow_dispatchonly (redundant for other triggers since platform handles checkout).copilot-evaluate-tests.lock.yml: Recompiled via
gh aw compile— fork guard removed from activationif:conditions.Checkout-GhAwPr.ps1: Removed the
isCrossRepositoryfork guard. Updated header docs and restore comments to accurately describe behavior for all trigger×fork combinations (including corrected step ordering).gh-aw-workflows.instructions.md: Updated all stale references to the removed fork guard. Documented
forks: ["*"]opt-in, clarified residual risk model for fork PRs, and updated troubleshooting table.Security Model
Fork PRs are safe because:
safe-outputs: add-comment: max: 1runtime-import) — forks cannot alter instructionsSKILL.mdif fork isn't rebased onmainGITHUB_TOKEN(checkout without execution)Testing
workflow_dispatchtested against fork PR Fix Android layout jump when navigating with IME open and NavBarIsVisible=false #34621if:conditionspull_requesttrigger on fork PRs can only be verified post-merge (GitHub Actions reads lock.yml from default branch)