fix: resolve pool validation errors breaking device-tests and ui-tests on SR3#34690
fix: resolve pool validation errors breaking device-tests and ui-tests on SR3#34690PureWeen merged 1 commit intorelease/10.0.1xx-sr3from
Conversation
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34690Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34690" |
There was a problem hiding this comment.
Pull request overview
Fixes Azure DevOps YAML pool validation failures in the SR3 device-tests and ui-tests pipelines by avoiding invalid pool references in internal builds, while also aligning CI triggers/Helix wiring with SR5.
Changes:
- Split agent pool parameters into internal/public variants and select them via
System.TeamProject(device-tests + ui-tests), plus adddarc-*branch triggers. - Add internal Helix variable group usage and forward Helix access token into Helix submission steps; increase device-test stage timeouts.
- Fix
_BuildOfficalIdtypo to_BuildOfficialIdacross pipeline YAML files.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| eng/pipelines/handlers.yml | Renames build ID variable usage to _BuildOfficialId. |
| eng/pipelines/common/variables.yml | Renames _BuildOfficalId to _BuildOfficialId in shared variables. |
| eng/pipelines/common/sdk-insertion.yml | Updates OfficialBuildId MSBuild property to use _BuildOfficialId. |
| eng/pipelines/ci-uitests.yml | Adds darc-* trigger; introduces internal/public pool split and conditional selection. |
| eng/pipelines/ci-official.yml | Updates source indexing command to use _BuildOfficialId. |
| eng/pipelines/ci-device-tests.yml | Adds darc-* trigger; conditional internal/public pools; internal Helix variable group; removes old Windows device-tests section. |
| eng/pipelines/arcade/variables.yml | Renames _BuildOfficalId to _BuildOfficialId for Arcade templates. |
| eng/pipelines/arcade/stage-device-tests.yml | Adds Helix parameters, forwards token to send-to-helix, adjusts timeouts, and adds HelixInternal-related arguments. |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsAndroid_ /p:HelixInternal=False | ||
| ${{ else }}: | ||
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsAndroid_ /p:HelixInternal=True |
There was a problem hiding this comment.
The added /p:HelixInternal=... MSBuild property appears to be ineffective for this pipeline because eng/helix_xharness.proj (the default helixProject) does not reference HelixInternal. If the intent is to switch between internal vs public Helix queues/endpoints, that logic needs to be implemented in helix_xharness.proj (or a different property should be used); otherwise consider removing this property to avoid confusion.
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsAndroid_ /p:HelixInternal=False | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsAndroid_ /p:HelixInternal=True | |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsAndroid_ | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsAndroid_ |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig)_CoreCLR /p:TestRunNamePrefix=DeviceTestsAndroid_CoreCLR_ /p:HelixInternal=False | ||
| ${{ else }}: | ||
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig)_CoreCLR /p:TestRunNamePrefix=DeviceTestsAndroid_CoreCLR_ /p:HelixInternal=True |
There was a problem hiding this comment.
The added /p:HelixInternal=... MSBuild property appears to be ineffective for this pipeline because eng/helix_xharness.proj (the default helixProject) does not reference HelixInternal. If the intent is to switch between internal vs public Helix queues/endpoints, that logic needs to be implemented in helix_xharness.proj (or a different property should be used); otherwise consider removing this property to avoid confusion.
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig)_CoreCLR /p:TestRunNamePrefix=DeviceTestsAndroid_CoreCLR_ /p:HelixInternal=False | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig)_CoreCLR /p:TestRunNamePrefix=DeviceTestsAndroid_CoreCLR_ /p:HelixInternal=True | |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig)_CoreCLR /p:TestRunNamePrefix=DeviceTestsAndroid_CoreCLR_ | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=android /p:TestRunNameSuffix=_$(_BuildConfig)_CoreCLR /p:TestRunNamePrefix=DeviceTestsAndroid_CoreCLR_ |
| - template: ${{ iif(eq(parameters.runAsPublic, 'true'), '/eng/common/templates/steps/send-to-helix.yml', '/eng/common/templates-official/steps/send-to-helix.yml@self') }} | ||
| parameters: |
There was a problem hiding this comment.
runAsPublic is declared as a boolean parameter in this template, but this iif/eq compares it to the string 'true'. This is inconsistent with the other new boolean comparisons in the file and can lead to selecting the wrong send-to-helix template. Use a boolean comparison (eq(parameters.runAsPublic, true/false)) or change runAsPublic to a string consistently.
| - name: HelixInternal | ||
| type: string | ||
| default: '' |
There was a problem hiding this comment.
HelixInternal is added as a template parameter but is not referenced anywhere in this template. This makes the API misleading and encourages callers to pass values that have no effect. Either remove HelixInternal entirely, or wire it into the Helix submission arguments/logic (and keep the parameter type consistent with how it’s consumed).
| - name: HelixInternal | |
| type: string | |
| default: '' |
| HelixProjectArguments: /p:TargetOS=ios /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsIOS_ /p:HelixInternal=False | ||
| ${{ else }}: | ||
| HelixProjectArguments: /p:TargetOS=ios /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsIOS_ /p:HelixInternal=True |
There was a problem hiding this comment.
The added /p:HelixInternal=... MSBuild property appears to be ineffective for this pipeline because eng/helix_xharness.proj (the default helixProject) does not reference HelixInternal. If the intent is to switch between internal vs public Helix queues/endpoints, that logic needs to be implemented in helix_xharness.proj (or a different property should be used); otherwise consider removing this property to avoid confusion.
| HelixProjectArguments: /p:TargetOS=ios /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsIOS_ /p:HelixInternal=False | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=ios /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsIOS_ /p:HelixInternal=True | |
| HelixProjectArguments: /p:TargetOS=ios /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsIOS_ | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=ios /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsIOS_ |
| HelixProjectArguments: /p:TargetOS=maccatalyst /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsMacCatalyst_ /p:HelixInternal=False | ||
| ${{ else }}: | ||
| HelixProjectArguments: /p:TargetOS=maccatalyst /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsMacCatalyst_ /p:HelixInternal=True |
There was a problem hiding this comment.
The added /p:HelixInternal=... MSBuild property appears to be ineffective for this pipeline because eng/helix_xharness.proj (the default helixProject) does not reference HelixInternal. If the intent is to switch between internal vs public Helix queues/endpoints, that logic needs to be implemented in helix_xharness.proj (or a different property should be used); otherwise consider removing this property to avoid confusion.
| HelixProjectArguments: /p:TargetOS=maccatalyst /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsMacCatalyst_ /p:HelixInternal=False | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=maccatalyst /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsMacCatalyst_ /p:HelixInternal=True | |
| HelixProjectArguments: /p:TargetOS=maccatalyst /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsMacCatalyst_ | |
| ${{ else }}: | |
| HelixProjectArguments: /p:TargetOS=maccatalyst /p:TestRunNameSuffix=_$(_BuildConfig) /p:TestRunNamePrefix=DeviceTestsMacCatalyst_ |
Root cause: Azure DevOps validates ALL pool name references in YAML,
including those in parameter defaults. The 'NetCore-Public' pool doesn't
exist in the dnceng/internal project, causing instant YAML validation
failures for both device-tests and ui-tests pipelines since January 2026.
Changes (aligned with release/10.0.1xx-sr5):
1. ci-device-tests.yml:
- Added darc-* branch trigger for Maestro dependency updates
- Split pool parameters into Internal/Public variants
- Added conditional pool selection based on System.TeamProject
- Added Helix variable group for internal builds
- Removed old common/device-tests.yml Windows section (now via Helix)
2. ci-uitests.yml:
- Added darc-* branch trigger
- Split pool parameters into Internal/Public variants
- Added conditional pool selection (internal uses NetCore1ESPool-Internal
and Azure Pipelines; public uses NetCore-Public and AcesShared)
3. stage-device-tests.yml:
- Added HelixAccessToken and HelixInternal parameters
- Forwarded tokens to all 5 send-to-helix invocations
- Increased timeouts: build 60→120min, MacCatalyst 60→240min
4. Typo fixes across 5 files:
- _BuildOfficalId → _BuildOfficialId (missing 'i')
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PureWeen
force-pushed
the
fix/sr3-ci-pools
branch
from
c887fb1 to
0164628
Compare
Note
Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!
Description
Follow-up to #34685. The device-tests and ui-tests pipelines have been failing with instant YAML validation errors since January 2026:
Root Cause
Azure DevOps validates ALL pool name references in YAML at parse time, including parameter defaults. The
NetCore-Publicpool only exists indnceng-public, not indnceng/internalwhere these pipelines run. The pool names were hardcoded in parameters rather than using conditional internal/public selection.Changes (aligned with
release/10.0.1xx-sr5)ci-device-tests.yml— Split pools into Internal/Public variants withSystem.TeamProjectconditional; addeddarc-*trigger; added Helix variable group; removed oldcommon/device-tests.ymlWindows sectionci-uitests.yml— Same internal/public pool split and conditional selection; addeddarc-*triggerstage-device-tests.yml— AddedHelixAccessToken/HelixInternalparameters forwarded to allsend-to-helixinvocations; increased timeouts_BuildOfficalId→_BuildOfficialIdacross 5 filesSupersedes
This PR supersedes #34685 which addressed the triggers and Helix plumbing but missed the pool validation root cause.