[Testing] Add Material3 test category and separate pipeline stage for android material3 UI tests#33409
Conversation
NafeelaNazhir
added
area-testing
dotnet-policy-service
bot
added
the
partner/syncfusion
sheiksyedm
changed the title
There was a problem hiding this comment.
Pull request overview
This pull request introduces infrastructure for running Material3-specific UI tests on Android by adding a new test category, dedicated pipeline stages, and a sample Material3 CheckBox test. The changes enable Material3 builds to be separated from standard test builds, ensuring proper isolation and validation of Material3 components.
- Adds
Material3to UITestCategories for organizing Material3-specific tests - Introduces dedicated Material3 build and test stages in the Azure Pipeline (Android-only)
- Implements Material3 build parameter (
useMaterial3) that enables/p:UseMaterial3=truefor Android builds - Adds sample Material3 CheckBox test with XAML page, code-behind, and screenshot verification
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
src/Controls/tests/TestCases.Shared.Tests/UITestCategories.cs |
Adds Material3 constant for test categorization |
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/IssueMaterial3CheckBoxTest.cs |
Implements Android-specific test for Material3 CheckBox default appearance with screenshot verification |
src/Controls/tests/TestCases.HostApp/Issues/IssueMaterial3CheckBox.xaml.cs |
Code-behind for Material3 CheckBox test page |
src/Controls/tests/TestCases.HostApp/Issues/IssueMaterial3CheckBox.xaml |
XAML layout for Material3 CheckBox test with default state and AutomationIds |
src/Controls/tests/TestCases.Android.Tests/snapshots/android/Material3CheckBox_DefaultAppearance.png |
Reference screenshot for Material3 CheckBox visual verification |
eng/pipelines/common/ui-tests.yml |
Adds Material3 build stage and Android test stage with separate artifact handling and testFilter |
eng/pipelines/common/ui-tests-build-sample.yml |
Implements useMaterial3 parameter with conditional build logic and Material3-specific artifact publishing |
| @@ -0,0 +1,25 @@ | |||
| #if ANDROID | |||
There was a problem hiding this comment.
The test is wrapped in an #if ANDROID directive which limits it to Android only. According to the UI Testing Guidelines (CodingGuidelineID: 1000002), platform-specific directives should only be used when there is a specific technical limitation. Since Material3 is Android-specific by design (as indicated by the PlatformAffected.Android in the HostApp), this usage is acceptable. However, the guideline states to "document any platform-specific limitations with clear comments". Consider adding a comment above the directive explaining why this test is Android-only, such as: // Material3 is Android-specific
src/Controls/tests/TestCases.HostApp/Issues/IssueMaterial3CheckBox.xaml
Outdated
Show resolved
Hide resolved
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/IssueMaterial3CheckBoxTest.cs
Outdated
Show resolved
Hide resolved
| # Collect and publish Android Material3 snapshot diffs | ||
| - template: ui-tests-collect-snapshot-diffs.yml | ||
| parameters: | ||
| platform: 'Android Material3' |
There was a problem hiding this comment.
The platform name in the artifact parameters is 'Android Material3' with a space, which may cause issues with artifact naming or filtering. Consider using a consistent naming pattern without spaces, such as 'AndroidMaterial3' or 'Android-Material3', to match the pattern used in artifact names like 'uitest-snapshot-results-android-material3-...'.
| platform: 'Android Material3' | |
| platform: 'AndroidMaterial3' |
src/Controls/tests/TestCases.HostApp/Issues/Material3CheckBoxDefaultAppearance.xaml.cs
Outdated
Show resolved
Hide resolved
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
This reverts commit 1a9533a.
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
This pull request adds support for running UI tests specifically targeting Material3 components on Android. It introduces new pipeline parameters and stages to enable Material3 builds and tests, updates artifact publishing logic to handle Material3-specific outputs, and adds a sample test case for a Material3 CheckBox. The changes ensure that Material3 tests are built, executed, and their results published separately from standard tests.
Pipeline and Build Enhancements:
useMaterial3parameter toui-tests-build-sample.ymland updated the build script to enable Material3 builds for Android when this parameter is set. [1] [2]ui-tests-build-sample.ymlto handle Material3-specific build and failure artifacts, ensuring separation from standard artifacts.Pipeline Stage and Test Execution Updates:
ui-tests.yml, including a new build stage (build_ui_tests_material3) for Android and a new test stage (android_ui_tests_material3) that downloads and runs Material3-specific tests. [1] [2]Test Infrastructure and Sample Test:
Material3test category toUITestCategories.csfor organizing Material3-specific tests.IssueMaterial3CheckBox.xamland code-behind) and a corresponding UI test (IssueMaterial3CheckBoxTest.cs) to validate the default appearance of a Material3 CheckBox on Android. [1] [2] [3]Pipeline stage:
Test execution result: