[Testing] Fix flaky EntryClearButtonShouldBeVisibleOnDarkTheme test#34341
[Testing] Fix flaky EntryClearButtonShouldBeVisibleOnDarkTheme test#34341
Conversation
The dark theme screenshot test fails in 67% of PR builds (61/91) with a consistent 13% image difference. The root cause is a race condition: after tapping ThemeButton to switch to dark mode, the VerifyScreenshot call fires before the WinUI theme transition completes. Changes: - HostApp: Add hidden ThemeLabel that updates text on theme change, providing a deterministic signal that the theme switch completed - Test: Wait for ThemeLabel text to show 'Dark' before proceeding - Test: Add retryTimeout of 3 seconds to VerifyScreenshot to handle any remaining rendering delay after theme propagation This follows the same pattern used by AppThemeFeatureTests which also uses retryTimeout for theme-change screenshot verification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34341Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34341" |
There was a problem hiding this comment.
Pull request overview
This PR fixes a highly flaky UI test (EntryClearButtonShouldBeVisibleOnDarkTheme) that was failing in ~67% of PR builds due to a race condition: the test would take a screenshot mid-theme-transition immediately after tapping the "Change theme" button, capturing the UI before WinUI's async theme rendering was complete, resulting in a consistent ~13% screenshot diff.
Changes:
- Added a hidden
ThemeLabel(viaHeightRequest=0, Opacity=0) to the HostApp page to serve as a synchronization signal updated in the button click handler. - Added
WaitForTextToBePresentInElement("ThemeLabel", "Dark")in the test to wait before taking the screenshot. - Added
retryTimeout: TimeSpan.FromSeconds(3)to bothVerifyScreenshot()calls in the dark theme test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/Controls/tests/TestCases.HostApp/Issues/Issue32886.cs |
Adds hidden ThemeLabel field and updates it in the button click handler as a synchronization signal |
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue32886.cs |
Adds WaitForTextToBePresentInElement wait and retryTimeout to VerifyScreenshot for the dark-theme test |
You can also share your feedback on Copilot code review. Take the survey.
| var newTheme = Application.Current.UserAppTheme != AppTheme.Dark ? AppTheme.Dark : AppTheme.Light; | ||
| Application.Current.UserAppTheme = newTheme; | ||
| _themeLabel.Text = newTheme == AppTheme.Dark ? "Dark" : "Light"; |
There was a problem hiding this comment.
The _themeLabel.Text = "Dark" update in Button_Clicked happens synchronously right after Application.Current.UserAppTheme = newTheme, but on Windows the actual WinUI visual theme propagation uses DispatcherQueue.TryEnqueue (see Application.Windows.cs:105), which is an asynchronous dispatch. So WaitForTextToBePresentInElement("ThemeLabel", "Dark") only confirms the MAUI-level property assignment occurred; it does not guarantee that the WinUI RequestedTheme update and the subsequent re-render of child controls (e.g., the Entry's clear button color) have completed.
As a result, the test can still take a screenshot mid-theme-render on Windows. The retryTimeout: TimeSpan.FromSeconds(3) on VerifyScreenshot is the real mechanism compensating for this. To make the label a stronger synchronization point, its text should be updated after awaiting the theme change, or inside a RequestedThemeChanged callback rather than synchronously after setting UserAppTheme. As an alternative, a RequestedThemeChanged event on Application.Current could set the label text when the theme change is actually applied.
Note: If the retryTimeout alone is considered sufficient, the ThemeLabel approach is still useful as it reduces unnecessary wait time on fast machines—but in that case the comment "deterministic signal" in the PR description slightly overstates the guarantee provided.
| VerifyScreenshot(cropBottom: 1550, retryTimeout: TimeSpan.FromSeconds(3)); | ||
| #else | ||
| VerifyScreenshot(); | ||
| VerifyScreenshot(retryTimeout: TimeSpan.FromSeconds(3)); |
There was a problem hiding this comment.
The AppThemeFeatureTests class (and most other screenshot tests in the codebase) consistently uses tolerance: 0.5 alongside retryTimeout for theme-change screenshots (e.g., VerifyScreenshot(tolerance: 0.5, retryTimeout: TimeSpan.FromSeconds(2))). This PR uses retryTimeout: TimeSpan.FromSeconds(3) without tolerance, which deviates from the established pattern for theme-change screenshot tests. Adding tolerance: 0.5 would be consistent with how similar tests handle minor cross-machine rendering differences (as seen in AppThemeFeatureTests.cs:31,40,55,67 etc.).
|
/azp run maui-pr-uitests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Note
Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!
Description
Fixes the flaky
EntryClearButtonShouldBeVisibleOnDarkThemetest which fails in 67% of PR builds (61 out of 91 failed builds in the last week) with a consistent 13% screenshot difference.Root Cause
The test taps a button to switch from light to dark theme, then immediately takes a screenshot. On Windows, the WinUI theme transition (
Application.Current.UserAppTheme = AppTheme.Dark) triggers asynchronous UI updates across all visual elements. The screenshot captures the page mid-transition, producing a consistent 13% difference from the baseline.Fix
Added a hidden ThemeLabel to the HostApp page (
HeightRequest=0, Opacity=0) that updates its text when the theme changes. This provides a deterministic signal that the theme change handler has executed.Wait for theme change completion in the test using
WaitForTextToBePresentInElement("ThemeLabel", "Dark")before proceeding to screenshot.Added
retryTimeout: TimeSpan.FromSeconds(3)toVerifyScreenshot()calls for the dark theme test, following the same pattern used byAppThemeFeatureTestswhich also usesretryTimeoutfor theme-change screenshots.Impact
This single test is responsible for the
Controls Entryjob failing in 61/91 PR builds. Fixing it should significantly reduce the overall UI test failure rate.