Skip to content

[Testing] Fix flaky EntryClearButtonShouldBeVisibleOnDarkTheme test#34341

Closed
PureWeen wants to merge 1 commit intomainfrom
fix/flaky-tests
Closed

[Testing] Fix flaky EntryClearButtonShouldBeVisibleOnDarkTheme test#34341
PureWeen wants to merge 1 commit intomainfrom
fix/flaky-tests

Conversation

@PureWeen
Copy link
Copy Markdown
Member

@PureWeen PureWeen commented Mar 4, 2026

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description

Fixes the flaky EntryClearButtonShouldBeVisibleOnDarkTheme test which fails in 67% of PR builds (61 out of 91 failed builds in the last week) with a consistent 13% screenshot difference.

Root Cause

The test taps a button to switch from light to dark theme, then immediately takes a screenshot. On Windows, the WinUI theme transition (Application.Current.UserAppTheme = AppTheme.Dark) triggers asynchronous UI updates across all visual elements. The screenshot captures the page mid-transition, producing a consistent 13% difference from the baseline.

Fix

  1. Added a hidden ThemeLabel to the HostApp page (HeightRequest=0, Opacity=0) that updates its text when the theme changes. This provides a deterministic signal that the theme change handler has executed.

  2. Wait for theme change completion in the test using WaitForTextToBePresentInElement("ThemeLabel", "Dark") before proceeding to screenshot.

  3. Added retryTimeout: TimeSpan.FromSeconds(3) to VerifyScreenshot() calls for the dark theme test, following the same pattern used by AppThemeFeatureTests which also uses retryTimeout for theme-change screenshots.

Impact

This single test is responsible for the Controls Entry job failing in 61/91 PR builds. Fixing it should significantly reduce the overall UI test failure rate.

The dark theme screenshot test fails in 67% of PR builds (61/91) with a
consistent 13% image difference. The root cause is a race condition:
after tapping ThemeButton to switch to dark mode, the VerifyScreenshot
call fires before the WinUI theme transition completes.

Changes:
- HostApp: Add hidden ThemeLabel that updates text on theme change,
  providing a deterministic signal that the theme switch completed
- Test: Wait for ThemeLabel text to show 'Dark' before proceeding
- Test: Add retryTimeout of 3 seconds to VerifyScreenshot to handle
  any remaining rendering delay after theme propagation

This follows the same pattern used by AppThemeFeatureTests which also
uses retryTimeout for theme-change screenshot verification.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 4, 2026 23:26
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 4, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34341

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34341"

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a highly flaky UI test (EntryClearButtonShouldBeVisibleOnDarkTheme) that was failing in ~67% of PR builds due to a race condition: the test would take a screenshot mid-theme-transition immediately after tapping the "Change theme" button, capturing the UI before WinUI's async theme rendering was complete, resulting in a consistent ~13% screenshot diff.

Changes:

  • Added a hidden ThemeLabel (via HeightRequest=0, Opacity=0) to the HostApp page to serve as a synchronization signal updated in the button click handler.
  • Added WaitForTextToBePresentInElement("ThemeLabel", "Dark") in the test to wait before taking the screenshot.
  • Added retryTimeout: TimeSpan.FromSeconds(3) to both VerifyScreenshot() calls in the dark theme test.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/Controls/tests/TestCases.HostApp/Issues/Issue32886.cs Adds hidden ThemeLabel field and updates it in the button click handler as a synchronization signal
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue32886.cs Adds WaitForTextToBePresentInElement wait and retryTimeout to VerifyScreenshot for the dark-theme test

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +53 to +55
var newTheme = Application.Current.UserAppTheme != AppTheme.Dark ? AppTheme.Dark : AppTheme.Light;
Application.Current.UserAppTheme = newTheme;
_themeLabel.Text = newTheme == AppTheme.Dark ? "Dark" : "Light";
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _themeLabel.Text = "Dark" update in Button_Clicked happens synchronously right after Application.Current.UserAppTheme = newTheme, but on Windows the actual WinUI visual theme propagation uses DispatcherQueue.TryEnqueue (see Application.Windows.cs:105), which is an asynchronous dispatch. So WaitForTextToBePresentInElement("ThemeLabel", "Dark") only confirms the MAUI-level property assignment occurred; it does not guarantee that the WinUI RequestedTheme update and the subsequent re-render of child controls (e.g., the Entry's clear button color) have completed.

As a result, the test can still take a screenshot mid-theme-render on Windows. The retryTimeout: TimeSpan.FromSeconds(3) on VerifyScreenshot is the real mechanism compensating for this. To make the label a stronger synchronization point, its text should be updated after awaiting the theme change, or inside a RequestedThemeChanged callback rather than synchronously after setting UserAppTheme. As an alternative, a RequestedThemeChanged event on Application.Current could set the label text when the theme change is actually applied.

Note: If the retryTimeout alone is considered sufficient, the ThemeLabel approach is still useful as it reduces unnecessary wait time on fast machines—but in that case the comment "deterministic signal" in the PR description slightly overstates the guarantee provided.

Copilot uses AI. Check for mistakes.
Comment on lines +54 to +56
VerifyScreenshot(cropBottom: 1550, retryTimeout: TimeSpan.FromSeconds(3));
#else
VerifyScreenshot();
VerifyScreenshot(retryTimeout: TimeSpan.FromSeconds(3));
Copy link

Copilot AI Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AppThemeFeatureTests class (and most other screenshot tests in the codebase) consistently uses tolerance: 0.5 alongside retryTimeout for theme-change screenshots (e.g., VerifyScreenshot(tolerance: 0.5, retryTimeout: TimeSpan.FromSeconds(2))). This PR uses retryTimeout: TimeSpan.FromSeconds(3) without tolerance, which deviates from the established pattern for theme-change screenshot tests. Adding tolerance: 0.5 would be consistent with how similar tests handle minor cross-machine rendering differences (as seen in AppThemeFeatureTests.cs:31,40,55,67 etc.).

Copilot uses AI. Check for mistakes.
@PureWeen
Copy link
Copy Markdown
Member Author

PureWeen commented Mar 4, 2026

/azp run maui-pr-uitests

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@sheiksyedm
Copy link
Copy Markdown
Contributor

@PureWeen This failure was already addressed in the February 21, 2026 Candidate PR through the PR linked below. This is not a flaky failure. The test‑related PR was merged directly into main without adding the required base images. I hope it will not failed on recent PRs.

#34233

@PureWeen PureWeen closed this Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants