flaky test: Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.BrowserToken_LoginPage_Failure_DisplayFailureMessage [FAIL]

[danmoseley]

Run Integration Tests / Dashboard (pull_request) test failed in #7516

fail: Microsoft.AspNetCore.SignalR.HubConnectionHandler[1]
      Error when dispatching 'OnDisconnectedAsync' on hub.
      System.ObjectDisposedException: Cannot access a disposed object.
      Object name: 'IServiceProvider'.
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateScope()
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.CreateAsyncScope(IServiceScopeFactory serviceScopeFactory)
         at Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher`1.OnDisconnectedAsync(HubConnectionContext connection, Exception exception)
         at Microsoft.AspNetCore.SignalR.HubConnectionHandler`1.HubOnDisconnectedAsync(HubConnectionContext connection, Exception exception)
fail: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager[3]
      Failed disposing connection Rj_raN-sKfcaDR0B0ci6wQ.
      System.ObjectDisposedException: Cannot access a disposed object.
      Object name: 'IServiceProvider'.
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateScope()
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.CreateAsyncScope(IServiceScopeFactory serviceScopeFactory)
         at Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher`1.OnDisconnectedAsync(HubConnectionContext connection, Exception exception)
         at Microsoft.AspNetCore.SignalR.HubConnectionHandler`1.HubOnDisconnectedAsync(HubConnectionContext connection, Exception exception)
         at Microsoft.AspNetCore.SignalR.HubConnectionHandler`1.RunHubAsync(HubConnectionContext connection)
         at Microsoft.AspNetCore.SignalR.HubConnectionHandler`1.OnConnectedAsync(ConnectionContext connection)
         at Microsoft.AspNetCore.SignalR.HubConnectionHandler`1.OnConnectedAsync(ConnectionContext connection)
         at Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext.ExecuteApplication(ConnectionDelegate connectionDelegate)
         at Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext.WaitOnTasks(Task applicationTask, Task transportTask, Boolean closeGracefully)
         at Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext.DisposeAsync(Boolean closeGracefully)
         at Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager.DisposeAndRemoveAsync(HttpConnectionContext connection, Boolean closeGracefully, HttpConnectionStopStatus status)
  Passed Aspire.Dashboard.Tests.Integration.StartupTests.Configuration_FileConfigDirectoryDoesntExist_Error [6 ms]
  Passed Aspire.Dashboard.Tests.Integration.StartupTests.Configuration_FileConfigDirectoryReloadsChanges_Success [316 ms]
  Passed Aspire.Dashboard.Tests.Integration.Playwright.AppBarTests.AppBar_Change_Theme [1 s]
[xUnit.net 00:00:12.69]     Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.BrowserToken_LoginPage_Failure_DisplayFailureMessage [FAIL]
[xUnit.net 00:00:12.69]       System.TimeoutException : The operation at /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs:81 timed out after reaching the limit of 10000ms.
[xUnit.net 00:00:12.69]       Stack Trace:
[xUnit.net 00:00:12.69]         /_/tests/Shared/AsyncTestHelpers.cs(149,0): at Microsoft.AspNetCore.InternalTesting.AsyncTestHelpers.TimeoutAfter(Task task, TimeSpan timeout, String filePath, Int32 lineNumber)
[xUnit.net 00:00:12.69]         /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs(78,0): at Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.<>c.<<BrowserToken_LoginPage_Failure_DisplayFailureMessage>b__3_0>d.MoveNext()
[xUnit.net 00:00:12.69]         --- End of stack trace from previous location ---
[xUnit.net 00:00:12.69]         /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/Infrastructure/PlaywrightTestsBase.cs(28,0): at Aspire.Dashboard.Tests.Integration.Playwright.Infrastructure.PlaywrightTestsBase`1.RunTestAsync(Func`2 test)
[xUnit.net 00:00:12.69]         /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/Infrastructure/PlaywrightTestsBase.cs(33,0): at Aspire.Dashboard.Tests.Integration.Playwright.Infrastructure.PlaywrightTestsBase`1.RunTestAsync(Func`2 test)
[xUnit.net 00:00:12.69]         /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs(61,0): at Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.BrowserToken_LoginPage_Failure_DisplayFailureMessage()
[xUnit.net 00:00:12.69]         --- End of stack trace from previous location ---
  Failed Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.BrowserToken_LoginPage_Failure_DisplayFailureMessage [10 s]
  Error Message:
   System.TimeoutException : The operation at /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs:81 timed out after reaching the limit of 10000ms.
  Stack Trace:
     at Microsoft.AspNetCore.InternalTesting.AsyncTestHelpers.TimeoutAfter(Task task, TimeSpan timeout, String filePath, Int32 lineNumber) in /_/tests/Shared/AsyncTestHelpers.cs:line 149
   at Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.<>c.<<BrowserToken_LoginPage_Failure_DisplayFailureMessage>b__3_0>d.MoveNext() in /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs:line 78
--- End of stack trace from previous location ---
   at Aspire.Dashboard.Tests.Integration.Playwright.Infrastructure.PlaywrightTestsBase`1.RunTestAsync(Func`2 test) in /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/Infrastructure/PlaywrightTestsBase.cs:line 28
   at Aspire.Dashboard.Tests.Integration.Playwright.Infrastructure.PlaywrightTestsBase`1.RunTestAsync(Func`2 test) in /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/Infrastructure/PlaywrightTestsBase.cs:line 33
   at Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.BrowserToken_LoginPage_Failure_DisplayFailureMessage() in /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs:line 61
--- End of stack trace from previous location ---

Error: System.TimeoutException : The operation at /_/tests/Aspire.Dashboard.Tests/Integration/Playwright/BrowserTokenAuthenticationTests.cs:81 timed out after reaching the limit of 10000ms.

Build Information

Build:
Build error leg or test failing: Aspire.Dashboard.Tests.Integration.Playwright.BrowserTokenAuthenticationTests.BrowserToken_LoginPage_Failure_DisplayFailureMessage
Pull request:

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.TimeoutException",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

[davidfowl]

@JamesNK I think we should up the default timeout on these tests (or get some recordings to figure out why they randomly hang?)

[danmoseley]

where is this 10000 timeout, I don't see it.

System.ObjectDisposedException: Cannot access a disposed object.

also, is this ^^ an uninteresting fallout of a timeout, or a bug to open somewhere

[sebastienros]

The issue I see with the timeout is that it's already at 10s and when it works on the GH agents it's less than 1s, same locally. So something is definitely hanging. I'll send a PR with 1 minute and re-run the same job to see if that helps, or find how long it can be blocked.

[sebastienros]

Got a failure after 7 retries on a 1 min timeout. Something else has a lower default timeout of 30s, but same outcome, increasing the timeout won't help. Every time it's fine it takes less than 1s. Otherwise no bigger delay helps: https://github.com/dotnet/aspire/actions/runs/13403341552/job/37439727190?pr=7672