Skip to content

Add optional parameter to configuration providers for non-blocking startup#1745

Open
GeorgeTsiokos wants to merge 5 commits intodapr:masterfrom
GeorgeTsiokos:feature/optional-config-resilience
Open

Add optional parameter to configuration providers for non-blocking startup#1745
GeorgeTsiokos wants to merge 5 commits intodapr:masterfrom
GeorgeTsiokos:feature/optional-config-resilience

Conversation

@GeorgeTsiokos
Copy link
Copy Markdown

@GeorgeTsiokos GeorgeTsiokos commented Mar 13, 2026

Summary

Resolves #1748

  • Adds an optional parameter to AddDaprConfigurationStore, AddStreamingDaprConfigurationStore, and AddDaprSecretStore extension methods so applications can start without blocking on Dapr sidecar availability
  • Hardens LoadInBackgroundAsync in both providers with narrowed exception handling (OperationCanceledException + DaprException) instead of a broad catch (Exception) that silently swallowed programming errors
  • Fixes resource leak by disposing CancellationTokenSource in both providers
  • Fixes StringComparer inconsistency (InvariantCultureIgnoreCaseOrdinalIgnoreCase) in FetchSecretsAsync
  • Makes fields readonly in DaprConfigurationStoreProvider for consistency with DaprSecretStoreConfigurationProvider
  • Restores UTF-8 BOM per .editorconfig and adds missing trailing newlines

Test plan

  • All 52 existing + new unit tests pass across net8.0, net9.0, net10.0
  • New unit tests use GetReloadToken() callbacks for deterministic assertions (no fragile Task.Delay polling)
  • Integration test: optional: true with delayed sidecar — config populates once sidecar starts (ShouldPopulateSecretsWhenSidecarStartsLate)
  • Integration test: optional: false (default) behavior unchanged — Build() blocks until sidecar ready (ShouldBlockAndPopulateSecretsWhenNotOptional)
  • Integration test: disposal during background loading exits cleanly without hanging (ShouldExitCleanlyWhenDisposedDuringBackgroundLoad)
  • Integration tests updated to xUnit v3 (TestContext.Current.CancellationToken pattern)

🤖 Generated with Claude Code

@GeorgeTsiokos GeorgeTsiokos requested review from a team as code owners March 13, 2026 10:37
@GeorgeTsiokos GeorgeTsiokos marked this pull request as draft March 13, 2026 11:47
@WhitWaldo
Copy link
Copy Markdown
Contributor

Do note that I just merged a PR to support xUnit v3 so your unit/integration tests may need a refresh.

…artup

Configuration and secret store providers now accept an `optional` parameter
that allows applications to start without blocking on sidecar availability.
When optional, configuration loads in the background with resilient retry
logic that handles transient Dapr errors gracefully.

- Add `optional` parameter to AddDaprConfigurationStore and AddDaprSecretStore
- Harden LoadInBackgroundAsync to catch OperationCanceledException and
  DaprException separately, avoiding silent swallowing of programming errors
- Dispose CancellationTokenSource in both providers to prevent resource leaks
- Fix StringComparer inconsistency (InvariantCultureIgnoreCase → OrdinalIgnoreCase)
- Mark fields readonly in DaprConfigurationStoreProvider for consistency
- Restore UTF-8 BOM per .editorconfig and add missing trailing newlines
- Add deterministic tests using reload tokens instead of fragile Task.Delay

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: George Tsiokos <george@tsiokos.com>
@GeorgeTsiokos GeorgeTsiokos force-pushed the feature/optional-config-resilience branch from 7acf13b to 50765d7 Compare March 15, 2026 20:36
Tests verify three scenarios using secretstores.local.file (no external deps):
- optional:true with delayed sidecar: config populates once sidecar starts
- optional:false (default): Build() blocks until sidecar ready, secrets populated
- Disposal during background load: exits cleanly with no hang

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: George Tsiokos <george@tsiokos.com>
@GeorgeTsiokos GeorgeTsiokos force-pushed the feature/optional-config-resilience branch from 50765d7 to d33da80 Compare March 15, 2026 20:37
@GeorgeTsiokos
Copy link
Copy Markdown
Author

GeorgeTsiokos commented Mar 15, 2026

A few items I'm intentionally leaving as-is — noting them here for visibility:

Pre-existing catch (Exception) in streaming subscription loop (DaprConfigurationStoreProvider.cs) — The broad catch in the SubscribeConfiguration retry loop predates this PR. It's there to reconnect on any stream error (including grpc status errors that aren't DaprException). Left unchanged to avoid scope creep.

UnsubscribeConfiguration throwing inside catch block — Same pre-existing code, same reasoning.

Exponential backoff — The flat sidecarWaitTimeout delay between retries is intentional. The sidecar typically becomes available within its startup window; exponential backoff would add unnecessary complexity for this use case.

Splitting the try block in LoadInBackgroundAsync — The current structure is deliberately compact to keep the retry logic readable. Happy to revisit if the reviewer feels strongly.

Non-optional blocking-failure test for config store — Pre-existing gap (the non-optional path throws on WaitForSidecar timeout, but there's no unit test for it). Out of scope for this PR.

IsOptional source/builder layer test — The existing end-to-end integration tests in SecretStoreOptionalTests.cs cover this path. A pure unit test for the source layer would be redundant.

@GeorgeTsiokos GeorgeTsiokos marked this pull request as ready for review March 15, 2026 23:28
@WhitWaldo WhitWaldo requested a review from Copilot April 13, 2026 16:51
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an optional parameter to Dapr configuration/secret store configuration providers so apps can start without blocking on sidecar availability, and expands test coverage (unit + integration) for the new behavior.

Changes:

  • Add optional/IsOptional plumbing across configuration sources, providers, and extension methods for configuration store, streaming configuration store, and secret store.
  • Introduce background loading logic and tighten exception handling/cancellation disposal behavior in providers.
  • Add new unit/integration tests (and a new integration test project + testcontainers harness) validating late sidecar startup and clean disposal.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
src/Dapr.Extensions.Configuration/DaprConfigurationStoreExtension.cs Adds optional parameter to configuration store extension methods and passes it into the source.
src/Dapr.Extensions.Configuration/DaprConfigurationStoreSource.cs Adds IsOptional to the source and forwards it to the provider.
src/Dapr.Extensions.Configuration/DaprConfigurationStoreProvider.cs Implements optional non-blocking startup via background loading; updates cancellation/disposal behavior.
src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationExtensions.cs Adds optional parameter across secret-store extension overloads and sets IsOptional.
src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationSource.cs Adds IsOptional and forwards it to the provider.
src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationProvider.cs Implements optional non-blocking startup via background loading; adds IDisposable and refactors fetch logic.
test/Dapr.Extensions.Configuration.Test/DaprConfigurationStoreProviderTest.cs Adds unit tests covering optional behavior for config store providers.
test/Dapr.Extensions.Configuration.Test/DaprSecretStoreConfigurationProviderTest.cs Adds unit tests covering optional behavior for secret store provider.
test/Dapr.IntegrationTest.Configuration/Dapr.IntegrationTest.Configuration.csproj New integration test project for configuration/secret optional behavior.
test/Dapr.IntegrationTest.Configuration/SecretStoreOptionalTests.cs New integration tests validating optional startup, late sidecar, and disposal behavior.
src/Dapr.Testcontainers/Common/DaprHarnessBuilder.cs Adds a builder method for the new secret store harness.
src/Dapr.Testcontainers/Harnesses/SecretStoreHarness.cs New harness for local file-based secret store used by integration tests.
all.sln Adds the new integration test project; also introduces x64/x86 solution configurations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +137 to 141
private async Task FetchDataAsync()
{
if (isStreaming)
{
subscribeTask = Task.Run(async () =>
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the streaming path, the subscription loop relies on broad exception handling and reconnection logic. Consider narrowing exception handling to expected transient failures and adding a backoff/delay before reconnecting to avoid hot-looping under persistent failures, and handle cancellation (OperationCanceledException) explicitly so shutdown doesn’t trigger unnecessary unsubscribe/retry behavior.

Copilot uses AI. Check for mistakes.
Comment on lines +267 to 273
private async Task FetchSecretsAsync()
{
var data = new Dictionary<string, string?>(StringComparer.OrdinalIgnoreCase);

if (secretDescriptors != null)
{
foreach (var secretDescriptor in secretDescriptors)
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Background loading is cancel-driven via cts, but FetchSecretsAsync() uses Dapr client calls without passing a cancellation token. To make disposal/shutdown reliable (especially with optional: true), consider passing cts.Token (or a linked token with a timeout) into the GetSecretAsync/GetBulkSecretAsync calls so in-flight fetches can be cancelled.

Copilot uses AI. Check for mistakes.
Comment on lines 68 to 72
public void Dispose()
{
cts.Cancel();
cts.Dispose();
}
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dispose() cancels and disposes the shared CancellationTokenSource, but background tasks started via Task.Run (and the streaming subscription task) may still be executing and attempting to register with cts.Token. Disposing the CTS while work is still running can cause ObjectDisposedException races inside token registration APIs. Consider storing the background task(s) and waiting for completion after cancellation (or avoid disposing the CTS if you can’t guarantee completion).

Copilot uses AI. Check for mistakes.
public void Dispose()
{
cts.Cancel();
cts.Dispose();
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the configuration store provider, Dispose() cancels and disposes the CancellationTokenSource while a fire-and-forget background task may still be running and registering with cts.Token. To avoid potential ObjectDisposedException races/unobserved exceptions, consider tracking the background task and ensuring it completes after cancellation before disposing the CTS (or skip CTS disposal if you can’t safely wait).

Suggested change
cts.Dispose();

Copilot uses AI. Check for mistakes.
Comment thread all.sln
Comment on lines +260 to +272
Debug|x64 = Debug|x64
Debug|x86 = Debug|x86
Release|Any CPU = Release|Any CPU
Release|x64 = Release|x64
Release|x86 = Release|x86
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x64.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x64.Build.0 = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x86.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x86.Build.0 = Debug|Any CPU
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change adds x64/x86 solution configurations and propagates them across nearly every project configuration entry. If this wasn’t intentional, it’s a large unrelated churn that will increase merge-conflict risk and make future diffs noisier; consider reverting the solution-platform expansion and limiting changes here to adding the new test project entry.

Suggested change
Debug|x64 = Debug|x64
Debug|x86 = Debug|x86
Release|Any CPU = Release|Any CPU
Release|x64 = Release|x64
Release|x86 = Release|x86
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x64.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x64.Build.0 = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x86.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|x86.Build.0 = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{C2DB4B64-B7C3-4FED-8753-C040F677C69A}.Debug|Any CPU.Build.0 = Debug|Any CPU

Copilot uses AI. Check for mistakes.
Comment on lines +20 to +23
var harness = new DaprHarnessBuilder(componentsDir)
.WithEnvironment(environment)
.BuildSecretStore();

Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The harness is disposed manually at the end of the test; if an assertion fails earlier, the harness won’t be disposed and containers/network resources may leak for the remainder of the test run. Consider declaring the harness with await using (it implements IAsyncDisposable) so cleanup happens even on failure.

Copilot uses AI. Check for mistakes.
Comment on lines +75 to +81
public override void Load()
{
if (isOptional)
{
Data = new Dictionary<string, string?>(StringComparer.OrdinalIgnoreCase);
_ = Task.Run(() => LoadInBackgroundAsync());
}
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With optional: true, Load() returns immediately and LoadInBackgroundAsync will populate Data on a background thread while the app may already be reading configuration. Please ensure FetchDataAsync() updates Data atomically (swap in a new dictionary) rather than mutating the existing dictionary in-place (not thread-safe), especially in the non-streaming code path.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add optional parameter to configuration providers

3 participants