Skip to content

Test observability#52

Merged
kommundsen merged 16 commits intomainfrom
feat/observability
Apr 5, 2026
Merged

Test observability#52
kommundsen merged 16 commits intomainfrom
feat/observability

Conversation

@kommundsen
Copy link
Copy Markdown
Owner

Description

Type of change

  • Bug fix
  • New feature / strategy
  • Refactor (no behavior change)
  • Documentation / chore

Checklist

  • dotnet test src/ passes
  • New behavior is covered by tests (TDD: Red → Green → Refactor)
  • Follows .editorconfig code style

- Records decision to flow ILogger through ConjectureSettings.Logger
  (default NullLogger.Instance) rather than a DI-based factory
- Documents [LoggerMessage] source generator for allocation-free log methods
- Captures hot-path protection policy: no per-draw or per-shrink logging
- Records auto-wiring bridge design (Action<string> delegate avoids test
  framework dependencies in Core)
- Notes deferred items: ActivitySource, Meter, F# API
- Adds Phase 6 implementation plan (16 cycles covering Log.cs, settings
  wiring, adapter auto-wiring, and docs)
…re.Core

Required by the Phase 6 logging implementation; provides ILogger,
NullLogger.Instance, and the [LoggerMessage] source generator support.
Version 10.0.0 matches the existing Microsoft.Extensions.* package versions.
- 11 log methods covering Information (1–6), Warning (7–8), Error (9),
  and Debug (10–11) levels with sequential EventIds
- [LoggerMessage] source generation ensures IsEnabled check happens before
  any string formatting — zero allocations when logging is disabled
- Covers generation, shrinking, targeting, database, and failure events
- Defaults to NullLogger.Instance; users may supply any ILogger via
  new ConjectureSettings { Logger = myLogger }
- ConjectureSettingsAttribute.Apply copies Logger from the baseline
  settings (attributes cannot set ILogger at compile time)
- Registers new public API in PublicAPI.Unshipped.txt
- GenerationCompleted (Info) after generation loop, including elapsed time
- PropertyTestFailure (Error) with hex seed on test failure
- HighUnsatisfiedRatio (Warning, once) when rejection ratio exceeds half the configured limit
- TargetingStarted/TargetingCompleted (Info) around the hill-climbing phase
- ShrinkingStarted (Info) at entry with initial node count
- ShrinkPassProgress (Debug) per pass with class name and progress flag
- ShrinkingCompleted (Info) at exit with final node count, step count, and elapsed time
- 2-arg overload retained (defaults to NullLogger) for existing test call sites
- TestRunner updated to pass settings.Logger at all three ShrinkAsync call sites
- Constructor accepts optional ILogger (defaults to NullLogger.Instance);
  wraps SQLite init in try/catch — logs DatabaseError and sets connection
  to null rather than throwing, so test runs continue without the cache
- Load logs DatabaseReplaying (EventId 6) when stored examples are found
- Save logs DatabaseSaved (EventId 11) on success
- All three methods guard on connection is null and catch/log SQLite errors
  without propagating — database is a caching layer, not a critical path
…ress

- Optional ILogger? logger = null parameter; existing call sites (benchmarks,
  old tests) unchanged
- Logs TargetingStepImproved (EventId 12, Debug) each time greedy search or
  random perturbation improves the score for a label
- TestRunner.RunGenerationCore passes settings.Logger to Climb
- Wraps an Action<string> delegate as ILogger, formatting as "[Level] message"
- IsEnabled respects a configurable minimum level (default: Information)
- FromWriteLine(null) returns NullLogger.Instance for safe null handling
- Uses Action<string> not ITestOutputHelper — Core stays free of test framework deps
- Detects Func<TestOutputHelper> factory in ConstructorArguments (xUnit v2's
  lazy init pattern), stores the instance, and calls Initialize/Uninitialize
  around the test so log writes don't throw
- Passes the same initialized instance to the test class constructor — test
  class and Conjecture share one TestOutputHelper rather than getting two
- Guards TargetingProportion deserialization against an xUnit v2 quirk that
  produces out-of-range values when the test class has constructor parameters
- Creates TestOutputHelperLogger from outputHelper.WriteLine when the
  test class declares ITestOutputHelper in its constructor
- Passes the logger to ConjectureSettings and ExampleDatabase; falls
  back to NullLogger when no constructor arguments are present
- Creates TestOutputHelperLogger from TestContext.Out.WriteLine in
  PropertyTestCommand.Execute() so Conjecture logs appear in NUnit
  test output automatically
- Passes the logger to ConjectureSettings and ExampleDatabase
- Uses a using alias to resolve ambiguity with NUnit.Framework.Internal.ILogger
- Creates TestOutputHelperLogger from Console.WriteLine in
  PropertyAttribute.ExecuteAsync() so Conjecture logs appear in
  MSTest output via CaptureTraceOutput
- Passes the logger to ConjectureSettings and ExampleDatabase
- Microsoft.Extensions.Logging.Abstractions flows transitively from Core
- Passing run: GenerationCompleted emitted at Information (EventId 1)
- Failing run: PropertyTestFailure at Error + ShrinkingStarted/Completed at Information
- Targeting run: TargetingStarted/TargetingCompleted at Information
- High assumption rejection: HighUnsatisfiedRatio at Warning
- NullLogger.Instance: no exceptions, completing normally
- Per-adapter auto-wired output examples (xUnit v2/v3, NUnit, MSTest)
- Custom ILogger integration via ConjectureSettings.Logger
- Log event catalog: EventIds 1-12, levels, and message templates
- Performance notes: [LoggerMessage] source generation, NullLogger defaults
- Added Observability entry to guides toc.yml
@kommundsen kommundsen added documentation Improvements or additions to documentation enhancement New feature or request labels Apr 5, 2026
@kommundsen kommundsen merged commit cc8607f into main Apr 5, 2026
1 check passed
@kommundsen kommundsen deleted the feat/observability branch April 5, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant