Skip to content

feat: add OpenTelemetry trace correlation for LoggerFactoryLogger#706

Merged
Aaronontheweb merged 18 commits intoakkadotnet:devfrom
Aaronontheweb:feature/otel-trace-correlation
Jan 29, 2026
Merged

feat: add OpenTelemetry trace correlation for LoggerFactoryLogger#706
Aaronontheweb merged 18 commits intoakkadotnet:devfrom
Aaronontheweb:feature/otel-trace-correlation

Conversation

@Aaronontheweb
Copy link
Member

Summary

Implements GitHub issue #700: Add OpenTelemetry trace correlation support to LoggerFactoryLogger.

Problem: Activity.Current (used by OpenTelemetry) doesn't flow across actor mailbox boundaries because it uses AsyncLocal<T>. This means logs generated by actors lose their trace context correlation.

Solution: Leverage LogEvent.ActivityContext added in Akka.NET 1.5.59, which captures trace context at log event creation time before the mailbox crossing.

Changes

  • AkkaLogState - Low-allocation struct using lazy enumeration (yield return) to propagate trace context in MEL state dictionaries. Stores ActivityTraceId/ActivitySpanId as structs directly to avoid ToString() allocations.

  • AkkaTraceContextProcessor - BaseProcessor<LogRecord> that extracts Akka.TraceId, Akka.SpanId, Akka.TraceFlags from log attributes and sets them on LogRecord for OpenTelemetry exporters.

  • AddAkkaTraceCorrelation() - Extension method on OpenTelemetryLoggerOptions for easy configuration.

  • LoggerFactoryLogger - Updated to include ActivityContext in log state when available.

  • Dependencies - Bumped Akka.NET to 1.5.59, Microsoft.Extensions to 8.0.0

  • Aspire Demo - Added demo project (src/Examples/Akka.Hosting.OpenTelemetry.Demo/) with Seq for visual validation

Usage

builder.Logging.AddOpenTelemetry(options =>
{
    options.SetResourceBuilder(ResourceBuilder.CreateDefault()
        .AddService("my-service"));

    // Register processor FIRST (before exporters)
    options.AddAkkaTraceCorrelation();

    options.AddOtlpExporter();
});

services.AddAkka("MySystem", configBuilder =>
{
    configBuilder.ConfigureLoggers(setup =>
    {
        setup.ClearLoggers();
        setup.AddLoggerFactory();
    });
});

Test plan

  • Unit tests for AkkaLogState (8 tests)
  • Unit tests for AkkaTraceContextProcessor (10 tests)
  • All existing tests pass (297 tests)
  • Manual validation with Aspire demo

Closes #700

Implements trace context preservation across actor mailbox boundaries
by leveraging LogEvent.ActivityContext added in Akka.NET 1.5.59.

Key changes:
- Add AkkaLogState struct with lazy enumeration for low-allocation trace
  context propagation in Microsoft.Extensions.Logging state dictionaries
- Add AkkaTraceContextProcessor to extract Akka trace context and set
  LogRecord.TraceId/SpanId/TraceFlags for OpenTelemetry exporters
- Add AddAkkaTraceCorrelation() extension method for easy configuration
- Update LoggerFactoryLogger to include ActivityContext in log state
- Bump Akka.NET to 1.5.59, Microsoft.Extensions to 8.0.0
- Add Aspire demo project for validation

Closes akkadotnet#700
@Aaronontheweb Aaronontheweb marked this pull request as ready for review January 28, 2026 23:14
@Aaronontheweb
Copy link
Member Author

Aaronontheweb commented Jan 28, 2026

Validated this approach works for Phobos log/trace correlation.

We created a prototype test using the AkkaLogState and AkkaTraceContextProcessor implementations from this PR to verify that trace context correctly propagates through the MEL state dictionary. The test confirms that TraceId, SpanId, and TraceFlags are all properly captured and accessible.

This will allow us to deprecate Phobos's AppendLogsToTrace feature (which was a workaround for this exact problem) and point users to LoggerFactoryLogger + AddAkkaTraceCorrelation() as the proper solution.

@Aaronontheweb
Copy link
Member Author

Some screenshots from the Aspire demo:

Traces

image

Logs Correlated to Trace

These were all captured via the ILoggingAdapter

image

Copy link
Member Author

@Aaronontheweb Aaronontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed some of the changes here, have some fixes to make

}
}

private static string SafeFormat(LogEvent log)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While all of these logging changes are still new, want to be careful about not err-ing out if there are formatting issues / config conflicts


private static void SetTraceContext(LogRecord record, ActivityTraceId traceId, ActivitySpanId spanId, ActivityTraceFlags traceFlags)
{
// LogRecord has internal setters, so we need to use reflection
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is true unfortunately, per the path the OTEL authors recommended to us open-telemetry/opentelemetry-dotnet#6085

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to update this to use UnsafeAccessor instead

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol can't - due to .NET Framework

@Aaronontheweb Aaronontheweb added the logging Logging support for Akka.Hosting. label Jan 29, 2026
Copy link
Member Author

@Aaronontheweb Aaronontheweb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detailed my changes - good to go for a beta release IMHO

<AkkaVersion>1.5.59</AkkaVersion>
<MicrosoftExtensionsVersion>[6.0.0,)</MicrosoftExtensionsVersion>
<SystemTextJsonVersion>[6.0.10,)</SystemTextJsonVersion>
<MicrosoftExtensionsVersion>[8.0.0,)</MicrosoftExtensionsVersion>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OTEL packages require these

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added documentation

Copy link
Contributor

@Arkatufus Arkatufus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Aaronontheweb Aaronontheweb merged commit 041a086 into akkadotnet:dev Jan 29, 2026
2 checks passed
@Aaronontheweb Aaronontheweb deleted the feature/otel-trace-correlation branch January 29, 2026 19:50
This was referenced Mar 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logging Logging support for Akka.Hosting.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add OpenTelemetry trace correlation support to LoggerFactoryLogger

2 participants