Best way to to support traceresponse and load balancer deferred sampling?

[plantfansam]

Hi all! Hopefully this is the place for this issue, since I don't have an OTEP in my back pocket. Feel free to bounce me elsewhere (I did try Slack, first)!

What are you trying to achieve?

Implementing load balancer deferred sampling using experimental traceresponse header.

Additional context.

I’ve been looking into load balancer deferred sampling: https://w3c.github.io/trace-context/#load-balancer-deferred-sampling, which is in the latest Trace Context editor’s draft. TL;DR, your load balancer starts (and maybe finishes) some spans, proxies a request somewhere, waits to hear back from the proxied-to service, then starts/finishes more spans, making a sampling choice based on the proxied-to service’s traceresponse header. I’m wondering if it’s possible to make this work in the load balancer in a spec-compliant way, given that AFAIK sampling decisions are made when spans are created. I see this OTEP for adding additional sampling hooks, but it seems dormant.

I think the most compliant thing to do would be to cache all of the span details (start time, end time, attrs, etc.) in the load balancer, then start and finish the spans once the proxied-to service has returned its sampling decision. Less compliant would be to start RECORD_ONLY load balancer spans, save their end timestamps somewhere (if you logically should finish the span before getting the traceresponse header back), then update the RECORD_ONLY spans’ sampling decisions upon reading the traceresponse header (this is the non-compliant part, I think!), then finish the spans (using their cached end timestamps).

So my questions are:

• Am I right that these mechanics are a bit unclear given the existing spec?
• Do we need an OTEP to do this more smoothly? If no, do we want to add some guidance in the spec for those grepping it for traceresponse?

[dyladan]

/cc @kalyanaj looks like a traceresponse implementer

[brunobat]

As far as I know, the traceresponse header is not used anywhere in the Spec or the Java impl.
We have requests for it, for debug purposes.

[dyladan]

I believe this is the same as #3811 which has a lot more activity and discussion. Closing in favor of that