OpenTelemetry #2

[yashgorana]

Description

OpenTelemetry

Syft Backend

• Update OTel packages
• Add OTel auto-instrumentation (programmatic instead of opentelemetry-instrument root proc)
  • fastapi
  • pymongo
  • botocore
• Fix OTel exporter (Jaeger --> OTLP)
• Fix OTel Spans (remove @instrument from classes, @service_methods are always instrumented)
• Custom workers (in-progress)
  • Plumb OTEL_SERVICE_NAME, OTEL_EXPORTER_* env vars to worker pool launch
• ZMQ tracing (in-progress)
  • instrument zmq messages except heartbeat
  • zmq messages need to include OTLP span for it to be picked up by the consumer (like Traceparent header)

Syft Client

• Add OTel span to Syft Client (todo. a trace should capture client -> backend ->worker -> db)
  • Include requests auto-instrumenter
  • Client needs an API endpoint to export client traces to backend.
• Backend uses syft client, in this case we need to export directly to internal provider (???)

Infra

• Dev - UI (tox -re dev.k8s.install.signoz)
• OTel operator

Affected Dependencies

List any dependencies that are required for this change.

How has this been tested?

• Describe the tests that you ran to verify your changes.
• Provide instructions so we can reproduce.
• List any relevant details for your test configuration.

Checklist

• I have followed the Contribution Guidelines and Code of Conduct
• I have commented my code following the OpenMined Styleguide
• I have labeled this PR with the relevant Type labels
• My changes are covered by tests

[madhavajay]

Great work! 🙌