botocore: add basic tracing for Bedrock InvokeModelWithStreamResponse

[xrmx]

Description

This adds to the botocore bedrock extensions ability to trace InvokeModelWithStreamResponse calls for amazon.titan, amazon.nova and anthropic claude models using the completion api for amazon.titan and the messages api for amazon.nova and anthropic.claude.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• tox -e py310-test-instrumentation-botocore-1

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[codefromthecrypt]

Can you ad-hoc test langchain? This is the more important PR as it is the default mode.

from langchain_aws import ChatBedrock


def main():
    llm = ChatBedrock(
        model_id='amazon.titan-text-lite-v1',
        streaming=True,
    )
    response_stream = llm.stream("Write a short poem on OpenTelemetry.")
    for chunk in response_stream:
        print(chunk.content, end="")

if __name__ == "__main__":
    main()

[codefromthecrypt]

sorry about the "requested changes" that was a clicking accident

[xrmx]

Can you ad-hoc test langchain? This is the more important PR as it is the default mode.

from langchain_aws import ChatBedrock


def main():
    llm = ChatBedrock(
        model_id='amazon.titan-text-lite-v1',
        streaming=True,
    )
    response_stream = llm.stream("Write a short poem on OpenTelemetry.")
    for chunk in response_stream:
        print(chunk.content, end="")

if __name__ == "__main__":
    main()

I did test it with Claude :

opentelemetry-instrument --metrics_exporter=None --logs_exporter=none --traces_exporter=console python3 langchaininvoke.py 
Here is a short poem about OpenTelemetry:

OpenTelemetry

Tracing spans across the stack,  
Metrics flowing down the track, 
Logs revealing what is back,
OpenTelemetry lights the path.

APIs for traces, metrics, logs,
Instrument what the system cogs,
Monitor all activity, 
Gain observability. 

Distributed tracing's friend, 
Performance issues it can end, 
Helps debug and comprehend, 
OpenTelemetry's ascend.

Promoted by the CNCF,  
Adopted near and far offshore,
Troubleshoot apps end-to-end,
OpenTelemetry's the trend.{
    "name": "chat anthropic.claude-v2",
    "context": {
        "trace_id": "0x9537c79190fad4e7ed7ef72a7f3b14dc",
        "span_id": "0x3ca92907de5050ed",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2025-01-27T09:06:48.816808Z",
    "end_time": "2025-01-27T09:06:54.934687Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "rpc.system": "aws-api",
        "rpc.service": "Bedrock Runtime",
        "rpc.method": "InvokeModelWithResponseStream",
        "aws.region": "us-east-1",
        "gen_ai.system": "aws.bedrock",
        "gen_ai.request.model": "anthropic.claude-v2",
        "gen_ai.operation.name": "chat",
        "gen_ai.request.max_tokens": 1024,
        "aws.request_id": "5e957c6e-b73d-4630-8608-c30ea3486473",
        "retry_attempts": 0,
        "http.status_code": 200,
        "gen_ai.usage.input_tokens": 18,
        "gen_ai.usage.output_tokens": 150,
        "gen_ai.response.finish_reasons": [
            "end_turn"
        ]
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.30.0.dev0",
            "telemetry.auto.version": "0.51b0.dev",
            "service.name": "unknown_service"
        },
        "schema_url": ""
    }
}

and titan:

opentelemetry-instrument --metrics_exporter=None --logs_exporter=none --traces_exporter=console python3 langchaininvoke.py 

The OpenTelemetry project is an open-source observability framework that provides a way to collect, process, and export telemetry data from applications.
It's a great tool for developers to monitor and troubleshoot their applications in production.
OpenTelemetry collects data from multiple sources, such as distributed tracing, logs, metrics, and traces.

It's easy to use and can be integrated with many popular programming languages and frameworks, including Java, Python, and JavaScript.

One of the key features of OpenTelemetry is its extensibility.

You can create custom collectors and exporters to collect data from any source you{
    "name": "text_completion amazon.titan-text-lite-v1",
    "context": {
        "trace_id": "0x5d9b0022535e0d2dd83adefa1d69050f",
        "span_id": "0xf62658b57325c053",
        "trace_state": "[]"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": null,
    "start_time": "2025-01-27T09:06:31.286196Z",
    "end_time": "2025-01-27T09:06:35.209370Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "rpc.system": "aws-api",
        "rpc.service": "Bedrock Runtime",
        "rpc.method": "InvokeModelWithResponseStream",
        "aws.region": "us-east-1",
        "gen_ai.system": "aws.bedrock",
        "gen_ai.request.model": "amazon.titan-text-lite-v1",
        "gen_ai.operation.name": "text_completion",
        "aws.request_id": "1ae8f25e-fff8-42ac-82a5-d76272177f8a",
        "retry_attempts": 0,
        "http.status_code": 200,
        "gen_ai.usage.input_tokens": 15,
        "gen_ai.usage.output_tokens": 128,
        "gen_ai.response.finish_reasons": [
            "LENGTH"
        ]
    },
    "events": [],
    "links": [],
    "resource": {
        "attributes": {
            "telemetry.sdk.language": "python",
            "telemetry.sdk.name": "opentelemetry",
            "telemetry.sdk.version": "1.30.0.dev0",
            "telemetry.auto.version": "0.51b0.dev",
            "service.name": "unknown_service"
        },
        "schema_url": ""
    }
}

[codefromthecrypt]

great. langchain uses will be happy, as will others!