Skip to content

Comments

Feature add new eventstream implementation to support bedrockruntime#InvokeModelWithBidirectionalStream aka bidi#3319

Open
Madrigal wants to merge 23 commits intomainfrom
feat-support-bedrock-bidi
Open

Feature add new eventstream implementation to support bedrockruntime#InvokeModelWithBidirectionalStream aka bidi#3319
Madrigal wants to merge 23 commits intomainfrom
feat-support-bedrock-bidi

Conversation

@Madrigal
Copy link
Contributor

@Madrigal Madrigal commented Feb 11, 2026

Change the codegen for EventStreams in Go

Expands on #3236, closes #3156

Sample diff for an existing operation can be seen at #3320

What was the problem

Some operations (notoriously bedrockagentruntime#invokeModelWithBidirectionalStream) don't return a response immediately, and rather wait for more input from the user before returning a response. Our generated code didn't handle this, and kept waiting forever for a response from the server. This is also a problem with q#Chat, which will be addressed on a separate PR.

This PR changes code generation so we return control to the caller without waiting for a response from the server.

What are the major things changing

  1. New event stream operations now return control immediately after a call to the operation. The output from the operation, if any, is now available only at res.GetInitialReply(). Output objects contain only eventStream and initialReply objects.
  2. The stack unwinding that happens when a response is received on the server now happens only when there's an actual response from the server. This means that client.Operation() completes before the middleware stack finishes executing, and it's only after that the middleware completes execution.

Goals of the PR

  1. Backwards compatibility: Existing clients should continue to work without making changes. All existing event stream operations continue using the same generated path, and don't change with this PR.
  2. Extensibility: If the service doesn't have an initial response but wants to add one later on, they can do so without breaking existing implementations. This is covered by the InitialReply channel

How do I review this PR?

The PR should be reviewed alongside aws/smithy-go#626, which contains the other side of codegen.

What are the different event stream operations?

There are different dimensions of event stream operations

  1. Read from an event stream
  2. Write to an event stream
  3. Has initial response members on the event stream
  4. Needs more than the initial request to send a response (only known at runtime, not modeled)

The existing operations that do this are

Read from Event Stream Write to Event Stream Has Initial Response Members Needs More Than Initial Request Operation
protocoltest.InputStream, protocoltest#InputStreamWithInitialRequest, protocoltest#DuplexStreamWithDistinctStreams
(no operations)
(no operations)
(no operations)
Most common use case, s3#SelectObjectContent,sagemakerruntime#InvokeEndpointWithResponseStream, bedrockagentcore#InvokeCodeInterpreter, bedrockruntime#ConverseStream, bedrockruntime#InvokeModelWithResponseStream, bedrockagentruntime#InvokeAgent, bedrockagentruntime#InvokeFlow, bedrockagentruntime#InvokeInlineAgent, bedrockagentruntime#OptimizePrompt, bedrockagentruntime#RetrieveAndGenerateStream, iotsitewise#InvokeAssistant, lambda#InvokeWithResponseStream, protocoltest#OutputStream, protocoltest#OutputStreamWithInitialResponse
(no operations)
cloudwatchlogs#GetLogObject, cloudwatchlogs#StartLiveTail, kinesis#SubscribeToShard
(no operations)
lexruntimev2#StartConversation, transcribestreaming#StartCallAnalyticsStreamTranscription, transcribestreaming#StartMedicalScribeStream, transcribestreaming#StartMedicalStreamTranscription, transcribestreaming#StartStreamTranscription, protocoltest#DuplexStream,, protocoltest#DuplexStreamWithInitialMessages`
qbusiness#Chat, bedrockruntime#InvokeModelWithBidirectionalStream
(no operations)
(no operations)

@Madrigal Madrigal requested a review from a team as a code owner February 11, 2026 05:09
@lucix-aws
Copy link
Contributor

Approved pending fixing the test failures (looks just like some compile issue, I imagine it's a minor miss).

@lucix-aws lucix-aws self-requested a review February 11, 2026 16:49
@lucix-aws
Copy link
Contributor

there are conflicts apparently

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for InvokeModelWithBidirectionalStream

2 participants