Skip to content

Add call_id to LLM events for correlating requests#4281

Merged
vinibrsl merged 1 commit intomainfrom
llm-event-correlation
Feb 3, 2026
Merged

Add call_id to LLM events for correlating requests#4281
vinibrsl merged 1 commit intomainfrom
llm-event-correlation

Conversation

@vinibrsl
Copy link
Member

@vinibrsl vinibrsl commented Jan 26, 2026

When monitoring LLM events, consumers need to know which events belong to the same API call. Before this change, there was no way to correlate LLMCallStartedEvent, LLMStreamChunkEvent, and LLMCallCompletedEvent belonging to the same request.


Note

Medium Risk
Introduces a new required field on all LLMEventBase events and threads it through multiple providers and streaming paths, which could break any remaining event emissions that don’t set call_id or rely on prior event schemas.

Overview
Adds call_id correlation to LLM events. LLMEventBase now includes a required call_id, and all LLM lifecycle events (started/chunks/completed/failed) are updated to emit this identifier.

Implements call scoping and propagation. Introduces llm_call_context() + get_current_call_id() (contextvars-backed) in BaseLLM, wraps LLM.call/LLM.acall and native provider call/acall implementations to establish a call scope, and attaches the current call_id to streaming chunks and error events.

Updates and adds tests. Adjusts existing streaming/event tests to include call_id, and adds VCR-backed tests + cassettes asserting same-call events share an ID and separate calls generate distinct IDs.

Written by Cursor Bugbot for commit 3059c48. This will update automatically on new commits. Configure here.

@vinibrsl vinibrsl force-pushed the llm-event-correlation branch from e69de9f to d4b2828 Compare January 26, 2026 15:16
@Vidit-Ostwal
Copy link
Contributor

Vidit-Ostwal commented Jan 26, 2026

Hey @vinibrsl,
Not really sure, but this PR was merged which kind of does the same functionality
#3842

While streaming, there is no way to figure out which streaming chunks belongs to same chunk event, this PR add the response_id which is directly given by the provider.

Though this only targets LLMStreamChunkEvent
But I can extend it to LLMCallStartedEvent and LLMCallCompletedEvent as well.

@vinibrsl
Copy link
Member Author

Hey @vinibrsl, Not really sure, but this PR was merged which kind of does the same functionality #3842

While streaming, there is no way to figure out which streaming chunks belongs to same chunk event, this PR add the response_id which is directly given by the provider.

Though this only targets LLMStreamChunkEvent But I can extend it to LLMCallStartedEvent and LLMCallCompletedEvent as well.

Thanks, @Vidit-Ostwal! That's great context.

Unless I'm missing something here, call_id serves a different purpose. When emitting LLMCallStartedEvent, for example, we don't have the response ID yet. Therefore, having some kind of ID that correlates all LLM events is still needed.

Let me know if you think we should consolidate somehow or if keeping both makes sense.

@Vidit-Ostwal
Copy link
Contributor

Vidit-Ostwal commented Jan 26, 2026

Hey @vinibrsl, Not really sure, but this PR was merged which kind of does the same functionality #3842
While streaming, there is no way to figure out which streaming chunks belongs to same chunk event, this PR add the response_id which is directly given by the provider.
Though this only targets LLMStreamChunkEvent But I can extend it to LLMCallStartedEvent and LLMCallCompletedEvent as well.

Thanks, @Vidit-Ostwal! That's great context.

Unless I'm missing something here, call_id serves a different purpose. When emitting LLMCallStartedEvent, for example, we don't have the response ID yet. Therefore, having some kind of ID that correlates all LLM events is still needed.

Let me know if you think we should consolidate somehow or if keeping both makes sense.

This makes more sense,

The PR I add only targets the LLMStreamChunkEvent, which only if successful will be populated, but there are also other LLMStream events which needs to be coupled together, this PR adds that.

Said that, this does makes the previous PR a bit redundant, as response_id has smaller scope, than call_id. which is trying to do the same functionality.
lmk if you think we should drop it, can make a PR for that.

@vinibrsl
Copy link
Member Author

Said that, this does makes the previous PR a bit redundant, as response_id has smaller scope, than call_id. which is trying to do the same functionality.

We could follow up by marking response_id as deprecated, and eventually drop it in the next version. Thanks!

@Vidit-Ostwal
Copy link
Contributor

Said that, this does makes the previous PR a bit redundant, as response_id has smaller scope, than call_id. which is trying to do the same functionality.

We could follow up by marking response_id as deprecated, and eventually drop it in the next version. Thanks!

It was merged today, 8 hrs back.

@vinibrsl vinibrsl force-pushed the llm-event-correlation branch from d4b2828 to 2f0cdf1 Compare February 2, 2026 20:05
When monitoring LLM events, consumers need to know which events belong
to the same API call. Before this change, there was no way to correlate
LLMCallStartedEvent, LLMStreamChunkEvent, and LLMCallCompletedEvent
belonging to the same request.
@vinibrsl vinibrsl force-pushed the llm-event-correlation branch from 2f0cdf1 to 3059c48 Compare February 3, 2026 12:56
@vinibrsl vinibrsl merged commit 576b74b into main Feb 3, 2026
45 checks passed
@vinibrsl vinibrsl deleted the llm-event-correlation branch February 3, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants