feat(app-server): propagate traces across tasks and core ops by owenlin0 · Pull Request #14387 · openai/codex

owenlin0 · 2026-03-11T20:07:44Z

Summary

This PR keeps app-server RPC request trace context alive for the full lifetime of the work that request kicks off (e.g. for thread/start, this is app-server rpc handler -> tokio background task -> core op submissions). Previously we lose trace lineage once the request handler returns or hands work off to background tasks.

This approach is especially relevant for thread/start and other RPC handlers that run in a non-blocking way. In the near future we'll most likely want to make all app-server handlers run in a non-blocking way by default, and only queue operations that must operate in order (e.g. thread RPCs per thread?), so we want to make sure tracing in app-server just generally works.

Depends on #14300

Before

After

What changed

Keep request-scoped trace context around until we send the final response or error, or the connection closes.
Thread that trace context through detached thread/start work so background startup stays attached to the originating request.
Pass request trace context through to downstream core operations, including:
- thread creation
- resume/fork flows
- turn submission
- review
- interrupt
- realtime conversation operations
Add tracing tests that verify:
- remote W3C trace context is preserved for thread/start
- remote W3C trace context is preserved for turn/start
- downstream core spans stay under the originating request span
- request-scoped tracing state is cleaned up correctly
Clean up shutdown behavior so detached background tasks and spawned threads are drained before process exit.

owenlin0 · 2026-03-11T21:11:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4f17b4cec

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codex-rs/core/src/thread_manager.rs

owenlin0 · 2026-03-11T21:38:54Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cdeed13284

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codex-rs/app-server/src/codex_message_processor.rs

codex-rs/app-server/src/app_server_tracing.rs

owenlin0 · 2026-03-11T22:04:55Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89fb20e99d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codex-rs/core/src/codex.rs

owenlin0 · 2026-03-11T22:22:54Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ed02aec2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-11T22:35:15Z

codex-rs/app-server/src/codex_message_processor.rs

+        self.background_tasks.close();
+        if tokio::time::timeout(Duration::from_secs(10), self.background_tasks.wait())
+            .await
+            .is_err()
+        {
+            warn!("timed out waiting for background tasks to shut down; proceeding");
+        }


Cancel timed-out background tasks before thread shutdown

drain_background_tasks only waits 10s, then logs and returns without aborting unfinished tasks. Shutdown then proceeds to shutdown_threads, which snapshots current threads. A still-running thread_start_task can create a new thread after that snapshot, leaving it alive past shutdown (resource leak / orphaned work).

Useful? React with 👍 / 👎.

valid but not worth the complexity atm

owenlin0 marked this pull request as ready for review March 11, 2026 20:07

owenlin0 force-pushed the owen/request-trace-lineage2 branch 2 times, most recently from 273b27c to 4a2ec69 Compare March 11, 2026 21:09

owenlin0 force-pushed the owen/request-trace-lineage2 branch from 4a2ec69 to a4f17b4 Compare March 11, 2026 21:12

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

codex-rs/core/src/thread_manager.rs Show resolved Hide resolved

owenlin0 force-pushed the owen/request-trace-lineage2 branch from a4f17b4 to cdeed13 Compare March 11, 2026 21:38

owenlin0 changed the title ~~feat(app-server): keep request traces attached to async work~~ feat(app-server): propagate traces across tasks and core op submissions Mar 11, 2026

owenlin0 changed the title ~~feat(app-server): propagate traces across tasks and core op submissions~~ feat(app-server): propagate traces across tasks and core ops Mar 11, 2026

owenlin0 requested review from apanasenko-oai and pakrym-oai March 11, 2026 21:45

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

codex-rs/app-server/src/codex_message_processor.rs Outdated Show resolved Hide resolved

codex-rs/app-server/src/app_server_tracing.rs Show resolved Hide resolved

owenlin0 force-pushed the owen/request-trace-lineage2 branch from 533026b to 21448b6 Compare March 11, 2026 21:53

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

codex-rs/core/src/codex.rs Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 11, 2026

View reviewed changes

apanasenko-oai approved these changes Mar 11, 2026

View reviewed changes

owenlin0 enabled auto-merge (squash) March 11, 2026 23:19

owenlin0 force-pushed the owen/request-trace-lineage2 branch from 7ed02ae to 7871cf0 Compare March 12, 2026 00:25

github-actions bot mentioned this pull request Mar 12, 2026

📊 AI CLI 工具社区动态日报 2026-03-12 gsscsd/big_model_radar#22

Open

owenlin0 added 5 commits March 11, 2026 18:33

feat(app-server): keep request traces attached to async work

96d1211

clean up

ea33f4e

clean up test

390babc

feedback

cea3e37

feedback

07964b8

owenlin0 force-pushed the owen/request-trace-lineage2 branch from 7871cf0 to 07964b8 Compare March 12, 2026 01:34

github-actions bot mentioned this pull request Mar 12, 2026

📊 AI CLI 工具社区动态日报 2026-03-12 rollysys/agents-radar#72

Open

owenlin0 disabled auto-merge March 12, 2026 03:14

owenlin0 merged commit 5bc82c5 into main Mar 12, 2026
35 of 36 checks passed

owenlin0 deleted the owen/request-trace-lineage2 branch March 12, 2026 03:18

github-actions bot locked and limited conversation to collaborators Mar 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(app-server): propagate traces across tasks and core ops#14387

feat(app-server): propagate traces across tasks and core ops#14387
owenlin0 merged 5 commits intomainfrom
owen/request-trace-lineage2

owenlin0 commented Mar 11, 2026 •

edited

Loading

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Mar 11, 2026

Uh oh!

owenlin0 Mar 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

owenlin0 commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

owenlin0 commented Mar 11, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

owenlin0 Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

owenlin0 commented Mar 11, 2026 •

edited

Loading