Skip to content

Add self-diagnostic mode to pipeline#982

Merged
mthrok merged 1 commit into
mainfrom
profile
Oct 9, 2025
Merged

Add self-diagnostic mode to pipeline#982
mthrok merged 1 commit into
mainfrom
profile

Conversation

@mthrok
Copy link
Copy Markdown
Collaborator

@mthrok mthrok commented Oct 9, 2025

This commit adds self-diagnostic mode, as outlined in #903

When the environment variable SPDL_PIPELINE_DIAGNOSTIC_MODE is set, the build_pipeline function builds the pipeline in self-diagnostic mode.

In this mode, the pipeline benchmark the stage function one-by-one, and reports the performance.

This is useful when one needs to profile the pipeline in prod environment without changing the code.

We will add a mechanism to configure callback and hook in self-diagnostic mode.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 9, 2025
@mthrok mthrok linked an issue Oct 9, 2025 that may be closed by this pull request
This commit adds self-diagnostic mode, as outlined in
#903

When the environment variable `SPDL_PIPELINE_DIAGNOSTIC_MODE`
is set, the build_pipeline function builds the pipeline in
self-diagnostic mode.

In this mode, the pipeline benchmark the stage function
one-by-one, and reports the performance.

This is useful when one needs to profile the pipeline in
prod environment without changing the code.

We will add a mechanism to configure callback and hook in
self-diagnostic mode.
@mthrok mthrok marked this pull request as ready for review October 9, 2025 22:57
@mthrok mthrok merged commit 89a57bc into main Oct 9, 2025
79 of 86 checks passed
@mthrok mthrok deleted the profile branch October 9, 2025 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC] Pipeline Diagnostic Mode

1 participant