Add MCP progress reporting to SDK generation, build, and pack tools#14298
Add MCP progress reporting to SDK generation, build, and pack tools#14298
Conversation
There was a problem hiding this comment.
Pull request overview
Adds MCP notifications/progress reporting to long-running azsdk-cli package operations so MCP clients can show heartbeats and milestone-based progress (helping avoid MCP request timeouts during generation/build/pack).
Changes:
- Introduces
ProgressReporterhelper to emit heartbeat progress updates during long-running work and milestone progress updates for step-based progress bars. - Updates
SdkGenerationTool,SdkBuildTool, andPackToolMCP tool methods to accept framework-injectedIProgress<ProgressNotificationValue>?and to report progress/heartbeats. - Updates and adds unit tests to cover signature changes and
ProgressReporterbehavior.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tools/azsdk-cli/Azure.Sdk.Tools.Cli/Tools/Package/SdkGenerationTool.cs | Adds MCP progress injection + milestone/heartbeat reporting around SDK generation flows. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli/Tools/Package/SdkBuildTool.cs | Adds MCP progress injection + milestone/heartbeat reporting around build execution. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli/Tools/Package/PackTool.cs | Adds MCP progress injection + milestone/heartbeat reporting around pack execution. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli/Helpers/ProgressReporter.cs | New helper to send MCP progress notifications with CLI logging fallback. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Tools/Package/SdkGenerationToolTests.cs | Updates tool calls to include the new progress parameter. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Tools/Package/SdkBuildToolTests.cs | Updates tool calls to include the new progress parameter. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Tools/Package/PackToolTests.cs | Updates tool calls to include the new progress parameter. |
| tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Helpers/ProgressReporterTests.cs | Adds unit tests for ProgressReporter heartbeats and reporting behavior. |
Comments suppressed due to low confidence (3)
tools/azsdk-cli/Azure.Sdk.Tools.Cli.Tests/Helpers/ProgressReporterTests.cs:69
- These unit tests rely on real-time Task.Delay timing and assert on the number/order of heartbeat logs (e.g., expecting >= 3 logs after 350ms). This can be flaky under load or on slower CI agents. Consider making the tests deterministic by driving completion with a TaskCompletionSource and verifying that heartbeats are emitted when the heartbeat delay wins, or by injecting a delay/timer abstraction into ProgressReporter for testability so you can advance time without wall-clock sleeps.
public async Task RunWithProgressAsync_SlowTask_EmitsHeartbeats()
{
// Arrange — task takes ~350ms, heartbeat every 100ms → expect initial + ~3 heartbeats
var result = await ProgressReporter.RunWithProgressAsync(
progress: null,
_logger,
"Slow operation",
async ct =>
{
await Task.Delay(350, ct);
return 42;
},
CancellationToken.None,
heartbeatInterval: TimeSpan.FromMilliseconds(100));
// Assert
Assert.That(result, Is.EqualTo(42));
// At least the initial message + 2 heartbeats (timing may vary slightly)
Assert.That(_logger.Logs.Count, Is.GreaterThanOrEqualTo(3));
// First message is the initial progress message (no elapsed)
Assert.That(_logger.Logs[0].ToString(), Does.Contain("Slow operation"));
Assert.That(_logger.Logs[0].ToString(), Does.Not.Contain("elapsed"));
// Subsequent messages include elapsed time
Assert.That(_logger.Logs[1].ToString(), Does.Contain("elapsed"));
}
tools/azsdk-cli/Azure.Sdk.Tools.Cli/Tools/Package/SdkGenerationTool.cs:83
- The initial progress notification always uses total: 3, but the tspconfig.yaml path reports milestones with total: 4 (steps 1-3). This can cause the client progress bar to jump/change totals mid-run. Consider computing the total based on which path is being used (tsp-location vs tspconfig) before the first ReportProgress, or omit total on the initial validation message until the branch is known so totals remain consistent for the full operation.
ProgressReporter.ReportProgress(progress, logger, 0, "Validating inputs", total: 3);
tools/azsdk-cli/Azure.Sdk.Tools.Cli/Helpers/ProgressReporter.cs:89
- RunWithProgressAsync reports "Xs elapsed" but tracks elapsed time by incrementing a counter by the heartbeat interval, which can drift from real wall-clock time due to scheduling delays (and can be noticeably wrong for long operations). Use a Stopwatch (or capture start time and compute actual elapsed each heartbeat) so the displayed elapsed time is accurate.
var workTask = work(ct);
var elapsed = TimeSpan.Zero;
while (!workTask.IsCompleted)
{
// Wait for either the work to complete or the heartbeat interval to elapse
try
{
await Task.WhenAny(workTask, Task.Delay(interval, ct));
}
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{
// Cancellation requested. Fail fast if work has not completed yet.
if (!workTask.IsCompleted)
{
throw;
}
// If work already completed concurrently, exit the loop and
// propagate the actual work result/exception below.
break;
}
if (!workTask.IsCompleted)
{
elapsed += interval;
var elapsedSeconds = (float)elapsed.TotalSeconds;
var message = $"{activityMessage}... ({(int)elapsedSeconds}s elapsed)";
ReportProgress(progress, logger, elapsedSeconds, message);
}
tools/azsdk-cli/Azure.Sdk.Tools.Cli/Helpers/ProgressReporter.cs
Outdated
Show resolved
Hide resolved
| else | ||
| { | ||
| // Fallback: log the progress message for CLI mode. | ||
| logger.LogInformation("[Progress] {Message}", message); |
There was a problem hiding this comment.
I think we could also have some ascii art here to represent progress on the CLI side, or at least a percentage value string.
| elapsed += interval; | ||
| var elapsedSeconds = (float)elapsed.TotalSeconds; | ||
| var message = $"{activityMessage}... ({(int)elapsedSeconds}s elapsed)"; | ||
| ReportProgress(progress, logger, elapsedSeconds, message); |
There was a problem hiding this comment.
In the pack tool, ReportProgress is called manually with values 1-4 for progressValue and a total of 4. But this runs in between those progress messages with a TotalSeconds value and with no total passed in, how does this show up to the user in terms of progress messages and percent to completion?
There was a problem hiding this comment.
They are different progress report. The step number and total are used to control the blue progress bar. The total seconds is the inline text of MCP tool run title. Please refer to my screenshot in description comment.
| progress, | ||
| logger, | ||
| $"Building {languageService.Language} SDK project", | ||
| async (token) => await languageService.BuildAsync(fullPath, CommandTimeoutInMinutes, token), |
There was a problem hiding this comment.
The wrapper/callback approach where the progress reporter runs the task seems overly complex. Stack traces will be more annoying to debug, and the reporter becoming a task runner feels like too much responsibility for the component. Is it enough to run Progress.Start() and Progress.Stop() around BuildAsync(), or some similar pattern? Possibly we could have annoying timing issues though where we report a final progress message after the main task is done.
There was a problem hiding this comment.
Updated it to using/idisposable pattern. Could you have another look? @benbp

Summary
Adds MCP
notifications/progresssupport to long-running tool operations(SdkGenerationTool,SdkBuildTool,PackTool) so MCP clients like VS Code can display real-time progress indicators.Changes
ProgressReporter— new helper with two reporting modes:ProgressReporterfrom a static class to an instance-based class. The constructor takesIProgress<T>,ILogger, andtotalSteps— declared once, not repeated at every call.NextStep(message)auto-increments the step counter and reports progress. No manual step index ortotal:arguments needed. ThrowsInvalidOperationExceptionif more steps are reported than declared.StartHeartbeat(message, ct)returns anIAsyncDisposablescope that sends periodic "still alive" notifications viaPeriodicTimer. Use withawait usingto guarantee the heartbeat loop is fully stopped (no race conditions) before proceeding. Replaces the callback-basedRunWithProgressAsync.Milestone steps
Related issues
#14198
#13335
Client experience
It has a blue progress bar, the tool run title is updated with the elapsed time - 15s.

updated with 30s

The build tool completed after 60 seconds in GH Copilot CLI

CLI progress bar:
