Partial orchestration workitem completion support (Merge after next dts dp release)

[YunchuWang]

This pull request introduces support for chunked (partial) orchestrator responses in the gRPC worker protocol, enabling orchestration responses that exceed a configurable size limit to be split into multiple parts. This ensures large orchestrator responses can be handled efficiently and reliably without exceeding message size limits. The changes include protocol updates, worker-side chunking logic, server-side chunk accumulation, and configuration options.

Chunked Orchestrator Response Support

• Added a new isPartial field to the OrchestratorResponse message in orchestrator_service.proto to indicate chunked responses, and updated the protocol version reference. [1] [2]
• Implemented logic in GrpcDurableTaskWorker.Processor.cs to automatically split oversized orchestrator responses into multiple chunks, sending each chunk with isPartial=true except for the last. This includes validation to prevent any single action from exceeding the maximum chunk size. [1] [2]
• Added a configurable property MaxCompleteOrchestrationWorkItemSizePerChunk (default 3.9MB) to GrpcDurableTaskWorkerOptions to control chunk size.

Server-Side Chunk Handling

• Updated TaskHubGrpcServer.cs to accumulate partial orchestrator response chunks per instance, assembling them into a complete response once all chunks are received, and then completing the orchestration work item. Handles both chunked and non-chunked responses. [1] [2] [3]

Other Updates and Minor Improvements

• Added a new orchestrator sample CacheClearingOrchestratorV2 demonstrating orchestration logic.
• Minor code hygiene and using directive updates in affected files. [1] [2] [3] [4] [5] [6]

These changes collectively improve the robustness of orchestrator communication in scenarios with large payloads and add flexibility for future scaling.

[Copilot]

Pull request overview

This PR adds support for partial (chunked) orchestration work item completions to handle scenarios where orchestration responses exceed gRPC message size limits. The implementation automatically splits large responses into multiple chunks that are sent sequentially to the backend.

Key changes:

• Added isPartial field to the OrchestratorResponse protobuf message to indicate chunked completions
• Introduced MaxCompleteOrchestrationWorkItemSizePerChunk configuration option (default 3.9MB) in GrpcDurableTaskWorkerOptions
• Implemented automatic chunking logic in CompleteOrchestratorTaskWithChunkingAsync that splits responses when they exceed the size threshold

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/Grpc/orchestrator_service.proto	Added isPartial field (field 8) to OrchestratorResponse message to support chunked completions
src/Grpc/versions.txt	Updated timestamp to reflect protobuf file update
src/Worker/Grpc/GrpcDurableTaskWorkerOptions.cs	Added MaxCompleteOrchestrationWorkItemSizePerChunk property with default value of 4089446 bytes (3.9MB)
src/Worker/Grpc/GrpcDurableTaskWorker.Processor.cs	Added CompleteOrchestratorTaskWithChunkingAsync method to handle automatic response chunking and modified orchestrator completion to use this new chunking method

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.