[FEATURE] Optional blockIds report from remote shuffle-server to individual spark driver #1538
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the feature
Currently, the client will report blockIds to shuffle-server, that will burden servers' memory pressure. Besides, when implementing serveless shuffle-server or quick graceful upgrade with #1373 , the current impl is not applicable for the future features.
So I think we could make the blockIds report to spark driver.
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!