[Core] Throttle infeasible resource warning#59790
[Core] Throttle infeasible resource warning#59790dayshah merged 2 commits intoray-project:masterfrom
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces throttling for infeasible resource warnings to prevent log spam, which is a valuable improvement. The implementation correctly leverages the Throttler utility and adds a new configuration option for the interval. The code is clear and follows the intended logic. I have one minor suggestion to improve the robustness of the time unit conversion.
ray-gardener
bot
added
question
Signed-off-by: win5923 <ken89@kimo.com>
win5923
force-pushed
the
infeasible-message-warning
branch
from
7a684ce to
77d0c91
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| "of hanging, set the 'RAY_enable_infeasible_task_early_exit=true'. " | ||
| "This feature will be turned on by default in a future release of Ray."; | ||
| RAY_LOG(WARNING) << error_message; | ||
| RAY_LOG_EVERY_MS(WARNING, 60000) << error_message; |
There was a problem hiding this comment.
PublishError call not throttled alongside log message
The log message is rate-limited to once every 60 seconds using RAY_LOG_EVERY_MS, but the gcs_publisher_->PublishError() call remains unthrottled. This means while the log noise is reduced, errors are still published to subscribers at full rate every time the callback executes. The PR description mentions wanting to address the warning being "published repeatedly," but only the logging portion is throttled. The published errors to GCS subscribers will continue at the same frequency as before.
|
can you update the pr description |
|
Thanks! Updated. |
Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: jasonwrwang <jasonwrwang@tencent.com>
Signed-off-by: win5923 <ken89@kimo.com>
Signed-off-by: win5923 <ken89@kimo.com>
Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: peterxcli <peterxcli@gmail.com>
Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: peterxcli <peterxcli@gmail.com>
2 participants
Description
Add rate limiting to infeasible resource request warnings using the RAY_LOG_EVERY_MS macro, with a 60s interval.
Related issues
Closes #59151
Additional information