Add structured retry observability and metrics hooks

[MagellaX]

Describe your changes

• Add structured retry observability: emit RetryEvent logs for client and throttled retries, with optional metrics callback gated by retry_metrics_enabled.
• Regenerate proto stubs (api/sandbox/task) and add Windows protoc plugin runner helper.
• Add tests for structured retry logging and metrics callback behavior.

Checklists

---

Compatibility checklist

• Client+Server: this change is compatible with old servers (additive logging/metrics only)
• Client forward compatibility: this change ensures client can accept data intended for later versions of itself

---

Release checklist

• Version file (modal_version/__init__.py) has been updated with the next logical version
• Changelog has been cleaned up and given an appropriate subhead

---

Changelog

• Add structured retry observability and optional metrics callback for gRPC retries; emit retry events (client and throttled) with JSON-friendly extras, gated by retry_metrics_enabled.

---

Note

Introduce structured RetryEvent logging and optional metrics callback for gRPC retries, gated by new retry_metrics_enabled config; add tests and a Windows protoc plugin runner.

• gRPC retry observability:
  • Add RetryEvent dataclass and _emit_retry_event for structured INFO logs with extras (attempt, delay, status, throttled, idempotency key).
  • Add _grpc_status_label helper and register_retry_metrics_callback to capture retry events when config.get("retry_metrics_enabled") is true.
  • Instrument _retry_transient_errors to emit events for both server-throttled and client retries.
• Configuration:
  • Add retry_metrics_enabled setting/env (MODAL_RETRY_METRICS_ENABLED) and document it in modal/config.py.
• Tests:
  • Add tests for structured retry logging and metrics callback behavior in test/grpc_utils_test.py.
• Tooling:
  • Add Windows protoc plugin runner protoc_plugin/plugin_runner.bat.

^{Written by Cursor Bugbot for commit 0a61279. This will update automatically on new commits. Configure here.}

[cursor]

PR Summary

Adds structured retry observability to gRPC client retries with optional metrics integration.

• Emit RetryEvent via _emit_retry_event for both server-throttled and client retries; includes fields like call_name, attempt, delay, grpc_status, throttled, and idempotency_key
• Introduce register_retry_metrics_callback and optional metrics emission when config.get("retry_metrics_enabled") is true
• New config toggle retry_metrics_enabled (MODAL_RETRY_METRICS_ENABLED) documented and wired in config.py
• Tests for structured logging and metrics callback (test/grpc_utils_test.py); minor typing update
• Add Windows protoc plugin helper protoc_plugin/plugin_runner.bat

^{Written by Cursor Bugbot for commit f8b990d. This will update automatically on new commits. Configure here.}

[MagellaX]

hey @charlesfrye any takes?