Better Rate-Limiting Characteristics #423
Description
Diode version
v1.7.1
Feature type
Change to existing functionality
Proposed feature or enhancement
The reconciler does not currently have a pacing mechanism, which leads to client timeouts when the Netbox instance is under high load. The diode reconciler should have a pacer that modulates the rate of ingest if it detects that server-side apply timeouts are occurring.
Use case
Diode is a powerful sidecar, and it should be considered a "safe asynchronous writer" to Netbox. To be "safe", though, the server should make a "best effort" attempt to commit writes from clients. Being resilient to server-side load is one step in the right direction, and a pacing mechanism is an implementation of that.