Update factory processing performance in queueing#410
Conversation
When handling extremely large worker pools, we're spending excessive amounts of time iterating the map of workers. This is causing slowdowns sometimes, and can be optimized at a small cost of memory util in the factory agent.
We need to pin deeply nested futures, and this was showing in our cluster tests on the new Rust runtime.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #410 +/- ##
==========================================
- Coverage 85.60% 85.56% -0.05%
==========================================
Files 73 73
Lines 13256 13329 +73
==========================================
+ Hits 11348 11405 +57
- Misses 1908 1924 +16 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…n) under low and high worker counts.
Benchmark: Factory Queuer Dispatch ThroughputMeasures raw stable-state dispatch speed — the factory and its entire worker pool
Analysis
|
e8801af to
7e43753
Compare
When handling extremely large worker pools, we're spending excessive amounts of time iterating the map of workers. This is causing slowdowns sometimes, and can be optimized at a small cost of memory util in the factory agent.
The key change is replacing
O(n)linear scans of the worker pool inchoose_target_workerforQueuerRoutingandStickyQueuerRoutingwithO(1)dispatch using aVecDequeof available workers. The improvement should be most visible with large worker pools.