Add LiveSelectionAlgorithm

[leszko]

Add LiveSelectionAlgorithm, which for now only takes into consideration the maxPrice (the Orchestrator's price needs to be below the maxPrice value set by Gateway). Then, it takes the first Orchestrators in the list (which is sorted by the initial latency).

This new selection algorithm ignores other factors like:

• Stake
• Performance Score
• Price

Fix https://linear.app/livepeer/issue/INF-239/public-os-selected-regardless-of-latency

[linear]

INF-239 Public Os selected regardless of latency

[codecov]

Codecov Report

Attention: Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.

Project coverage is 31.75223%. Comparing base (f05f129) to head (85f1915).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
server/ai_session.go	0.00000%	4 Missing ⚠️

Additional details and impacted files

@@                 Coverage Diff                 @@
##              master       #3678         +/-   ##
===================================================
+ Coverage   31.73926%   31.75223%   +0.01297%     
===================================================
  Files            156         156                 
  Lines          47342       47351          +9     
===================================================
+ Hits           15026       15035          +9     
  Misses         31434       31434                 
  Partials         882         882                 

Files with missing lines	Coverage Δ
server/selection_algorithm.go	91.48936% <100.00000%> (+0.90112%)	⬆️
server/ai_session.go	7.20930% <0.00000%> (ø)

... and 2 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f05f129...85f1915. Read the comment docs.

Files with missing lines	Coverage Δ
server/selection_algorithm.go	91.48936% <100.00000%> (+0.90112%)	⬆️
server/ai_session.go	7.20930% <0.00000%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[j0sh]

Does this mean gateways will tend to select the same orchestrators every time?

This is OK for now but I wonder about the effects of concentrating work among a few "favored" Os. It also might end up affecting startup time if the gateway's preferred orchs are busy.

A refinement for later could be to have a maximum latency target, and randomly select from orchestrators under that target. If nobody is available within the target, then the target can be broadened incrementally.

[leszko]

Does this mean gateways will tend to select the same orchestrators every time?
Yes, but it will be mostly like it worked when we had only TensorDock machines. The best latency always takes precedence.

This is OK for now but I wonder about the effects of concentrating work among a few "favored" Os. It also might end up affecting startup time if the gateway's preferred orchs are busy.
If Os are busy, then they'll be out of pool, so it will not affect the startup time. Busy Os are automatically removed from the pool (every 6m).

A refinement for later could be to have a maximum latency target, and randomly select from orchestrators under that target. If nobody is available within the target, then the target can be broadened incrementally.
Yeah, we'll still need to think what is the best Selection Algorithm for Public Os. I think that now, the simple solution of always sorting by latency is good enough. But when we have more traffic, then the future is to design the criteria that will be considered during the selection, like: price, stake, etc.