perf: use a constant number of ranch instances

[v0idpwn]

Instead of starting one ranch instance per pool, use the same N ranch instances for all pools.

Each ranch was starting a minimum of 10 acceptors (and consequently, 10 connection supervisors). With 10_000 pools, these are 200_000 processes, which consume a sizeable amount of memory and resources. They also added complexity in managing the separate ranch instances (need to start and finish them at appropriate times, specially because they weren't linked to the pool).

The ranch acceptors/connection supervisors aren't a bottleneck, and if they were, we can control the number of acceptors/supervisors through configuration. In synthesis, there's no benefit in starting bringing multiple ranch instances, only the resource consumption drawback.

[abc3]

Yeah, a good initiative. Just adding some context for why it was originally done this way.

The initial approach aimed to provide tenant-level network isolation and reduce the risk of cross-tenant access through internal proxying. It also allowed us to observe TCP proxying behavior under these conditions.

We considered both unique ports per tenant and shared ones. For the shared setup, the idea was to maintain a pool of Ranch instances and select one via something like :erlang.phash2(tenant_id, ...) to avoid a single point of failure. We started with unique ports and planned to revisit these options later.

Note that with a single port the number of concurrent connections from a single client machine is limited by the number of available local ports on the client, which is typically around 64k.

Creating Ranch instances is resource-intensive mainly during initialization. When idle, they add negligible overhead.

Here are load charts with 10k local Ranch instances. Scheduler usage is about the same as without them, and memory stays around 1GB

[v0idpwn]

Note that with a single port the number of concurrent connections from a single client machine is limited by the number of available local ports on the client, which is typically around 64k.

That's a great point! Will add sharding in the PR. (edit: done on 43da5f3)

Creating Ranch instances is resource-intensive mainly during initialization. When idle, they add negligible overhead.

The memory burden is considerable. In a sample production node, we have just 4700 pools, and :ranch_conns_sup and ranch_acceptor alone take over 1gb of ram in process memory (with 47000 acceptors and 47000 ranch_conns_sup). That's over 1/3 of the total ram consumption of the node. Another option would be to reduce the acceptor count, but specially since "regular" connections use shared ranch instances, I generally think that these connections should too.

[v0idpwn]

I went with a default of 4 shards per mode, which is probably enough for most workloads. If we need, we can increase it in production.

[abc3]

Here we need to get a value from ranch by id, because the port is 0. Although, it might be better to start the range from a defined value to make this behavior more deterministic

Suggested change

	Logger.notice("Proxy started #{opts.mode}(local=#{opts.local}) on port #{port}")
	port = :ranch.get_port(key)
	Logger.notice("Proxy started #{opts.mode}(local=#{opts.local}) on port #{port}")

[v0idpwn]

Added the port!

I'd argue that starting on a range may be "less deterministic". Here's the behaviour is more predictable: we listen on 0, and the OS gives us a port that will always be free. If we try to determine it ourselves, we might get conflicts, etc.

[abc3]

we can choose something that we are sure will be free, for example, starting at 45_000

[abc3]

and we will always know that these ports are used by local proxies

[v0idpwn]

There's no such thing as surely free ports, specially in the ephemeral range :P

[abc3]

well, there are a bunch of strict "safe" examples that supavisor uses: 4000, session's, transaction's, etc 😜

[v0idpwn]

But these aren't in the ephemeral range...

[v0idpwn]

We could maybe pick something outside it though (like 12_000 or smth) 🤔