feature: headscale support

[realies]

not sure how useful this is given network latency performance reduction, but i already implemented it, so here we are

[AlexCheema]

not sure how useful this is given network latency performance reduction, but i already implemented it, so here we are

I agree about network latency, however this will be useful for exo v2 where we have distributed algo's that work better over slow connections with high latency.

Have you tested this with your own headscale server?

[realies]

didn't think work can be distributed in ways that scales well without nanosecond level latency between all hidden layers, but i'm not very familiar with modern nnet architectures either;

yeah, seemed to work fine over headscale, tested with a mbp and 2xnvidia gpus, all in different networks;

at times when exo is started it produces strange output, sometimes complete gibberish, other times repeating words, but that manifested when testing with two nodes over local udp too

[realies]

@AlexCheema, can we merge this?

[Evanev7]

Thanks so much for your contribution and for taking the time to open this PR.

Since this repository has been fully rewritten and the license has changed, I’m closing all existing open PRs to avoid confusion and to align with the new codebase.

I really appreciate your interest in the project, and you’re very welcome to open a new PR against the updated version if you’d like and we look forward to reviewing it!