NAT traversal

[iduartgomez]

This is a general tracking issue to monitor the status of the different techniques that will help with clients behind NAT when it comes to establishing direct connection between peers.

Prelude

One of the problems of NAT traversal is the variance in the network characteristics and configurations when trying to establish a connection between two peers, at every layer of the network and from the pov of both hw (router configuration, LAN, ISP, etc.) and software (for example, is a browser handling the connection for you or do you have direct access to the interface?). A good overview and taxonomy of the different scenarios can be read here.

Some important/interesting resources re. NAT traversal status within libp2p:

• High level overview: https://docs.libp2p.io/concepts/nat/
• Holistic specification for different hole punching techniques: https://github.com/libp2p/specs/blob/master/connections/hole-punching.md
• Overall tracking issue for NAT traversal amongst the different libp2p implementations: NAT traversal tracking issue libp2p/specs#312
• Project flare (decentralized hole punching): Project Flare aka. decentralised hole punching protocol/web3-dev-team#21
• Very good practical and theoretical read on the problems posed by NAT: https://tailscale.com/blog/how-nat-traversal-works/

rust-libp2p status

Overall tracking issue in their repo: libp2p/rust-libp2p#2052

• Automatic router configuration (e.g. UPnP). Not implemented, only an issue: UPnP in the TcpConfig libp2p/rust-libp2p#558
• STUN-like methods (hole-punching)
  • Port reuse in TCP. Implemented. We should enable this and should help.
  • In theory the Identify protocol (which we will be using) helps with this since it provides the observed addresses to anyone querying, so they can communicate through those addresses back (this is handled automatically by libp2p AFAIK).
  • AutoNAT. Lets peers request dial-backs. (protocols/: Add basic AutoNAT implementation libp2p/rust-libp2p#2262)
• TURN-like methods
  • https://docs.libp2p.io/concepts/circuit-relay/
  • Circuit Relay v1 (spec). Completed and we could use it in theory. The problem with TURN like strategies is one of scalability and cost though. Not a reliable long term solution probably.
  • Circuit Relay v2 (spec). Under development: protocols/relay: Implement circuit relay v2 protocol  libp2p/rust-libp2p#2059
  • protocols/: Implement Direct Connection Upgrade through Relay (DCUtR) libp2p/rust-libp2p#2438

For more detail check their tracking issue.

Notes

• Also worth mentioning is QUIC support, which we could use instead of TCP, QUIC builds on top of UDP and will make NAT traversal issues easier too. On top I believe is an straight upgrade in our case since our use case will trend towards a high number of connections but relatively short-lived and small in size on average, and here QUIC will be a good improvement over TCP, and I don't see any downsides to using it over TCP really. QUIC support within rust-libp2p seems to be finally near completion: Libp2p quic second attempt libp2p/rust-libp2p#2159
  • Hole punching success rate in QUIC is higher that over TCP and equivalent to that of UDP, without having to deal with all the quirks of UDP cause those are handled by QUIC (and is encrypted by default too). 80% according to (MIT paper on Hole Punching, 2006).
• Lack of concurrent attempts is a hindrance towards successful hole punching and there is an outstanding issue to address this: core/: Concurrent connection attempts - aka. happy eyeball libp2p/rust-libp2p#1896 (comment)
  • Solved in core/: Concurrent connection attempts libp2p/rust-libp2p#2248
• Multistream select is important for hole punching, since both peers are attempting to initiate the connection and this is at odds to how is currently implemented (there must be one initiator). Ongoing effort to extend the protocol to allow for multiple initiators:
  • misc/multistream-select: Implement simultaneous open extension libp2p/rust-libp2p#2066

Solutions

Working on top of UDP to add our own solutions disqualifies a big chunk of libp2p, as is not possible to use UDP directly as transport protocol in libp2p (support is under development but only in the Go impl, and is just a prototype). There may be an option, in theory, to build up our own protocol and plugin it in the libp2p (is all based around traits/interfaces so I believe is doable, but we would have to look up) and to use other components that we find useful (e.g. all the identity protocol).

However this would mean building up a significant part of the plumbing and packet handling libp2p does for you and would be a large effort so I would say this should be a last option (unless we decide we don't want to use libp2p at all). Then there is the obvious caveat: If a higher number of developers with more resources have taken a long time to tackle this and haven't succeeded (yet) why would we be faster moving? This would be a major distraction from building our core logic, so if we have to at least would be nice to have the support of other developers. Then, I would favor an other approach, that is:

• Identify what additional functionalities/methods are beneficial regarding the efforts to add NAT traversal capabilities.
• Monitor their implementation, and if necessary contribute upstream and help the libp2p folks to push them through the finish line (this would have the added benefit of gaining familiarity with libp2p internals). A bunch of the work to be done has not been done at all, but there is a lot of work which is near completion so I expect within reasonable time there will be more improvements. I believe their idea is to extend Project Flare aka. decentralised hole punching protocol/web3-dev-team#21 (e.g. Go impl: Project Flare(decentralised Hole Punching) Phase1 Meta Issue libp2p/go-libp2p#1039) to all implementations (including Rust) so that is what we can use as guideline and to be the end status when it comes to NAT traversal support in libp2p.
  • If necessary fork and replace/modify whatever parts we deem necessary, however that would be a last option too, ideally we want to to contribute any work upstream and have synergies with the libp2p team and benefit from their expertise.

So we can continue with the current efforts and focus on other areas which are core to our project and make a reasonable assumption that, within reasonable time this will be improved, and if necessary help upstream to improve the situation re. NAT when we run into practical problems as we test.

[iduartgomez]

For the record I am, right now at least, behind a normal (full-cone) NAT, which is the most usual case for home routers and the reason hole-punching is relatively successful apparently.

https://dh2i.com/kbs/kbs-2961448-understanding-different-nat-types-and-hole-punching/

[iduartgomez]

Other options we have been pondering is working directly with a QUIC implementation, this way we could add NAT traversal functionality ourselves and they have encryption built-in:

• the cons is that we would lose some of the libp2p building blocks and it's modular architecture; we would have to include a layer for reliably identifying and connecting peers.
• the pros is that we would have more control from the interface up while leveraging all the goods that the implementations give us (like built-in encryption). That means we could even potentially add UPnP along hole punching.
  We would be using one of this two crates to build on top:
  • https://github.com/quinn-rs/quinn
  • https://github.com/cloudflare/quiche

[mxinden]

👋 Max here, maintainer of (rust-) libp2p. It is great to see interest in libp2p and great to see the many tracking issues and specifications being useful. Feel free to reach out in case you have any questions.

In theory the Identify protocol (which we will be using) helps with this since it provides the observed addresses to anyone querying, so they can communicate through those addresses back (this is handled automatically by libp2p AFAIK).

👍 though the identify is not sufficient in all cases, as e.g. NAT mappings need to be kept alive. Long term we need to support AutoNAT in rust-libp2p.

Lack of concurrent attempts is a hindrance towards successful hole punching and there is an outstanding issue to address this: core/: Concurrent connection attempts - aka. happy eyeball libp2p/rust-libp2p#1896 (comment)

My current work item ;)

[iduartgomez]

Hi @mxinden, thanks for answering! Do you have a rough estimate on when we could see AutoNAT land? And QUIC? (Correct me if wrong but when QUIC is added it will indirectly unblock some of the NAT traversal stuff.)

Likewise, there is anything we can do to help things move re. NAT traversal (helping implementing any of the issues, even if is something easy to get started).

[mxinden]

Do you have a rough estimate on when we could see AutoNAT land?

It is hard to estimate when AutoNAT will land. Though note that AutoNAT is only a small piece of the puzzle and that basic hole punching does not depend on it. I hope for basic hole punching (libp2p/rust-libp2p#2059, libp2p/rust-libp2p#2076, libp2p/rust-libp2p#1896) to be merged and released by end of this year.

And QUIC? (Correct me if wrong but when QUIC is added it will indirectly unblock some of the NAT traversal stuff.)

While not as successful as QUIC, NAT hole punching via TCP is already a great win. In other words, NAT hole punching is not blocked on QUIC support in rust-libp2p. That said, we definitely want to support QUIC, among other things for its increased NAT hole punching success rate.

Likewise, there is anything we can do to help things move re. NAT traversal (helping implementing any of the issues, even if is something easy to get started).

Help is always very much appreciated.

• At first it would be great if you could report any issues you face getting started with rust-libp2p. It's always useful to gain insights from newcomers.

• Test protocols/relay: Implement circuit relay v2 protocol  libp2p/rust-libp2p#2059.

• In my eyes, implementing the AutoNAT protocol in rust-libp2p would be a good follow-up task. It is relatively self-contained. A first attempt can be found on Autonat libp2p/rust-libp2p#1672. A draft of the protocol specification is on add a spec for autonat libp2p/specs#362. I find https://datatracker.ietf.org/doc/html/rfc8445#section-5 to be a great read. I hope our implementation can adopt many of its concepts.

[iduartgomez]

Does not the relay protocol depend on having one server which is not behind NAT listening? In our use case this is not a reliable strategy (although could be fit somehow to cover the extra peers which are behind symmetric NAT maybe).

Help is always very much appreciated. ...

Awesome, I will report any issues I have (none so far, except for this current issue, and I like the modular nature quite a bit) and will try to help out contributing.

[mxinden]

Does not the relay protocol depend on having one server which is not behind NAT listening? In our use case this is not a reliable strategy (although could be fit somehow to cover the extra peers which are behind symmetric NAT maybe).

Yes, libp2p-circuit-relay requires the relay server to be publicly routable. Note however that there is no need to restrict a network to a single central relay server. Instead all public nodes of a network can act as such a server. With libp2p-circuit-relay-v2 the overhead for a server is small. Non public nodes can discover relay servers via e.g. the Kademlia DHT, e.g. looking for the 20 closest public relay servers to their own peer id. This allows us to do hole punching in a decentralized fashion.

I am not aware of any hole punching mechanism that does not require some TURN-like mechanism via some public nodes. E.g. WebRTC requires some mechanism to exchange SDP packets.