draft-ford-midcom-p2p-01.txt

Peer-to-Peer (P2P) communication across middleboxes

              |                                             |
           Client A                                      Client B
        10.0.0.1:1234                                 10.1.1.3:1234

   NAT A has assigned its own UDP port 62000 to the communication
   session between A and S, and NAT B has assigned its port 31000 to the
   session between B and S.  By communicating through server S, A and B
   learn each other's public IP addresses and port numbers as observed
   by S.  Client A now starts sending UDP messages to port 31001 at
   address 138.76.29.7 (note the port number increment), and client B
   simultaneously starts sending messages to port 62001 at address
   155.99.25.11.  If NATs A and B assign port numbers to new sessions
   sequentially, and if not much time has passed since the A-S and B-S
   sessions were initiated, then a working bi-directional communication
   channel between A and B should result.  A's messages to B cause NAT A
   to open up a new session, to which NAT A will (hopefully) assign
   public port number 62001, because 62001 is next in sequence after the
   port number 62000 it previously assigned to the session between A and
   S.  Similarly, B's messages to A will cause NAT B to open a new
   session, to which it will (hopefully) assign port number 31001.  If
   both clients have correctly guessed the port numbers each NAT assigns
   to the new sessions, then a bi-directional UDP communication channel
   will have been established as shown below.

                                  Server S
                              18.181.0.31:1234
                                     |
                                     |
              +----------------------+----------------------+
              |                                             |
            NAT A                                         NAT B
   A-S 155.99.25.11:62000                        B-S 138.76.29.7:31000
   A-B 155.99.25.11:62001                        B-A 138.76.29.7:31001
              |                                             |
              |                                             |
           Client A                                      Client B
        10.0.0.1:1234                                 10.1.1.3:1234

   Obviously there are many things that can cause this trick to fail.
   If the predicted port number at either NAT already happens to be in
   use by an unrelated session, then the NAT will skip over that port
   number and the connection attempt will fail.  If either NAT sometimes
   or always chooses port numbers non-sequentially, then the trick will
   fail.  If a different client behind NAT A (or B respectively) opens
   up a new outgoing UDP connection to any external destination after A
   (B) establishes its connection with S but before sending its first



Ford, Srisuresh & Kegel                                        [Page 16]

Internet-Draft     P2P applications across middleboxes      October 2003


   message to B (A), then the unrelated client will inadvertently
   "steal" the desired port number.  This trick is therefore much less
   likely to work when either NAT involved is under load.

   Since in practice a P2P application implementing this trick would
   still need to work if the NATs are cone NATs, or if one is a cone NAT
   and the other is a symmetric NAT, the application would need to
   detect beforehand what kind of NAT is involved on either end [STUN]
   and modify its behavior accordingly, increasing the complexity of the
   algorithm and the general brittleness of the network.  Finally, port
   number prediction has no chance of working if either client is behind
   two or more levels of NAT and the NAT(s) closest to the client are
   symmetric.  For all of these reasons, it is NOT recommended that new
   applications implement this trick; it is mentioned here for
   historical and informational purposes.

3.5. Simultaneous TCP open

   There is a method that can be used in some cases to establish direct
   peer-to-peer TCP connections between a pair of nodes that are both
   behind existing middleboxes.  Most TCP sessions start with one
   endpoint sending a SYN packet, to which the other party responds with
   a SYN-ACK packet.  It is possible and legal, however, for two
   endpoints to start a TCP session by simultaneously sending each other
   SYN packets, to which each party subsequently responds with a
   separate ACK.  This procedure is known as a "simultaneous open."

   If a middlebox receives a TCP SYN packet from outside the private
   network attempting to initiate an incoming TCP connection, the
   middlebox will normally reject the connection attempt by either
   dropping the SYN packet or sending back a TCP RST (connection reset)
   packet.  If, however, the SYN packet arrives with source and
   destination addresses and port numbers that correspond to a TCP
   session that the middlebox believes is already active, then the
   middlebox will allow the packet to pass through.  In particular, if
   the middlebox has just recently seen and transmitted an outgoing SYN
   packet with the same addresses and port numbers, then it will
   consider the session active and allow the incoming SYN through.  If
   clients A and B can each correctly predict the public port number
   that its respective middlebox will assign the next outgoing TCP
   connection, and if each client initiates an outgoing TCP connection
   with the other client timed so that each client's outgoing SYN passes
   through its local middlebox before either SYN reaches the opposite
   middlebox, then a working peer-to-peer TCP connection will result.

   Unfortunately, this trick may be even more fragile and timing-
   sensitive than the UDP port number prediction trick described above.
   First, unless both middleboxes are simple firewalls or implement cone



Ford, Srisuresh & Kegel                                        [Page 17]

Internet-Draft     P2P applications across middleboxes      October 2003


   NAT behavior on their TCP traffic, all the same things can go wrong
   with each side's attempt to predict the public port numbers that the
   respective NATs will assign to the new sessions.  In addition, if
   either client's SYN arrives at the opposite middlebox too quickly,
   then the remote middlebox may reject the SYN with a RST packet,
   causing the local middlebox in turn to close the new session and make
   future SYN retransmission attempts using the same port numbers
   futile.  Finally, even though support for simultaneous open is
   technically a mandatory part of the TCP specification [TCP], it is
   not implemented correctly in some common operating systems.  For this
   reason, this trick is likewise mentioned here only for historical
   reasons; it is NOT recommended for use by applications.  Applications
   that require efficient, direct peer-to-peer communication over
   existing NATs should use UDP.


4. Application design guidelines

4.1. What works with P2P middleboxes

  Since UDP hole punching is the most efficient existing method of
  establishing direct peer-to-peer communication between two nodes
  that are both behind NATs, and it works with a wide variety of
  existing NATs, it is recommended that applications use this
  technique if efficient peer-to-peer communication is required, 
  but be prepared to fall back on simple relaying when direct
  communication cannot be established. 

4.2. Peers behind the same NAT

  In practice there may be a fairly large number of users who 
  have not two IP addresses, but three or more. In these cases,
  it is hard or impossible to tell which addresses to send to 
  the registration server. The applications should send all its
  addresses, in such a case.

4.3. Peer discovery
 
  Applications sending packets to several addresses to discover
  which one is best to use for a given peer may become a 
  significant source of 'space junk' littering the net, as the
  peer may have chosen to use routable addresses improperly as
  an internal LAN (e.g. 11.0.1.1, which is assigned to the DOD).
  Thus applications should exercise caution when sending the
  speculative hello packets.

4.4. TCP P2P applications




Ford, Srisuresh & Kegel                                        [Page 18]

Internet-Draft     P2P applications across middleboxes      October 2003


  The sockets API, used widely by application developers, is 
  designed with client-server applications in mind. In its 
  native form, only a single socket can bind to a TCP or UDP
  port. An application is not allowed to have multiple
  sockets binding to the same port (TCP or UDP) to initiate 
  simultaneous sessions with multiple external nodes (or) 
  use one socket to listen on the port and the other sockets
  to initiate outgoing sessions.

  The above single-socket-to-port bind restriction is not a
  problem however with UDP, because UDP is a datagram based
  protocol. UDP P2P application designers could use a single
  socket to send as well as receive datagrams from multiple
  peers using recvfrom() and sendto() calls.

  This is not the case with TCP. With TCP, each incoming and
  outgoing connection is to be associated with a separate
  socket. Linux sockets API addresses this problem with the
  aid of SO_REUSEADDR option. On FreeBSD and NetBSD, this 
  option does not seem to work; but, changing it to use the
  BSD-specific SetReuseAddress call (which Linux doesn't
  have and isn't in the Single Unix Standard) seems to work.
  Win32 API offers an equivalent SetReuseAddress call. 
  Using any of the above mentioned options, an application
  could use multiple sockets to reuse a TCP port. Say, open
  two TCP stream sockets bound to the same port, do a 
  listen() on one and a connect() from the other.
 
4.5. Use of midcom protocol

  If the applications know the middleboxes they would be 
  traversing and these middleboxes implement the midcom
  protocol, applications could use the midcom protocol to
  ease their way through the middleboxes. 

  For example, P2P applications require that NAT middleboxes
  preserve end-point port bindings. If midcom is supported on
  the middleboxes, P2P applications can exercise control over
  port binding (or address binding) parameters such as lifetime,
  maxidletime, and directionality so the applications can both
  connect to external peers as well as receive connections from
  external peers; and do not need to send periodic keep-alives to
  keep the port binding alive. When the application no longer needs
  the binding, the application could simply dismantle the binding, 
  also using the midcom protocol. 


5. NAT Design Guidelines



Ford, Srisuresh & Kegel                                        [Page 19]

Internet-Draft     P2P applications across middleboxes      October 2003



   This section discusses considerations in the design of network
   address translators, as they affect peer-to-peer applications.   

5.1. Deprecate the use of symmetric NATs

   Symmetric NATs gained popularity with client-server
   applications such as web browsers, which only need to initiate
   outgoing connections. However, in the recent times, P2P 
   applications such as Instant messaging and audio conferencing
   have been in wide use. Symmetric NATs do not support the
   concept of retaining endpoint identity and are not suitable
   for P2P applications. Deprecating symmetric NATs is 
   recommended to support P2P applications. 

   A P2P-middlebox must implement Cone NAT behavior for UDP
   traffic, allowing applications to establish robust P2P
   connectivity using the UDP hole punching technique.  
   Ideally, a P2P-middlebox should also allow applications to
   make P2P connections via both TCP and UDP.

5.2. Add incremental cone-NAT support to symmetric NAT devices

   One way for a symmetric NAT device to extend support to P2P
   applications would be to divide its assignable port
   namespace, reserving a portion of its ports for one-to-one
   sessions and a different set of ports for one-to-many
   sessions.

   Further, a NAT device may be explicitly configured with
   applications and hosts that need the P2P feature, so the
   NAT device can auto magically assign a P2P port from the
   right port block. 

5.3. Maintain consistent port bindings for UDP ports

   The primary and most important recommendation of this document for
   NAT designers is that the NAT maintain a consistent and stable
   port binding between a given (internal IP address, internal UDP
   port) pair and a corresponding (public IP address, public UDP 
   port) pair for as long as any active sessions exist using that
   port binding. The NAT may filter incoming traffic on a 
   per-session basis, by examining both the source and destination
   IP addresses and port numbers in each packet. When a node on the
   private network initiates connection to a new external
   destination, using the same source IP address and UDP port as an
   existing translated UDP session, the NAT should ensure that the
   new UDP session is given the same public IP address and UDP port



Ford, Srisuresh & Kegel                                        [Page 20]

Internet-Draft     P2P applications across middleboxes      October 2003


   numbers as the existing session.   

5.3.1. Preserving port numbers