rfc-0_6-draft.html

p2p技术已经是主流

                       Details about the Deflate compression scheme 
                       may be found at http://www.gzip.org/zlib/
                       and http://www.faqs.org/rfcs/rfc1951.html 

                4      Reserved. This field is currently reserved for
                       future use.  It must be set to 0. 

                3-0    ID Len Value 1-15 can be stored here.  Since 
                       this will not be zero, it ensures this byte 
                       will not be 0x0. 

ID:             The raw binary data in this field is the extension ID.
                The length of this field can range between 1 and 15 
                bytes, and is determined by the Flags field. See 
                section 2.3.2 below on suggestions and rules for 
                creating extension IDs.  No byte in the extension 
                header may be 0x0.

Data Length:    This is the length of the raw extension data.  Please
                note that most Gnutella clients will drop messages, 
                and possibly connections if the message size is 
                larger than a certain threshold (which varies 
                according to message type).  Please pay attention to 
                these limits when creating and bundling new 
                extensions. 

                This field uses an encoding technique that ensures 
                that 0x0 is never the value of any byte.  Steps were 
                also taken to ensure that the encoding is compact. In
                this technique, a length field is the concatenation 
                of length chunks.  The format of each length chunk 
                (which contains 6 bits of length info) is described 
                in bit level below:

                Format:
                76543210
                MLxxxxxx
                        
                M = 1 if there is another length chunk in the 
                sequence, else 0

                L = 1 if this is the last length chunk in the 
                sequence, else 0

                xxxxxx = 6 bits of data.

                01aaaaaa ==> aaaaaa (2^6 values = 0-63)

                10bbbbbb 01aaaaaa ==> bbbbbbaaaaaa 
                (2^12 values = 0-4095)

                10ccccccc 10bbbbbb 01aaaaaa ==> ccccccbbbbbbaaaaaa 
                (2^18 values = 0-262143)

                Boundary Cases:
                0      =                       01 000000b = 0x40 
                63     =                       01 111111b = 0x7f 
                64     =            10 000001  01 000000b = 0x8140 
                4095   =            10 111111  01 111111b = 0xbf7f 
                4096   = 10 000001  10 000000  01 000000b = 0x818040
                262143 = 10 111111  10 111111  01 111111b = 0xbfbf7f
                
                As you see, when the bits are concatenated, the 
                number is in big endian format.

Extension Data  The actual extension data.  The format of this field 
                varies between extensions.  A servent that does not 
                recognize and extension will not be able to parse the
                Extension data, but since the length of this field is
                specified by Data Length, it can still skip to the 
                next extension.  Note that extensions MAY be empty.


2.3.2 Creating Extension IDs

The Extension ID field in the GGEP header is a binary field 
consisting of between 1 and 15 bytes.  It cannot contain the 
byte 0x0, and one must be able to compare IDs with a simple binary 
comparison.  Aside from those rules, GGEP does not mandate any 
particular format, but does encourage the creation of short IDs that 
are free from conflicts.  One should also note that Extension IDs are
meant to be consumed by machines.  Still, the following rules apply.

GDF Registered Extensions:
Any Extension ID of less than 4 bytes MUST be stored in the 
appropriate GDF database.  Any Extension ID of less than 3 bytes must
also be approved by the GDF.  The format of the extension data must 
also be registered.

VendorID Extensions
This simple technique allows for the creation of ExtensionIDs based 
upon uses the following format VendorID.BinaryID

VendorID for a Gnutella servent is a 4 byte value that has been 
registered in the GDF Peer Codes database.  In the QueryHit 
Descriptor, this case is case insensitive.  With ExtensionIDs, the 
case matters, as one must be able to perform a binary comparison on 
the ID.  This means an ExtensionID of "SWAP.1" and "swap.1" are 
different, but both "belong" the vendor who ones the code "SWAP."

This technique may be good for experimental and strictly vendor-
specific extensions, but should be avoided for extension that may be
useful for other vendors as well. Marking an extension by a vendor ID
makes it harder for other vendors to use the extension in their 
servents.

Extension implementers SHOULD publish the ID, format, and expected
data size for their extensions in the GDF database called 
"GGEP Extensions." located at
http://groups.yahoo.com/group/the_gdf/database?method=reportRows&tbl=10
(Requires GDF membership)


3 Protocol Usage

Apart from the protocol definition in section 2, there are also 
some guidelines on how to use the protocol. These are not absolutely
necessary to participate in the network, but very important for an
effective network.


3.1 Flow Control

It is very important that all servents have a system for regulating 
the data that passes through a connection.

The most simple way is to close a connection if it gets overloaded.
A better way is to drop broadcasted packets to reduce the amount of
bandwidth used.  A much better way is to do the following:

Implement an output queue, listing pending outgoing messages in
FIFO order.  As long as the queue is less than, say, 25% of its
max size (in bytes queued, not in amount of messages), do nothing.
If the queue gets filled above 50%, enter flow-control mode.  You
stay in flow-control mode (FC mode for short) as long as the queue
does not drop below 25%.  This is called "hysteresis".

The queue size SHOULD be at least 150% of the maximum admissible
message size.

In FC mode, all incoming queries on the connection are dropped.
The rationale is that we would not want to queue back potentially
large results for this connection since it has a throughput problem.

Messages to be sent to the node (i.e. queued on the output queue)
are prioritized:

* For broadcasted messages, the more hops the packet has traveled,
  the less prioritary it is.  Or the less hops, the more prioritary.
  This means your own queries are the most prioritary (hops = 0).

* For replies (query hits), the more hops the packet has traveled,
  the more prioritary it is.  This is to maximize network usefulness.
  The packet was relayed by many hosts, so it should not be dropped
  or the bandwidth it used would become truly wasted.

* Individual messages are prioritized thusly, from the most
  prioritary to the least: Push, Query Hit, Pong, Query, Ping.
  The Bye message being special, it is always sent (i.e. the queue
  cannot be in FC mode since it needs to be cleared before sending
  Bye).

Normally, all messages are accepted.  However, when the message to
enqueue would make the queue fill to more than 100% of its maximum
size, any queued message less prioritary in the queue is dropped.
If enough room could be made, enqueue the packet.  Otherwise, if the
message is a Query, a Pong or a Ping, drop it.  If not, send a
Bye 502 (Send Queue Overflow) message.

Other known flow-control algorithms:

SACHRIFC is a simple, but still very effective flow control system 
that drops less important packets first. It can be found at:
http://groups.yahoo.com/group/the_gdf/message/5726

A more advanced flow control system can be found at:
http://www.grouter.net/gnutella/


3.2 Network Structure

[TODO: Ultrapeer is so important that the information required to 
implement it will be included here.]

Originally, all Gnutella nodes were connected to each other randomly.
It worked fine for users with broadband connections, but not for 
users with slow modems. That problem can be solved by organizing the 
network in a more structured form. 


3.2.1 Ultrapeer system

[TODO: Describe ultrapeer system here. Handshaking etc. Reference to QRP when used]
[TODO: Ultrapeer marked pongs: size field = power of 2]

The Ultrapeer system has been found effective for this purpose. It is
a scheme to have a hierarchical Gnutella network by categorizing the 
nodes on the network as leaves and ultrapeers. A leaf keeps only a 
small number of connections open, and that is to ultrapeers. An 
ultrapeer acts as a proxy to the Gnutella network for the leaves 
connected to it. This has an effect of making the Gnutella network 
scale, by reducing the number of nodes on the network involved in 
message handling and routing, as well as reducing the actual traffic 
among them.

An ultrapeer only forwards a query to a leaf if it believes the leaf 
can answer it, and leaves never relay queries between ultrapeers. 
Ultrapeers are connected to each other and to "normal" Gnutella hosts
(hosts that do not implement the Ultrapeer system).

An ultrapeer decides what queries to forward to leaf nodes using the
Query Routing Protocol, QRP, which is described in section 3.2.2 
below. If both an ultrapeer and a leaf node supports another protocol
for deciding which queries are forwarded that MAY be used instead.
QRP routing is not used between ultrapeers/normal hosts.

It is RECOMMENDED that servents implement the Ultrapeer system, or 
any future system for decreasing the bandwidth load on modem users.

For more information please read the original specification at:
http://www.limewire.com/developer/Ultrapeers.html


3.2.1.1 Ultrapeer election

Since Gnutella is a decentralized system, ultrapeers are elected 
without the use of a central server. It is up to each node to 
determine if it is to become an ultrapeer or a shielded leaf node.
First, there are some basic requirements that must be satisfied to 
even consider becoming an ultrapeer.

* Not firewalled.  This can usually be approximated by looking at 
whether the host has received incoming connections. 

* Suitable operating system.  Some operating systems handle large 
numbers of sockets better than others.  Linux, Windows 2000/NT/XP, 
and Mac OS/X will typically make better ultrapeers than Windows 
95/98/ME or Mac Classic. 

* Sufficient bandwidth.  At least 15KB/s downstream and 10KB/s 
upstream bandwidth is recommended.  This can be approximated by 
looking at the maximum upload and download throughput. 

* Sufficient uptime.  Ultrapeers should have long expected uptimes.  
A reasonable heuristic is that the expected future uptime is 
proportional to the current uptime.  That is, nodes should not become
ultrapeers until they have been running at least a few hours.

* Sufficient RAM and CPU speed.  Ultrapeers need memory for storing 
routing tables and CPU speed for outing all incoming queries. Exactly
how much is needed depends how efficiently it is implemented and must
be experimented with.

If the above criterias are met, a node is said to be ultrapeer 
capable.  Note that this is not the same as actually being an 
ultrapeer.

Wheneither an ultrapeer capable node will actually become an 
ultrapeer depends on if there is need for more ultrapeers on the
network, and on how well the above criterias are met.  The need for
ultrapeers can be estimated from the noumber of ultrapeers found, and
can be communicated when new connections are established (see below).


3.2.1.2 Ultrapeer Handshaking

Ultrapeer capatibilities and information is exchanges during the 
handskaking sequence when trying to establishing a new Gnutella
connection (see section 2.1). The following new headers are used:

* X-Ultrapeer: "True" signals that node is an ultrapeer, "False" 
signals that the node wants to be a shielded leaf node.

* X-Ultrapeer-Needed: Used to balance the number of ultrapeers. [TODO: Write more about this one]

* X-Try-Ultrapeers: Like X-Try (see section 2.1), but contains only 
addresses of ultrapeers.

* X-Query-Routing: Signals support for the Query Routing Protocol
(section 3.2.2). The header value is the QRP version (curretly 0.1).

It is important to note that headers can be sent in any order.  
Also, case is ignored in "True" and "False".

Here is a sample interaction where a leaf connects to an ultrapeer.

   Leaf                             Ultrapeer
   -----------------------------------------------------------
   GNUTELLA CONNECT/0.6<cr><lf>
   User-Agent: LimeWire/1.0<cr><lf>
   X-Ultrapeer: False<cr><lf>
   X-Query-Routing: 0.1<cr><lf>
   <cr><lf>
                                    GNUTELLA/0.6 200 OK<cr><lf>
                                    User-Agent: LimeWire/1.0<cr><lf>
                                    X-Ultrapeer: False<cr><lf>
                                    X-Ultrapeer-Needed: False<cr><lf>
                                    X-Query-Routing: 0.1<cr><lf>
                                    X-Try: 24.37.144:6346,<cr><lf>