protocol.txt

libbt-1.04

*  resp = {
*     failure reason => 'error text'
*       - or -
*     interval => 12345
*     peers => {
*         peer id => 'identifier'
*         ip => 'ip-address' -or- 'hostname'
*         port => 12345
*     }
*  }


Tracker responses are bencoded dictionaries. If a tracker response has a
key 'failure reason', then that maps to a human readable string which
explains why the query failed, and no other keys are required.
Otherwise, it must have two keys - 'interval', which maps to the number
of seconds the downloader should wait between regular rerequests, and
'peers'. 'peers' maps to a list of dictionaries corresponding to peers,
each of which contains the keys 'peer id', 'ip', and 'port', which map
to the peer's self-selected id, ip address or dns name as a string, and
port number, respectively. Note that downloaders may rerequest on
nonscheduled times if an event happens or they need more peers.

If you want to make any extensions to metainfo files or tracker queries,
please coordinate with Bram Cohen <mailto:[e-mail]> to make
sure that all extensions are done compatibly.

* BitTorrent's peer protocol operates over TCP
*
*   peer = {
*   }
*
BitTorrent's peer protocol operates over TCP. It performs efficiently
without setting any socket options.

Peer connections are symmetrical. Messages sent in both directions look
the same, and data can flow in either direction.

The peer protocol refers to pieces of the file by index as described in
the metainfo file, starting at zero. When a peer finishes downloading a
piece and checks that the hash matches, it announces that it has that
piece to all of its peers.

Connections contain two bits of state on either end - choked or not, and
interested or not. Choking is a notification that no data will be sent
until unchoking happens. The reasoning and common techniques behind
choking are explained later in this document.

Data transfer takes place whenever one side is interested and the other
side is not choking. Interest state must be kept up to date at all times
- whenever a downloader doesn't have something they currently would ask
a peer for in unchoked, they must express lack of interest, despite
being choked. Implementing this properly is tricky, but makes it
possible for downloaders to know which peers will start downloading
immediately if unchoked.

Connections start out choked and not interested.

When data is being transferred, downloaders should keep several piece
requests queued up at once in order to get good TCP performance (this is
called 'pipelining'.) On the other side, requests which can't be written
out to the TCP buffer immediately should be queued up in memory rather
than kept in an application-level network buffer, so they can all be
thrown out when a choke happens.

*    Handshake:
*    \x13BitTorrent protocol\0\0\0\0\0\0\0\0<sha1 info hash><20byte peerid>
*
*    Keep alive:
*    \x0000000
*
*    0 - choke:
*    \x0000001\00
*    
*    1 - unchoke:
*    \x0000001\01
*
*    2 - interested:
*    \x0000001\02
* 
*    3 - not interested:
*    \x0000001\03
*
*    4 - have:
*    \x0000005\04<index>
*
*    5 - bitfield:
*    \x00000??\05<bitfield-data>
*
*    6 - request:
*    \x000000d\06<index><begin><length>
*    length must be less than 2^17, recommended 2^15, regardless of pieces size
*
*    7 - piece:
*    \x???????\07<index><begin><data>
*
*    8 - cancel:
*    \x000000d\08<index><begin><length>

The peer wire protocol consists of a handshake followed by a
never-ending stream of length-prefixed messages. The handshake starts
with character ninteen followed by the string 'BitTorrent protocol'. The
leading character is a length prefix, put there in the hope that other
new protocols may do the same and thus be trivially distinguishable from
each other.

All later integers sent in the protocol are encoded as four bytes
big-endian.

After the fixed headers come eight reserved bytes, which are all zero in
all current implementations. If you wish to extend the protocol using
these bytes, please coordinate with Bram Cohen
<mailto:[e-mail]> to make sure all extensions are done
compatibly.

Next comes the 20 byte sha1 hash of the bencoded form of the 'info'
value from the metainfo file. (This is the same value which is announced
as info_hash to the tracker, only here it's raw instead of quoted here).
If both sides don't send the same value, they sever the connection. The
one possible exception is if a downloader wants to do multiple downloads
over a single port, they may wait for incoming connections to give a
download hash first, and respond with the same one if it's in their list.

After the download hash comes the 20-byte peer id which is reported in
tracker requests and contained in peer lists in tracker responses. If
the receiving side's peer id doesn't match the one the initiating side
expects, it severs the connection.

That's it for handshaking, next comes an alternating stream of length
prefixes and messages. Messages of length zero are keepalives, and
ignored. Keepalives are generally sent once every two minutes, but note
that timeouts can be done much more quickly when data is expected.

All non-keepalive messages start with a single byte which gives their
type. The possible values are -


    * 0 - choke
    * 1 - unchoke
    * 2 - interested
    * 3 - not interested
    * 4 - have
    * 5 - bitfield
    * 6 - request
    * 7 - piece
    * 8 - cancel

Choke, unchoke, interested, and not interested have no payload.

Bitfield is only ever sent as the first message. Its payload is a
bitfield with each index that downloader has sent set to one and the
rest set to zero. Downloaders which don't have anything yet may skip the
bitfield message. The first byte of the bitfield corresponds to indices
0-7 from high bit to low bit, respectively. The next one 8-15, etc.
Spare bits at the end are set to zero.

The have message's payload is a single number, the index which that
downloader just completed and checked the hash of.

Request messages contain an index, begin, and length. The last two are
byte offsets. Length is generally a power of two unless it gets
truncated by the end of the file. All current implementations use 2^15 ,
and close connections which request an amount greater than 2^17 .

Cancel messages have the same payload as request messages. They are
generally only sent towards the end of a download, during what's called
'endgame mode'. When a download is almost complete, there's a tendency
for the last few pieces to all be downloaded off a single hosed modem
line, taking a very long time. To make sure the last few pieces come in
quickly, once requests for all pieces a given downloader doesn't have
yet are currently pending, it sends requests for everything to everyone
it's downloading from. To keep this from becoming horribly inefficient,
it sends cancels to everyone else every time a piece arrives.

Piece messages contain an index, begin, and piece. Note that they are
correlated with request messages implicitly. It's possible for an
unexpected piece to arrive if choke and unchoke messages are sent in
quick succession and/or transfer is going very slowly.

Downloaders generally download pieces in random order, which does a
reasonably good job of keeping them from having a strict subset or
superset of the pieces of any of their peers.

Choking is done for several reasons. TCP congestion control behaves very
poorly when sending over many connections at once. Also, choking lets
each peer use a tit-for-tat-ish algorithm to ensure that they get a
consistent download rate.

The choking algorithm described below is the currently deployed one. It
is very important that all new algorithms work well both in a network
consisting entirely of themselves and in a network consisting mostly of
this one.

There are several criteria a good choking algorithm should meet. It
should cap the number of simultaneous uploads for good TCP performance.
It should avoid choking and unchoking quickly, known as fibrillation. It
should reciprocate to peers who let it download. Finally, it should try
out unused connections once in a while to find out if they might be
better than the currently used ones, known as optimistic unchoking.

The currently deployed choking algorithm avoids fibrillation by only
changing who's choked once every ten seconds. It does reciprocation and
number of uploads capping by unchoking the four peers which it has the
best download rates from and are interested. Peers which have a better
upload rate but aren't interested get unchoked and if they become
interested the worst uploader gets choked. If a downloader has a
complete file, it uses its upload rate rather than its download rate to
decide who to unchoke.

For optimistic unchoking, at any one time there is a single peer which
is unchoked regardless of it's upload rate (if interested, it counts as
one of the four allowed downloaders.) Which peer is optimistically
unchoked rotates every 30 seconds. To give them a decent chance of
getting a complete piece to upload, new connections are three times as
likely to start as the current optimistic unchoke as anywhere else in
the rotation.