readme

一个WEB服务器的性能测试工具

    Connection length [replies/conn]: 1.000

** "Request" Results

The first line in the "Request"-related results give the rate at which
HTTP requests were issued and the period-length that the rate
corresponds to.  In the example below, the request rate was 10.1
requests per second, which corresponds to 99.1 milliseconds per
request.

    Request rate: 10.1 req/s (99.1 ms/req)

As long as no persistent connections are employed, the "Request"
results are typically very similar or identical to the "Connection"
results.  However, when persistent connections are used, several
requests can be issued on a single connection in which case the
results would be different.

The next line gives the average size of the HTTP request in bytes.  In
the line show below, the average request size was 57 bytes.

    Request size [B]: 57.0


** "Reply" Results

For simple measurements, the section with the "Reply" results is
probably the most interesting one.  The first line gives statistics on
the reply rate:

    Reply rate [replies/s]: min 10.0 avg 10.0 max 10.0 stddev 0.0 (1 samples)

The line above indicates that the minimum ("min"), average ("avg"),
and maximum ("max") reply rate was ten replies per second.  Given
these numbers, the standard deviation is, of course, zero.  The last
number shows that only one reply rate sample was acquired.  The
present version of httperf collects one rate sample about once every
five seconds.  To obtain a meaningful standard deviation, it is
recommended to run each test long enough so at least thirty samples
are obtained---this would correspond to a test duration of at least
150 seconds, or two and a half minutes.

The next line gives information on how long it took for the server to
respond and how long it took to receive the reply.  The line below
shows that it took 4.1 milliseconds between sending the first byte of
the request and receiving the first byte of the reply.  The time to
"transfer", or read, the reply was too short to be measured, so it
shows up as zero (as we'll see below, the entire reply fit into a
single TCP segment and that's why the transfer time was measured as
zero).

    Reply time [ms]: response 4.1 transfer 0.0

Next follow some statistics on the size of the reply---all numbers are
reported in bytes.  Specifically, the average length of reply headers,
the average length of the content, and the average length of reply
footers are given (HTTP/1.1 uses footers to realize the "chunked"
transfer encoding).  For convenience, the average total number of
bytes in the replies is also given.  In the example below, the average
header length ("header") was 219 bytes, the average content length
("content") was 204 bytes, and there were no footers ("footer"),
yielding a total reply length of 423 bytes on average.

    Reply size [B]: header 219.0 content 204.0 footer 0.0 (total 423.0)

The final piece in this section is a histogram on the status codes
received in the replies.  The example below shows that all 100 replies
were "successful" replies as they contained a status code of 200
(presumably):

    Reply status: 1xx=0 2xx=100 3xx=0 4xx=0 5xx=0


** Miscellaneous Results

This section starts with a summary of the CPU time the client
consumed.  The line below shows that 2.71 seconds were spent executing
in user mode ("user"), 7.08 seconds were spent executing in system
mode ("system") and that this corresponds to 27.4% user mode execution
and 71.5% system execution.  The total utilization was almost exactly
100%, which is expected given that httperf is a CPU hog:

    CPU time [s]: user 2.71 system 7.08 (user 27.4% system 71.5% total 98.8%)

Note that any time the total CPU utilization is significantly less
than 100%, some other processes must have been running on the client
machine while httperf was executing.  This makes it likely that the
results are "polluted" and the test should be rerun.

The next line gives the average network throughput in kilobytes per
second (where a kilobyte is 1024 bytes) and in megabits per second
(where a megabit is 10^6 bit).  The line below shows an average
network bandwidth of about 4.7 kilobyte per second.  The megabit per
second number is zero due to rounding errors.

    Net I/O: 4.7 KB/s (0.0*10^6 bps)

The network bandwidth is computed from the number of bytes sent and
received on TCP connections.  This means that it accounts for the
network payload only (i.e., it doesn't account for protocol headers)
and does not take into account retransmissions that may occur at the
TCP level.

** "Errors"

The final section contains statistics on the errors that occurred
during the test.  The "total" figure shows the total number of errors
that occurred.  The two lines below show that in our example run there
were no errors:

    Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
    Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

The meaning of each error is described below:

 total:
	The sum of all following error counts.

 client-timo:
	Each time a request is made to the server, a watchdog timer
	is started.  If no (partial) response is received by the time
	the watchdog timer expires, httperf times out that request
	a increments this error counter.  This is the most common error
	when driving a server into overload.

 socket-timo
	The number of times a TCP connection failed with a
	socket-level time out (ETIMEDOUT).

 connrefused
	The number of times a TCP connection attempt failed with
	a "connection refused by server" error (ECONNREFUSED).

 connreset
	The number of times a TCP connection failed due to a reset
	(close) by the server.

 fd-unavail
	The number of times the httperf client was out of file
	descriptors.  Whenever this count is bigger than
	zero, the test results are meaning less because the client
	was overloaded (see discussion on setting --timeout below).

 addrunavail
	The number of times the client was out of TCP port numbers
	(EADDRNOTAVAIL).  This error should never occur.  If it
	does, the results should be discarded.

 ftab-full
	The number of times the system's file descriptor table
	was full.  Again, this error should never occur.  If it
	does, the results should be discarded.

 other
	The number of times other errors occurred.  Whenever this
	occurs, it is necessary to track down the actual error
	reason.  This can be done by compiling httperf with
	debug support and specifying option --debug 1.


** Selecting appropriate timeout values

Since the client machine has only a limited set of resource available,
it cannot sustain arbitrarily high HTTP request rates.  One limit is
that there are only roughly 60,000 TCP port numbers that can be in use
at any given time.  Since, on HP-UX, it takes one minute for a TCP
connection to be fully closed (leave the TIME_WAIT state), the maximum
rate a client can sustain is about 1,000 requests per second.

The actual sustainable rate is typically lower than this because
before running out of TCP ports, a client is likely to run out of file
descriptors (one file descriptor is required per open TCP connection).
By default, HP-UX 10.20 allows 1024 file descriptors per process.
Without a watchdog timer, httperf could potentially quickly use up all
available file descriptors, at which point it could not induce any new
load on the server (this would primarily happen when the server is
overloaded).  To avoid this problem, httperf requires that the web
server must respond within the time specified by option --timeout.  If
it does not respond within that time, the client considers the
connection to be "dead" and closes it (and increases the "client-timo"
error count).  The only exception to this rule is that after sending a
request, httperf allows the server to take some additional time before
it starts responding (to accommodate HTTP requests that take a long
time to complete on the server).  This additional time is called the
"server think time" and can be specified by option --think-timeout.
By default, this additional think time is zero, so by default the
server has to be able to respond within the time allowed by the
--timeout option.

In practice, we found that with a --timeout value of 1 second, an HP
9000/735 machine running HP-UX 10.20 can sustain a rate of about 700
connections per second before it starts to run out of file descriptor
(the exact rate depends, of course, on a number of factors).  To
achieve web server loads bigger than that, it is necessary to employ
several independent machines, each running one copy of httperf.  A
timeout of one second effectively means that "slow" connections will
typically timeout before TCP even gets a chance to retransmit (the
initial retransmission timeout is on the order of 3 seconds).  This is
usually OK, except that one should keep in mind that it has the effect
of truncating the connection life time distribution.