curl.1

harvest是一个下载html网页得机器人

.B "curl --interface eth0:1 http://www.netscape.com/"

If this option is used several times, the last one will be used.
.IP "-I/--head"
(HTTP/FTP/FILE)
Fetch the HTTP-header only! HTTP-servers feature the command HEAD
which this uses to get nothing but the header of a document. When used
on a FTP or FILE file, curl displays the file size and last modification
time only.

If this option is used twice, the second will again disable header only.
.IP "-j/--junk-session-cookies"
(HTTP) When curl is told to read cookies from a given file, this option will
make it discard all "session cookies". This will basicly have the same effect
as if a new session is started. Typical browsers always discard session
cookies when they're closed down. (Added in 7.9.7)

If this option is used several times, each occurrence will toggle this on/off.
.IP "-k/--insecure"
(SSL) This option explicitly allows curl to perform "insecure" SSL connections
and transfers. Starting with curl 7.10, all SSL connections will be attempted
to be made secure by using the CA certificate bundle installed by
default. This makes all connections considered "insecure" to fail unless
-k/--insecure is used.

This option is ignored if --cacert or --capath is used!

If this option is used twice, the second time will again disable it.
.IP "--krb4 <level>"
(FTP) Enable kerberos4 authentication and use. The level must be entered and
should be one of 'clear', 'safe', 'confidential' or 'private'. Should you use
a level that is not one of these, 'private' will instead be used.

If this option is used several times, the last one will be used.
.IP "-K/--config <config file>"
Specify which config file to read curl arguments from. The config file is a
text file in which command line arguments can be written which then will be
used as if they were written on the actual command line. Options and their
parameters must be specified on the same config file line. If the parameter is
to contain white spaces, the parameter must be inclosed within quotes.  If the
first column of a config line is a '#' character, the rest of the line will be
treated as a comment.

Specify the filename as '-' to make curl read the file from stdin.

Note that to be able to specify a URL in the config file, you need to specify
it using the --url option, and not by simply writing the URL on its own
line. So, it could look similar to this:

url = "http://curl.haxx.se/docs/"

This option can be used multiple times.
.IP "--limit-rate <speed>"
Specify the maximum transfer rate you want curl to use. This feature is useful
if you have a limited pipe and you'd like your transfer not use your entire
bandwidth.

The given speed is measured in bytes/second, unless a suffix is
appended. Appending 'k' or 'K' will count the number as kilobytes, 'm' or M'
makes it megabytes while 'g' or 'G' makes it gigabytes. Examples: 200K, 3m and
1G.

This option was introduced in curl 7.10.

If this option is used several times, the last one will be used.
.IP "-l/--list-only"
(FTP)
When listing an FTP directory, this switch forces a name-only view.
Especially useful if you want to machine-parse the contents of an FTP
directory since the normal directory view doesn't use a standard look
or format.

This option causes an FTP NLST command to be sent.  Some FTP servers
list only files in their response to NLST; they do not include
subdirectories and symbolic links.

If this option is used twice, the second will again disable list only.
.IP "-L/--location"
(HTTP/HTTPS) If the server reports that the requested page has a different
location (indicated with the header line Location:) this flag will let curl
attempt to reattempt the get on the new place. If used together with -i or -I,
headers from all requested pages will be shown. If authentication is used,
curl will only send its credentials to the initial host, so if a redirect
takes curl to a different host, it won't intercept the user+password. See also
\fI--location-trusted\fP on how to change this.

If this option is used twice, the second will again disable location following.
.IP "--location-trusted"
(HTTP/HTTPS) Like \fI--location\fP, but will allow sending the name + password
to all hosts that the site may redirect to. This may or may not introduce a
security breach if the site redirects you do a site to which you'll send your
authentication info (which is plaintext in the case of HTTP Basic
authentication).

If this option is used twice, the second will again disable location following.
.IP "--max-filesize <bytes>"
Specify the maximum size (in bytes) of a file to download. If the file
requested is larger than this value, the transfer will not start and curl will
return with exit code 63.

NOTE: The file size is not always known prior to download, and for such files
this option has no effect even if the file transfer ends up being larger than
this given limit. This concerns both FTP and HTTP transfers.
.IP "-m/--max-time <seconds>"
Maximum time in seconds that you allow the whole operation to take.  This is
useful for preventing your batch jobs from hanging for hours due to slow
networks or links going down.  This doesn't work fully in win32 systems.  See
also the \fI--connect-timeout\fP option.

If this option is used several times, the last one will be used.
.IP "-M/--manual"
Manual. Display the huge help text.
.IP "-n/--netrc"
Makes curl scan the
.I .netrc
file in the user's home directory for login name and password. This is
typically used for ftp on unix. If used with http, curl will enable user
authentication. See
.BR netrc(4)
or
.BR ftp(1)
for details on the file format. Curl will not complain if that file
hasn't the right permissions (it should not be world nor group
readable). The environment variable "HOME" is used to find the home
directory.

A quick and very simple example of how to setup a
.I .netrc
to allow curl to ftp to the machine host.domain.com with user name
\&'myself' and password 'secret' should look similar to:

.B "machine host.domain.com login myself password secret"

If this option is used twice, the second will again disable netrc usage.
.IP "--negotiate"
(HTTP) Enables GSS-Negotiate authentication. The GSS-Negotiate method was
designed by Microsoft and is used in their web aplications. It is primarily
meant as a support for Kerberos5 authentication but may be also used along
with another authentication methods. For more information see IETF draft
draft-brezak-spnego-http-04.txt. (Added in 7.10.6)

\fBNOTE\fP that this option requiures that the library was built with GSSAPI
support. This is not very common. Use \fIcurl --version\fP to see if your
version supports GSS-Negotiate.

If this option is used several times, the following occurrences make no
difference.
.IP "-N/--no-buffer"
Disables the buffering of the output stream. In normal work situations, curl
will use a standard buffered output stream that will have the effect that it
will output the data in chunks, not necessarily exactly when the data arrives.
Using this option will disable that buffering.

If this option is used twice, the second will again switch on buffering.
.IP "--ntlm"
(HTTP) Enables NTLM authentication. The NTLM authentication method was
designed by Microsoft and is used by IIS web servers. It is a proprietary
protocol, reversed engineered by clever people and implemented in curl based
on their efforts. This kind of behavior should not be endorsed, you should
encourage everyone who uses NTLM to switch to a public and documented
authentication method instead. Such as Digest. (Added in 7.10.6)

\fBNOTE\fP that this option requiures that the library was built with SSL
support. Use \fIcurl --version\fP to see if your version supports NTLM.

If this option is used several times, the following occurrences make no
difference.
.IP "-o/--output <file>"
Write output to <file> instead of stdout. If you are using {} or [] to fetch
multiple documents, you can use '#' followed by a number in the <file>
specifier. That variable will be replaced with the current string for the URL
being fetched. Like in:

  curl http://{one,two}.site.com -o "file_#1.txt"

or use several variables like:

  curl http://{site,host}.host[1-5].com -o "#1_#2"

You may use this option as many times as you have number of URLs.

See also the --create-dirs option to create the local directories dynamically.
.IP "-O/--remote-name"
Write output to a local file named like the remote file we get. (Only the file
part of the remote file is used, the path is cut off.)

You may use this option as many times as you have number of URLs.
.IP "-p/--proxytunnel"
When an HTTP proxy is used, this option will cause non-HTTP protocols to
attempt to tunnel through the proxy instead of merely using it to do HTTP-like
operations. The tunnel approach is made with the HTTP proxy CONNECT request
and requires that the proxy allows direct connect to the remote port number
curl wants to tunnel through to.

If this option is used twice, the second will again disable proxy tunnel.
.IP "-P/--ftpport <address>"
(FTP)
Reverses the initiator/listener roles when connecting with ftp. This
switch makes Curl use the PORT command instead of PASV. In
practice, PORT tells the server to connect to the client's specified
address and port, while PASV asks the server for an ip address and
port to connect to. <address> should be one of:
.RS
.TP 12
.B interface
i.e "eth0" to specify which interface's IP address you want to use  (Unix only)
.TP
.B "IP address"
i.e "192.168.10.1" to specify exact IP number
.TP
.B "host name"
i.e "my.host.domain" to specify machine
.TP
.B "-"
(any single-letter string) to make it pick the machine's default
.RE

If this option is used several times, the last one will be used.
.IP "-q"
If used as the first parameter on the command line, the
.I $HOME/.curlrc
file will not be read and used as a config file.
.IP "-Q/--quote <comand>"
(FTP) Send an arbitrary command to the remote FTP server, by using the QUOTE
command of the server. Not all servers support this command, and the set of
QUOTE commands are server specific! Quote commands are sent BEFORE the
transfer is taking place. To make commands take place after a successful
transfer, prefix them with a dash '-'. You may specify any amount of commands
to be run before and after the transfer. If the server returns failure for one
of the commands, the entire operation will be aborted.

This option can be used multiple times.
.IP "--random-file <file>"
(HTTPS) Specify the path name to file containing what will be considered as
random data. The data is used to seed the random engine for SSL connections.
See also the
.I "--edg-file"
option.
.IP "-r/--range <range>"
(HTTP/FTP)
Retrieve a byte range (i.e a partial document) from a HTTP/1.1 or FTP
server. Ranges can be specified in a number of ways.
.RS
.TP 10
.B 0-499
specifies the first 500 bytes
.TP
.B 500-999
specifies the second 500 bytes
.TP
.B -500
specifies the last 500 bytes
.TP
.B 9500
specifies the bytes from offset 9500 and forward
.TP
.B 0-0,-1
specifies the first and last byte only(*)(H)
.TP
.B 500-700,600-799
specifies 300 bytes from offset 500(H)
.TP
.B 100-199,500-599
specifies two separate 100 bytes ranges(*)(H)
.RE

(*) = NOTE that this will cause the server to reply with a multipart
response!

You should also be aware that many HTTP/1.1 servers do not have this feature
enabled, so that when you attempt to get a range, you'll instead get the whole
document.

FTP range downloads only support the simple syntax 'start-stop' (optionally
with one of the numbers omitted). It depends on the non-RFC command SIZE.

If this option is used several times, the last one will be used.
.IP "-R/--remote-time"
When used, this will make libcurl attempt to figure out the timestamp of the
remote file, and if that is available make the local file get that same
timestamp.

If this option is used twice, the second time disables this again.
.IP "-s/--silent"
Silent mode. Don't show progress meter or error messages.  Makes
Curl mute.

If this option is used twice, the second will again disable mute.
.IP "-S/--show-error"
When used with -s it makes curl show error message if it fails.

If this option is used twice, the second will again disable show error.
.IP "--stderr <file>"
Redirect all writes to stderr to the specified file instead. If the file name
is a plain '-', it is instead written to stdout. This option has no point when
you're using a shell with decent redirecting capabilities.

If this option is used several times, the last one will be used.
.IP "-t/--telnet-option <OPT=val>"
Pass options to the telnet protocol. Supported options are:

TTYPE=<term> Sets the terminal type.

XDISPLOC=<X display> Sets the X display location.

NEW_ENV=<var,val> Sets an environment variable.
.IP "-T/--upload-file <file>"
This transfers the specified local file to the remote URL. If there is no file
part in the specified URL, Curl will append the local file name. NOTE that you
must use a trailing / on the last directory to really prove to Curl that there
is no file name or curl will think that your last directory name is the remote
file name to use. That will most likely cause the upload operation to fail. If
this is used on a http(s) server, the PUT command will be used.

Use the file name "-" (a single dash) to use stdin instead of a given file.

Before 7.10.8, when this option was used several times, the last one was used.

In curl 7.10.8 and later, you can specify one -T for each URL on the command
line. Each -T + URL pair specifies what to upload and to where. curl also
supports "globbing" of the -T argument, meaning that you can upload multiple
files to a single URL by using the same URL globbing style supported in the
URL, like this:

curl -T "{file1,file2}" http://www.uploadtothissite.com

or even

curl -T "img[1-1000].png" ftp://ftp.picturemania.com/upload/
.IP "--trace <file>"
Enables a full trace dump of all incoming and outgoing data, including
descriptive information, to the given output file. Use "-" as filename to have
the output sent to stdout.

If this option is used several times, the last one will be used. (Added in
7.9.7)
.IP "--trace-ascii <file>"
Enables a full trace dump of all incoming and outgoing data, including
descriptive information, to the given output file. Use "-" as filename to have
the output sent to stdout.

This is very similar to --trace, but leaves out the hex part and only shows
the ASCII part of the dump. It makes smaller output that might be easier to
read for untrained humans.

If this option is used several times, the last one will be used. (Added in
7.9.7)
.IP "-u/--user <user:password>"
Specify user and password to use when fetching. Read the MANUAL for detailed
examples of how to use this. If no password is specified, curl will ask for it
interactively.

You can also use the --digest option to enable Digest authentication when
communicating with HTTP 1.1 servers.

If this option is used several times, the last one will be used.
.IP "-U/--proxy-user <user:password>"
Specify user and password to use for Proxy authentication. If no
password is specified, curl will ask for it interactively.

If this option is used several times, the last one will be used.
.IP "--url <URL>"
Specify a URL to fetch. This option is mostly handy when you want to specify