wget.pod

一个从网络上自动下载文件的自由工具

Beginning with Wget 1.7, if you use B<-c> on a non-empty file, and
it turns out that the server does not support continued downloading,
Wget will refuse to start the download from scratch, which would
effectively ruin existing contents.  If you really want the download to
start from scratch, remove the file.

Also beginning with Wget 1.7, if you use B<-c> on a file which is of
equal size as the one on the server, Wget will refuse to download the
file and print an explanatory message.  The same happens when the file
is smaller on the server than locally (presumably because it was changed
on the server since your last download attempt)---because "continuing"
is not meaningful, no download occurs.

On the other side of the coin, while using B<-c>, any file that's
bigger on the server than locally will be considered an incomplete
download and only C<(length(remote) - length(local))> bytes will be
downloaded and tacked onto the end of the local file.  This behavior can
be desirable in certain cases---for instance, you can use B<wget -c>
to download just the new portion that's been appended to a data
collection or log file.

However, if the file is bigger on the server because it's been
I<changed>, as opposed to just I<appended> to, you'll end up
with a garbled file.  Wget has no way of verifying that the local file
is really a valid prefix of the remote file.  You need to be especially
careful of this when using B<-c> in conjunction with B<-r>,
since every file will be considered as an "incomplete download" candidate.

Another instance where you'll get a garbled file if you try to use
B<-c> is if you have a lame HTTP proxy that inserts a
"transfer interrupted" string into the local file.  In the future a
"rollback" option may be added to deal with this case.

Note that B<-c> only works with FTP servers and with HTTP
servers that support the C<Range> header.


=item B<--progress=>I<type>

Select the type of the progress indicator you wish to use.  Legal
indicators are "dot" and "bar".

The "bar" indicator is used by default.  It draws an ASCII progress
bar graphics (a.k.a "thermometer" display) indicating the status of
retrieval.  If the output is not a TTY, the "dot" bar will be used by
default.

Use B<--progress=dot> to switch to the "dot" display.  It traces
the retrieval by printing dots on the screen, each dot representing a
fixed amount of downloaded data.

When using the dotted retrieval, you may also set the I<style> by
specifying the type as B<dot:>I<style>.  Different styles assign
different meaning to one dot.  With the C<default> style each dot
represents 1K, there are ten dots in a cluster and 50 dots in a line.
The C<binary> style has a more "computer"-like orientation---8K
dots, 16-dots clusters and 48 dots per line (which makes for 384K
lines).  The C<mega> style is suitable for downloading very large
files---each dot represents 64K retrieved, there are eight dots in a
cluster, and 48 dots on each line (so each line contains 3M).

Note that you can set the default style using the C<progress>
command in F<.wgetrc>.  That setting may be overridden from the
command line.  The exception is that, when the output is not a TTY, the
"dot" progress will be favored over "bar".  To force the bar output,
use B<--progress=bar:force>.


=item B<-N>


=item B<--timestamping>

Turn on time-stamping.  


=item B<-S>


=item B<--server-response>

Print the headers sent by HTTP servers and responses sent by
FTP servers.


=item B<--spider>

When invoked with this option, Wget will behave as a Web I<spider>,
which means that it will not download the pages, just check that they
are there.  For example, you can use Wget to check your bookmarks:

	
	wget --spider --force-html -i bookmarks.html


This feature needs much more work for Wget to get close to the
functionality of real web spiders.


=item B<-T seconds>


=item B<--timeout=>I<seconds>

Set the network timeout to I<seconds> seconds.  This is equivalent
to specifying B<--dns-timeout>, B<--connect-timeout>, and
B<--read-timeout>, all at the same time.

When interacting with the network, Wget can check for timeout and
abort the operation if it takes too long.  This prevents anomalies
like hanging reads and infinite connects.  The only timeout enabled by
default is a 900-second read timeout.  Setting a timeout to 0 disables
it altogether.  Unless you know what you are doing, it is best not to
change the default timeout settings.

All timeout-related options accept decimal values, as well as
subsecond values.  For example, B<0.1> seconds is a legal (though
unwise) choice of timeout.  Subsecond timeouts are useful for checking
server response times or for testing network latency.


=item B<--dns-timeout=>I<seconds>

Set the DNS lookup timeout to I<seconds> seconds.  DNS lookups that
don't complete within the specified time will fail.  By default, there
is no timeout on DNS lookups, other than that implemented by system
libraries.


=item B<--connect-timeout=>I<seconds>

Set the connect timeout to I<seconds> seconds.  TCP connections that
take longer to establish will be aborted.  By default, there is no
connect timeout, other than that implemented by system libraries.


=item B<--read-timeout=>I<seconds>

Set the read (and write) timeout to I<seconds> seconds.  The
"time" of this timeout refers to I<idle time>: if, at any point in
the download, no data is received for more than the specified number
of seconds, reading fails and the download is restarted.  This option
does not directly affect the duration of the entire download.

Of course, the remote server may choose to terminate the connection
sooner than this option requires.  The default read timeout is 900
seconds.


=item B<--limit-rate=>I<amount>

Limit the download speed to I<amount> bytes per second.  Amount may
be expressed in bytes, kilobytes with the B<k> suffix, or megabytes
with the B<m> suffix.  For example, B<--limit-rate=20k> will
limit the retrieval rate to 20KB/s.  This is useful when, for whatever
reason, you don't want Wget to consume the entire available bandwidth.

This option allows the use of decimal numbers, usually in conjunction
with power suffixes; for example, B<--limit-rate=2.5k> is a legal
value.

Note that Wget implements the limiting by sleeping the appropriate
amount of time after a network read that took less time than specified
by the rate.  Eventually this strategy causes the TCP transfer to slow
down to approximately the specified rate.  However, it may take some
time for this balance to be achieved, so don't be surprised if limiting
the rate doesn't work well with very small files.


=item B<-w> I<seconds>


=item B<--wait=>I<seconds>

Wait the specified number of seconds between the retrievals.  Use of
this option is recommended, as it lightens the server load by making the
requests less frequent.  Instead of in seconds, the time can be
specified in minutes using the C<m> suffix, in hours using C<h>
suffix, or in days using C<d> suffix.

Specifying a large value for this option is useful if the network or the
destination host is down, so that Wget can wait long enough to
reasonably expect the network error to be fixed before the retry.  The
waiting interval specified by this function is influenced by
C<--random-wait>, which see.


=item B<--waitretry=>I<seconds>

If you don't want Wget to wait between I<every> retrieval, but only
between retries of failed downloads, you can use this option.  Wget will
use I<linear backoff>, waiting 1 second after the first failure on a
given file, then waiting 2 seconds after the second failure on that
file, up to the maximum number of I<seconds> you specify.  Therefore,
a value of 10 will actually make Wget wait up to (1 + 2 + ... + 10) = 55
seconds per file.

Note that this option is turned on by default in the global
F<wgetrc> file.


=item B<--random-wait>

Some web sites may perform log analysis to identify retrieval programs
such as Wget by looking for statistically significant similarities in
the time between requests. This option causes the time between requests
to vary between 0.5 and 1.5 * I<wait> seconds, where I<wait> was
specified using the B<--wait> option, in order to mask Wget's
presence from such analysis.

A 2001 article in a publication devoted to development on a popular
consumer platform provided code to perform this analysis on the fly.
Its author suggested blocking at the class C address level to ensure
automated retrieval programs were blocked despite changing DHCP-supplied
addresses.

The B<--random-wait> option was inspired by this ill-advised
recommendation to block many unrelated users from a web site due to the
actions of one.


=item B<--no-proxy>

Don't use proxies, even if the appropriate C<*_proxy> environment
variable is defined.



=item B<-Q> I<quota>


=item B<--quota=>I<quota>

Specify download quota for automatic retrievals.  The value can be
specified in bytes (default), kilobytes (with B<k> suffix), or
megabytes (with B<m> suffix).

Note that quota will never affect downloading a single file.  So if you
specify B<wget -Q10k ftp://wuarchive.wustl.edu/ls-lR.gz>, all of the
F<ls-lR.gz> will be downloaded.  The same goes even when several
URLs are specified on the command-line.  However, quota is
respected when retrieving either recursively, or from an input file.
Thus you may safely type B<wget -Q2m -i sites>---download will be
aborted when the quota is exceeded.

Setting quota to 0 or to B<inf> unlimits the download quota.


=item B<--no-dns-cache>

Turn off caching of DNS lookups.  Normally, Wget remembers the IP
addresses it looked up from DNS so it doesn't have to repeatedly
contact the DNS server for the same (typically small) set of hosts it
retrieves from.  This cache exists in memory only; a new Wget run will
contact DNS again.

However, it has been reported that in some situations it is not
desirable to cache host names, even for the duration of a
short-running application like Wget.  With this option Wget issues a
new DNS lookup (more precisely, a new call to C<gethostbyname> or
C<getaddrinfo>) each time it makes a new connection.  Please note
that this option will I<not> affect caching that might be
performed by the resolving library or by an external caching layer,
such as NSCD.

If you don't understand exactly what this option does, you probably
won't need it.


=item B<--restrict-file-names=>I<mode>

Change which characters found in remote URLs may show up in local file
names generated from those URLs.  Characters that are I<restricted>
by this option are escaped, i.e. replaced with B<%HH>, where
B<HH> is the hexadecimal number that corresponds to the restricted
character.

By default, Wget escapes the characters that are not valid as part of
file names on your operating system, as well as control characters that
are typically unprintable.  This option is useful for changing these
defaults, either because you are downloading to a non-native partition,
or because you want to disable escaping of the control characters.

When mode is set to "unix", Wget escapes the character B</> and
the control characters in the ranges 0--31 and 128--159.  This is the
default on Unix-like OS'es.

When mode is set to "windows", Wget escapes the characters B<\>,
B<|>, B</>, B<:>, B<?>, B<">, B<*>, B<E<lt>>,
B<E<gt>>, and the control characters in the ranges 0--31 and 128--159.
In addition to this, Wget in Windows mode uses B<+> instead of
B<:> to separate host and port in local file names, and uses
B<@> instead of B<?> to separate the query portion of the file
name from the rest.  Therefore, a URL that would be saved as
B<www.xemacs.org:4300/search.pl?input=blah> in Unix mode would be
saved as B<www.xemacs.org+4300/search.pl@input=blah> in Windows
mode.  This mode is the default on Windows.

If you append B<,nocontrol> to the mode, as in
B<unix,nocontrol>, escaping of the control characters is also
switched off.  You can use B<--restrict-file-names=nocontrol> to
turn off escaping of control characters without affecting the choice of
the OS to use as file name restriction mode.


=item B<-4>


=item B<--inet4-only>


=item B<-6>


=item B<--inet6-only>

Force connecting to IPv4 or IPv6 addresses.  With B<--inet4-only>
or B<-4>, Wget will only connect to IPv4 hosts, ignoring AAAA
records in DNS, and refusing to connect to IPv6 addresses specified in
URLs.  Conversely, with B<--inet6-only> or B<-6>, Wget will
only connect to IPv6 hosts and ignore A records and IPv4 addresses.

Neither options should be needed normally.  By default, an IPv6-aware
Wget will use the address family specified by the host's DNS record.
If the DNS responds with both IPv4 and IPv6 addresses, Wget will try
them in sequence until it finds one it can connect to.  (Also see
C<--prefer-family> option described below.)

These options can be used to deliberately force the use of IPv4 or
IPv6 address families on dual family systems, usually to aid debugging
or to deal with broken network configuration.  Only one of
B<--inet6-only> and B<--inet4-only> may be specified at the
same time.  Neither option is available in Wget compiled without IPv6
support.


=item B<--prefer-family=IPv4/IPv6/none>

When given a choice of several addresses, connect to the addresses
with specified address family first.  IPv4 addresses are preferred by
default.

This avoids spurious errors and connect attempts when accessing hosts
that resolve to both IPv6 and IPv4 addresses from IPv4 networks.  For
example, B<www.kame.net> resolves to
B<2001:200:0:8002:203:47ff:fea5:3085> and to
B<203.178.141.194>.  When the preferred family is C<IPv4>, the
IPv4 address is used first; when the preferred family is C<IPv6>,
the IPv6 address is used first; if the specified value is C<none>,
the address order returned by DNS is used without change.

Unlike B<-4> and B<-6>, this option doesn't inhibit access to
any address family, it only changes the I<order> in which the
addresses are accessed.  Also note that the reordering performed by
this option is I<stable>---it doesn't affect order of addresses of
the same family.  That is, the relative order of all IPv4 addresses
and of all IPv6 addresses remains intact in all cases.


=item B<--retry-connrefused>

Consider "connection refused" a transient error and try again.
Normally Wget gives up on a URL when it is unable to connect to the
site because failure to connect is taken as a sign that the server is
not running at all and that retries would not help.  This option is
for mirroring unreliable sites whose servers tend to disappear for
short periods of time.


=item B<--user=>I<user>


=item B<--password=>I<password>

Specify the username I<user> and password I<password> for both
FTP and HTTP file retrieval.  These parameters can be overridden
using the B<--ftp-user> and B<--ftp-password> options for 
FTP connections and the B<--http-user> and B<--http-password> 
options for HTTP connections.

=back



=head2 Directory Options