changelog

Wget很好的处理了http和ftp的下载,很值得学习的经典代码

2008-03-19  Micah Cowan  <[e-mail]>

	* utils.c (test_dir_matches_p): More tests related for
	dir_matches_p.

2008-03-17  Micah Cowan  <[e-mail]>

	* connect.c: Include sys/time.h to support use of the select
	function on older systems.

2008-02-11  Benno Schulenberg  <[e-mail]>

	* http.c: More accurate and descriptive messages for when a file
	won't be retrieved during spider-mode.

2008-02-10  Micah Cowan  <[e-mail]>

	* http.c: Added existence_checked member to the http_stat
	struct.
	(gethttp): Mark hs->existence_checked when we've checked whether
	a file-to-download exists; so we don't check it again if the
	connection gets lost (and potentially pick a new "unique" name).
	This fixes bug 22251.
	* progress.c (create_image): Add space for an extra column in
	the "eta" portion of the progress bar image; to deal with
	too-long Czech translation.
	* main.c, http.c, init.c: Added --auth-no-challenge option, to
	bring back 1.10.2 unsafe auth behavior when needed. This fixes
	bug #22242.

2007-02-10  Hrvoje Niksic  <[e-mail]>

	* ftp.c (ftp_retrieve_glob): Print both arguments of fnmatch in
	fnmatch error message.
	(ftp_retrieve_glob): Don't match with fnmatch if we're only
	supposed to get one file.

2008-02-07  Micah Cowan  <[e-mail]>

	* progress.c (create_image): Remove assertion on exceeding
	screen width, which given the less-than-robust code there, can
	be broken by a number of factors (such as large file downloads).

2008-02-06  Micah Cowan  <[e-mail]>

	* progress.c (countcols): Use strlen() when mbtowc or wcwidth
	not available (or not using NLS).
	* utils.c: Ensure we use single-byte separators when not doing
	NLS progress-bars.
	* wget.h: Determine whether to use NLS for progress-bars, based
	on whether wcwidth and mbtowc are available.

2008-02-03  Micah Cowan  <[e-mail]>

	* progress.c (create_image): Use number of characters/columns
	consumed, rather than number of bytes, to determine how much of
	a line we've used. Fixes assertion errors and field alignment
	bugs (#22161, #20481)
	(get_eta, count_cols): Added to support the changes for
	create_image.
	* http.c (http_loop): Put no-clobber logic back into http_loop,
	before starting to fetch, for when we're not doing
	content-disposition.

2008-01-31  Micah Cowan  <[e-mail]>

	* http.c (gethttp): Don't derive hs->contlen from possibly
	invalid/missing Content-Length; instead, get the appropriate
	value from the Content-Range header values.
	(parse_content_range): Handle '*' instance-length field.

2008-01-30  Micah Cowan  <[e-mail]>

	* version.c: Bumped version number to 1.11.1-devel.

2008-01-25  Micah Cowan  <[e-mail]>

	* Makefile.in, cmpt.c, connect.c, connect.h, convert.c,
	cookies.c, cookies.h, ftp-basic.c, ftp-ls.c, ftp-opie.c, ftp.c,
	ftp.h, gen-md5.c, gen-md5.h, gnu-md5.c, gnu-md5.h, gnutls.c,
	hash.c, hash.h, host.c, host.h, html-parse.c, html-parse.h,
	html-url.c, http-ntlm.c, http-ntlm.h, http.c, http.h, init.c,
	init.h, log.c, log.h, main.c, mswindows.c, mswindows.h, netrc.c,
	netrc.h, openssl.c, options.h, progress.c, progress.h, ptimer.c,
	ptimer.h, recur.c, recur.h, res.c, res.h, retr.c, retr.h,
	spider.c, spider.h, ssl.h, sysdep.h, test.c, test.h, url.c,
	url.h, utils.c, utils.h, wget.h, xmalloc.c, xmalloc.h: Updated
	copyright year.
	* version.c: Bumped version number to 1.11.

2007-12-10  Micah Cowan  <[e-mail]>

	* main.c: The option is --content-disposition, not
	--no-content-disposition (at the moment).

2007-12-05  Micah Cowan  <[e-mail]>

	* utils.c (subdir_p): Handle the case where d1 is "".
	* convert.c (convert_all_links): Don't return without
	deallocating timer.

2007-11-28  Micah Cowan  <[e-mail]>

	* Makefile.in, cmpt.c, connect.c, connect.h, convert.c,
	convert.h, cookies.c, cookies.h, ftp-basic.c, ftp-ls.c,
	ftp-opie.c, ftp.c, ftp.h, gen-md5.c, gen-md5.h, gnutls.c,
	hash.c, hash.h, host.c, host.h, html-parse.c, html-parse.h,
	html-url.c, http-ntlm.c, http-ntlm.h, http.c, http.h, init.c,
	init.h, log.c, log.h, main.c, mswindows.c, mswindows.h,
	netrc.c, netrc.h, openssl.c, options.h, progress.c, progress.h,
	ptimer.c, ptimer.h, recur.c, recur.h, res.c, res.h, retr.c,
	retr.h, safe-ctype.c, safe-ctype.h, spider.c, spider.h, ssl.h,
	sysdep.h, test.c, test.h, url.c, url.h, utils.c, utils.h,
	wget.h, xmalloc.c, xmalloc.h: Updated license exception for
	OpenSSL, per the SFLC.

2007-10-10  Micah Cowan  <[e-mail]>

	* http-ntlm.c: Include openssl/opensslv.h explicitly, instead of
	hoping it'll be included by accident in openssl/des.h.

2007-10-09  Gisle Vanem  <[e-mail]>

	* mswindows.c: 'argc' and 'argv' in 'windows_main()' are no longer
	needed.  Hence simply the prototype. Free 'exec_name' at exit.

2007-10-08  Micah Cowan  <[e-mail]>

	* http.c (http_loop): Add send_head_first conditional back
	around code that needs it, but not around the last-modified
	header-parsing stuff this time. Removed no-longer-useful (was it
	ever?) restart_loop boolean, continuing unconditionally at end
	of send_head_first conditional block (if we haven't jumped out).

2007-10-04  Micah Cowan  <[e-mail]>

	* http.c (http_loop): We've got_name if content_disposition
	support isn't on; make sure we continue properly in that case,
	even though we're not sending HEAD.

2007-10-02  Gisle Vanem  <[e-mail]>
	
	* ftp.c: Use "_listing" for MSDOS (".listing" is illegal).

	* url.c: Update comment for 'filechr_not_windows'.

	* utils.c: Include <process.h> for 'getpid()' on Watcom.

2007-10-02  Micah Cowan  <[e-mail]>

	* ftp.c (getftp, ftp_loop_internal), http.c (http_loop), main
	(main): Use datetime_str instead of time_str, for those who have
	potentially long-running sessions. Based on suggestions by Saso
	Tomat <[e-mail]> and Steven M. Schweda
	<[e-mail]>.
	* http.c (gethttp): Warn about host lookup failures. Adjusted
	from Stephen Gildea's patch.

2007-10-02  Stephen Gildea  <[e-mail]>

	* connect.c (connect_to_host): Warn about host lookup failures.

2007-09-25  Micah Cowan  <[e-mail]>

	* Makefile.in: Use EXEEXT instead of exeext.

2007-09-24  Gisle Vanem  <[e-mail]>

	* connect.c, init.c, main.c, openssl.c, options.h, sysdep.h,
	url.c, utils.c: Added support for building on MS-DOS.

2007-09-24  Jochen Roderburg  <[e-mail]>

	* http.c (http_zero): Remove no-longer-used local_size variable.
	Fixes bug #21057.

2007-09-12  Micah Cowan  <[e-mail]>

	* http.c (http_loop): Remove send_head_first from condition for
	parsing timestamp.

2007-08-29  Micah Cowan  <[e-mail]>

	* openssl.c (ssl_init): Re un-const-ified the meth local
	variable, to match current versions of openssl.
	* spider.c: Removed visited_url function, as it may be very
	inefficient.
	(print_broken_links): Removed traversal of referrers, until such
	time as a more efficient implementation can be written.
	* spider.h: Replaced declaration of visited_url with an
	empty-bodied, function-like macro.

2007-08-27  Gisle Vanem	 <[e-mail]>

	* mswindows.c (run_with_timeout): Ensure that the correct
	conversion specification is used for the return result of
	the GetLastError function.
	* getopt.c: Fix missing (but, accidentally, legal) comment
	delimiter after licensing text.
	* recur.c (retrieve_tree): Inserted missing cast for strip_auth.
	Includes adjustment by Ralf Wildenhues.
	* openssl.c (ssl_init): const-ified the meth local variable.
	* main.c: Include all the static function definitions in the
	"#ifndef TESTING" clause, leaving just the definitions for
	exec_name (not set), and opt.
	* utils.c (run_with_timeout): Now returns bool, to align with
	declaration in utils.h.

2007-08-27  Micah Cowan  <[e-mail]>

	* wget.h: Added macro replacement for ngettext, for environs
	that lack NLS.

2007-08-26  Micah Cowan  <[e-mail]>

	* spider.c (print_broken_links): Fixed incorrect plurals msgid
	usage, switched to use ngettext function.

2007-08-24  Micah Cowan  <[e-mail]>

	* http.c (http_loop): Introduced time_came_from_head boolean
	flag, to help avoid parsing the same Last-Modified header twice.
	Replaced spidering returns of RETRUNNEEDED for some situations,
	to RETROK, as otherwise it will be interpreted as an error.
	RETRUNNEEDED appears never to be referenced outside of
	http.c (and wget.h), and, when returned by gethttp, is
	translated by http_loop to RETROK.
	* url.c (are_urls_equal): Don't call getchar_from_escaped_string
	if u2 is shorter than u1.
	(getchar_from_escaped_string): Don't decode reserved characters.
	Handle illegally appearing '%'s as literal '%'s. Ensure hex
	digits before attempting to decode.
	(test_are_urls_equal): Added tests to handle u2 shorter than u1,
	and %2f not treated the same as /.
	* spider.c (in_url_list_p): Don't call are_urls_equal if one of
	them is NULL.

2007-08-23  Joshua David Williams  <[e-mail]>

	* spider.c (in_url_list_p): Removed the bool verbose argument

2007-08-22  Mauro Tortonesi  <[e-mail]>

	* http.c (http_loop): Fall back to GET if HEAD fails with a 500 or 501
	error code.

2007-08-21  Mauro Tortonesi  <[e-mail]>
	
	* http.c (http_loop): Send preliminary HEAD request if -N is given and
	the destination file exists already.

2007-08-10  Mauro Tortonesi  <[e-mail]>

	* http.c (http_loop): Fixed HTTP HEAD requests logic when --spider is
	given.

2007-08-10  Ralf Wildenhues  <[e-mail]>

	* url.c (append_uri_pathel): Do not assume dest string to be
	zero-terminated.
	(test_append_uri_pathel): Terminate string to fix test failure.

2007-08-09  Ralf Wildenhues  <[e-mail]>

	* url.c (url_string): Use comparison, not assignment, in
	check for auth_mode == URL_AUTH_HIDE_PASSWD.

2007-08-09  Micah Cowan  <[e-mail]>

	* http.c (http_loop): If we got a HEAD and then a GET, and the
	GET had a timestamp, use that one, not any we may have gotten
	from the HEAD.

2007-08-08  Micah Cowan  <[e-mail]>

	* init.c (defaults): Content disposition will not be default,
	since it currently results in extra round-trips.

2007-07-31  Micah Cowan  <[e-mail]>

	* http.c (gethttp): Set contlen = -1 when we encounter a
	negative-valued Content-Length header, so we don't consider it
	an internal error later on and call abort().

2007-07-29  Micah Cowan  <[e-mail]>

	* url.h, url.c (url_string): Replaced bool arg of the url_string
	function with enum url_auth_mode, with added option to
	completely remove user/pass auth information.
	* http.c, ftp.c, url.c, recur.c: Adapted call to url_string
	function to fit new usage.
	* recur.c (retrieve_tree): Remove auth info from Referer header.

2007-07-28  Micah Cowan  <[e-mail]>

	* options.h, init.c, retr.c, main.c: renamed opt maxredirect
	field to max_redirect, for improved consistency.
	* init.c: changed max_redirect parser from cmd_number_inf to
	cmd_number, as infinite redirects may not be appropriate.
	Alternatively, if cmd_number_inf should be used, then
	opt.max_redirect's value should be checked a bit differently in
	retr.c, to allow for the "infinite" meaning of zero.

2007-07-25  Micah Cowan  <[e-mail]>

	* http.c (create_authorization_line)
	(basic_authentication_encode, known_authentication_scheme_p)
	(load_cookies): Moved declarations up.
	(basic_authed_hosts): Added. Tracks what hosts have issued Basic
	challenge and been given the global username, password.
	(maybe_send_basic_creds): Added. Sends Basic creds for hosts that
	have issued Basic challenges.
	(register_basic_auth_host): Added. Instantiates
	basic_authed_hosts if necessary, then registers the host that
	has issued a challenge.
	(gethttp) <auth>: Only send authentication credentials after
	we've received a challenge from that host. This is a stop-gap
	fix until a proper fix can be implemented; still isn't quite
	right, as we should only be sending credentials automatically
	for authenticated paths and below, and not for the entire host.

2007-07-16  Joshua David Williams  <[e-mail]>

	* options.h: added maxredirect to options struct
	* init.c: added maxredirect to list of variables
	* retr.c (retrieve_url): replaced MAX_REDIRECTIONS with opt.maxredirect
	* main.c: added option --max-redirect

2007-07-16  Joshua David Williams  <[e-mail]>

	* test.h: tests made more verbose; now displays the name
	of each test run.

2007-07-10  Mauro Tortonesi  <[e-mail]>

	* http.c (http_loop): Fixed the HTTP requests logic. Now it skips the 
	preliminary HEAD request if either -O or --no-content-disposition are 
	given, and neither --spider and -N are given.

2007-07-05  Micah Cowan  <[e-mail]>

	* cmpt.c, connect.c, connect.h, convert.c, convert.h:
	* cookies.c, cookies.h, ftp-basic.c, ftp.c, ftp.h, ftp-ls.c:
	* ftp-opie.c, gen-md5.c, gen-md5.h, getopt.c, getopt.h, gnu-md5.c:
	* gnu-md5.h, gnutls.c, hash.c, hash.h, host.c, host.h:
	* html-parse.c, html-parse.h, html-url.c, http.c, http.h:
	* http-ntlm.c, http-ntlm.h, init.c, init.h, log.c, log.h, main.c:
	* Makefile.in, mswindows.c, mswindows.h, netrc.c, netrc.h:
	* openssl.c, options.h, progress.c, progress.h, ptimer.c:
	* ptimer.h, recur.c, recur.h, res.c, res.h, retr.c, retr.h:
	* safe-ctype.c, safe-ctype.h, spider.c, spider.h, ssl.h, sysdep.h:
	* test.c, test.h, url.c, url.h, utils.c, utils.h, wget.h:
	* xmalloc.c, xmalloc.h:
	Updated GPL reference to version 3 or later, removed FSF
	address.

2007-07-04  Mauro Tortonesi  <[e-mail]>

	* http.c (http_loop): Skip HEAD request and start immediately with GET
	if -O is given.

2007-02-02  Hrvoje Niksic  <[e-mail]>

	* http.c (print_server_response): Escape non-printable characters
	in server respone.

2007-02-02  Hrvoje Niksic  <[e-mail]>

	* netrc.c: Don't make netrc_list static, as it prevents
	compilation with DEBUG_MALLOC.

	* utils.c (aprintf): Don't use vasprintf when DEBUG_MALLOC is
	requested because, in that case, we want the calls to malloc to be
	coming from us.

2007-01-23  Hrvoje Niksic  <[e-mail]>

	* cookies.c (parse_set_cookie): Would erroneously discard cookies
	with unparsable expiry time.

2007-01-23  Hrvoje Niksic  <[e-mail]>

	* progress.c (create_image): Check for ETA overflow.
	(print_row_stats): Ditto.

2007-01-09  Mauro Tortonesi  <[e-mail]>

	* init.c (cmd_spec_prefer_family): Small fix to get rid of a gcc
	warning about strict-aliasing violation.

2007-01-09  Steven M. Schweda  <[e-mail]>

	* ftp-basic.c (ftp_syst): Fixed segfault if response text is missing.

2006-12-29  Gisle Vanem  <[e-mail]>

	* mswindows.c: Avoid a warning if 'ws_hangup()' is unused.

2006-12-27  Mauro Tortonesi  <[e-mail]>

	* http.c (parse_content_disposition): Consider directory prefix, if
	specified.

2006-11-21  Hrvoje Niksic  <[e-mail]>

	* retr.c (retrieve_from_file): Ditto.
	(url_uses_proxy): New function.

	* main.c (main): Don't check for opt.use_proxy when deciding
	whether to call retrieve_url or retrieve_tree; check whether the
	proxy would be used for *this* URL.

2006-10-17  Mike Grant  <[e-mail]>

	* ftp.c (ftp_loop_internal): Would incorrectly skip changing