changelog

网络爬虫程序

version 0.3 (Jul 14 1997)
-----------
* enhanced X Window  user interface - now supports keyboard focus traversing 
  between widgets (work not perfect)
* most of widgets are modified
* new feature added - updating remote URL references in local tree to local in 
  HTML documents
* now is posible to enter more starting URLs  
* many bug fixes

version 0.3pl1 (Aug 6 1997)
--------------
* avoid to change modification time of file (I wana to implement document tree
  synchronisation soon)
* removed bug which results in hang when try to transfer moved robots.txt file
* now moved URLs are correctly rewritten in HTML document (broken in 0.3)
* more verbose reporting about moved documents

version 0.5 (Sep 25 1997)
--------------
* now every host name is converted to lower case to prevent redundance
* some changes in widget library 
* implemented transparent "reget" with FTP or HTTP protocol. Not every HTTP
  server supports reget. (Apache 1.2, Netscape, MSIIS, and ever HTTP/1.1 
  compliant server)
* now all files are at first stored with temporary name (possible use of reget
  in another run of program). When download is finished file gets true filename.
* new mode "resume regets" is implemented
* code restructulisation
* functions to convert date string to internal format (synchronisation ...)
* new mode "singlepage" added - download only one HML document with all inline
  objects (pictures, ...)
* server side map are now handled correctly
* repaired bug when anchor names are not writen in local URLs when rewriting
  (broken in 0.3, 0.3pl1, in previous versions was good)
* changes in file naming rules (each directory index is now stored in _._.html
  file not in index.html or ftp_dir_index.html) == better reverse transformation 
  from filename to URL. 
* implemented HTTP and FTP synchronization
* added new mode to SButton widget and its succesors to emulate on/off button
* Toggle implemented transparently (mixed use of SButton > , CheckButton , 
  CheckME)
* asynchronous connect when running in X Window mode
* !!!!!!!!!!!! changed name for subdirectory where www documents are stored from
  !!!!!!!!!!!! "www" to "http" (this make one of my colleague very sick :-))
* timeouts are now handled via "select()"
* now is each URL added to hash table too for better performance in 
  was_before() function - this means litle more work for each URL but when
  working on big set of URLs this will save lot of CPU time.
* simple SSL support by using of SSLeay
* removed some bugs
* added FTP proxy support
* update X Window interface and scheduler to reflect all changes
* updated documentation

version 0.5pl1 (Sep 30 1997)
--------------
* removed bug which avoid use of X Window interface when compiled without SSL
  support
* start to rewrite some of widgets
* all modes which scans local document tree now scans only  desired directories
* removed bug when pavuk sometimes hangs for long period if you try to schedule 

version 0.6 (Nov 11 1997)
-----------
* all command line parameters are handled transparently via param table
* each parameter is now posible to handle in "pavukrc" file
* !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
* WOW WOW WOW I finaly solve that problem with that dirty TreeWidget !!!!
* !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
* keyboard control for TreeWidget (ScrollTreeWidget)
* removed one big memory leak in get_abs_file_path()
* Combo widget
* Configuration managment via so called scenarios
* many bug fixes in X window interface
* more command line switches (oposits for booleans)
* removed bug in file_is_html() while checking if file successfuly opened
* removed bug in close_socket() -> "if (sock < 0) close(sock)" 
  ^^^^.. I love you strace.

version 0.6pl1 (Nov 13 1997)
--------------
* removed mistake with list parameters ( -asite , -dsite , -ddomain ...)
* removed bugs in -v -h parameters checking

version 0.6pl2 (Nov 16 1997)
--------------
* repared some bugs - scenario loading, Domain Allow/disallow switch ...
* extended scenario loader/saver to allow scenario dir selection
* repared html parser - \n or \r inside parsed tag results in bugy result
* command-line scenario saver

version 0.6pl3 (Dec 2 1997)
--------------
* limitation for size of transfered document added (-maxsize)
* limitation for MIME type of transfered document via HTTP/HTTPS 
  (-amimet/-dmimet)
* authorization for HTTP proxy added
* repared bug - Xtoolkit standart parameter were not recognized
* repared bug - when parent document were not successuly processed ,
  stays locked
* repared bug - when using HTTP proxy && conecting to SSL server
* added SSL proxy support
* added Gopher proxy support
* added gatewaing FTP and Gopher via HTTP proxy
* better FTP data connection handling
* progres meter on terminal (-progres)
* Log widget implemented

version 0.7 (Dec 30 1997)
-----------
* rewriten message reporting system for X Window - now based on Log widget
* added NLS support via GNU gettext
* created slovak message catalog by [e-mail] (zatial bez diakritiky)
* implemented removing of improper files directories (in sync mode)
* bug in FTP synchronization removed - bugy reply code check
* some needless FTP commands are not send while retrieving directory list -
  (MDTM . RETR)
* ftp data connection is established befor REST while restarting FTP transfer - 
  sometimes FTP server starts transfer from beginig instead of from given 
  position (I dont known why)
* checking of file size when synchronizing (FTP only)
* better FTP control connection handling
* some bug fixes
* loging messages to file
* solved problems with FTP synchronization

version 0.7pl1 (Jan 13 1998)
--------------
* added support for HTTP/HTTPS URLs with authentification informations :
  http://user:password@host:port/....
* in sync mode used standart UTC time instead of localtime - gmtime()
* ftp command MDTM sent only when required
* handling of HTML tag <META HTTP-EQUIV="Refresh" Content="..; URL=...">
* added in file stored authentification informations (read manual for authinfo
  file format)
* added more entries into mime type selection dialog 
  (from apache mime.types file)
* now pavuk sets return code of program to number of failed transfers
* now you can optionaly omit some directory levels from local doc tree 
  (try set -base_level $nr at command line and you will see what this means) 
* checking of write() fail
* progres is now reported corectly when restarting transfer
* changed some of widgets to have translatable strings
* repared bug in ScrollWin widget code , when TreeList or Log widget sometimes 
  jumps up
* asynchronous DNS name resolving via external process
  (breakable in X11 interface)
* dirty solved error in Col and Row widget when resizable widget gets zero size
* german message catalog by J黵gen Grieb

version 0.7pl2 (Jan 15 1998)
--------------
* repared compile bug in update_links.c (when compiling without X Window
  interface support)
* implemented buffered DNS requests in dns_gethostbyname()
* repared bug when downloading FTP directory via HTTP gateway and gateway 
  returns HTML document with local nor remote URLs
* implemeted so called dirty ftp proxy (-ftp_dirtyproxy) using CONNECT 
  request to HTTP proxy.
* repared bug in filename_to_url() http.password and http.user are not 
  initialised to NULL
* synchronisation with FTP<->HTTP gateway is now posible
* to translatable message catalog added geometry of window

version 0.7pl3 (Jan 26 1998)
--------------
* in sync mode is now reported corectly ,that document is up to date
* implemented active FTP data connection
* new slovak message catalog in ISO-8859-2 encoding by me
* you can now specify directory wrom which will be message catalog loaded
  (-msgcat or NLSMessageCatalogDir:)
* rewriten passing of X-attributes to be smarter translatable
* now each comand line switch can have own help text ==> easier management
  of massage catalogs && self documenting switches
* rewriten all interface dependent staff to easier support GTK
* some initial GTK things done
 
version 0.8 (Feb 27 1998)
-----------
* automake/autoconf compilation-configuration scripts == very easy
  installation
* GTK interface
* gnu-win32 portability
* rewriten HTML parsing code + HTML4.0 support
* fcntl locking on systems, where flock not supported
* some bugs in X-interface solved
* GTK Calendar widget
* minor bug fixes
* restriction on document creation time implemented
* rewriten parts of X-toolkit interface to look similiar as GTK interface
* czech message catalog by Petr Vyhnalek 

version 0.8pl1 (Mar 25 1998)
--------------
* some memory leaks removed
* URL based synchronisation
* command line scheduling (-schedule) 
* repared configure script : don't fail configuring GTK interface when Xpm or 
  Xext libraries not succesfully checked, gettext in glibc2 
* cyclic rescheduling (-reschedule)
* limit set of documents only on starting site (-dont_leave_site/-leave_site) 
* limit set of documents only on starting directory on starting site
  (-dont_leave_dir/-leave_dir)
* updated GTK interface for GTK+-0.99.4 =<
* inline objects are on same level of tree as parent when checking deep limit
* new option (-leave_level) to limit number of levels outside from starting 
  site 
* you can now disable compiling of URL tree preview (big memory save)
  run configure script with --disable-tree 
* solved bug in xinterface.c , which causes segfault in sprintf with some 
  versions of libc.
* man page is installable via make install
* solved problems in widgets, which refuse to run Xt interface in some 
  configurations

version 0.8pl2 (Mar 30 1998)
--------------
* repared bug in url_to_absolute_url() , when relative URL start with / , 
  was oddly rewriten.
* localedir in configure script now point in right place
* added pavuk.spec to distribution (for building RPMS)
* repared configure script to detect right Xext,Xt library in some i
  configurations
* extended set of unsafe characters in URL for encoding

version 0.8pl3 (Jun 9 1998)
--------------
* repared bug when pavuk seg faults if redirecting to unsupported protocol
* repared bug when pavuk miss part of tag between atribute name and value 
  of attribute while rewriting links inside HTML document
* repared bug in GTK interface - reading of uninitialised values

version 0.8pl4 (Jul 19 1998)
--------------
* added function CardBoxSwitchTo() to allow switching of Tabs in CardBox widget
* added "Open URL" dialog to File menu
* new mode "dontstore" implemented, for fetching files to proxy-cache
  servers
* added logo to About dialog

version 0.9 (Aug 5 1998)
-----------
* repared bug in HTTP proxy code
* totaly rewriten internal handling of URL tree !!!!!! 
  (thank to Marc David Rovners base idea and my hard long work :-) )
* now icons works in tree preview with GTK interface as in Xt interface
* updated czech message catalog
* window delete event is now handled right in GTK interface
 
version 0.9pl1 (Aug 9 1998)
--------------
* solved problems while compiling v0.9 without GUI
* repared bugs excelently reported by Dmitry Semenov
  - HTTP reget doesn't work in sync mode
  - -preserve_time doesn't work with FTP and only in sync mode
* I have get working menu with Tree preview in GTK interface :-) as in Xt 
  interface