readme

harvest是一个下载html网页得机器人

This directory holds the RootNode and LeafNode enumeration programs.

staturl is used to enumerate LeafNode URLs.  It simply prints an MD5 for
each URL, or a timestamp for each file: URL.  It will print timestamps
for other URLs like ftp: or gopher: (Gopher+ protocol) sometime soon.

enum is used to enumerate RootNode URLs.  enum calls fileenum, ftpenum,
gopherenum, and httpenum to perform the RootNode URL enumeration using an
access-type-specific algorithm.