url-filter-default

harvest是一个下载html网页得机器人

# $Id: URL-filter-default,v 2.3 2000/02/03 12:45:56 sxw Exp $
#
# Default URL-path filter file
#
# Prevents images, movies and sound files from being transferred and then
# thrown out
#
Deny	\.gif$
Deny	\.GIF$
Deny	\.jpeg$
Deny	\.JPEG$
Deny	\.jpg$
Deny	\.JPG$
Deny	\.png$
Deny	\.PNG$
Deny	\.tif$
Deny	\.TIF$
Deny	\.tiff$
Deny	\.TIFF$
Deny	\.mpg$
Deny	\.MPG$
Deny	\.mpeg$
Deny	\.MPEG$
Deny	\.mov$
Deny	\.MOV$
Deny	\.au$
Deny	\.AU$
Deny	\.mp3$
Deny	\.MP3$
Deny	\.snd$
Deny	\.SND$
Deny	\.wrl$
Deny	\.WRL$
Allow	.