glimpse.1

harvest是一个下载html网页得机器人

both patterns.
\fIglimpse -F 'gnu;\\.c$' 'define;DEFAULT'\fR
will output all lines containing both 'define' and 'DEFAULT'
(anywhere in the line, not necessarily in order) in
files whose name contains 'gnu' and ends with .c.
\fIglimpse '{political,computer};science'\fR will match 'political science' 
or 'science of computers'.
The NOT operation works only together with the -W option and it is
generally applies only to the whole file rather to individual records.
Its output may sometimes seem counterintuitive.
Use with care.
\fIglimpse -W 'fame;~glory'\fR will output all lines containing 'fame'
in all files that contain 'fame' but do not contain 'glory';
This is the most common use of NOT, and in this case it works
as expected.
\fIglimpse -W '~{fame;glory}'\fR will be limited to files that do
not contain both words, and will output all lines containing one
of them.
.TP
\fBWild cards\fP
The symbol '#' is used to denote a sequence 
of any number (including 0) 
of arbitrary characters (see LIMITATIONS).  
The symbol # is equivalent to .* in egrep.
In fact, .* will work too, because it is a valid regular expression
(see below), but unless this is part of an actual regular expression,
# will work faster. 
(Currently glimpse is experiencing some problems with #.)
.TP
\fBCombination of exact and approximate matching\fP
Any pattern inside angle brackets <> must match the text exactly even
if the match is with errors.  For example, <mathemat>ics matches
mathematical with one error (replacing the last s with an a), but
mathe<matics> does not match mathematical no matter how many errors are
allowed.
(This option is buggy at the moment.)
.TP
\fBRegular expressions\fP
Since the index is word based, a regular expression must match
words that appear in the index for glimpse to find it.
Glimpse first strips the regular expression from all non-alphabetic
characters, and searches the index for all remaining words.
It then applies the regular expression matching algorithm to the
files found in the index.
For example, \fIglimpse\fP 'abc.*xyz' will search the index
for all files that contain both 'abc' and 'xyz', and then
search directly for 'abc.*xyz' in those files.
(If you use glimpse \-w 'abc.*xyz', then 'abcxyz' will not be found, 
because glimpse
will think that abc and xyz need to be matches to whole words.)
The syntax of regular expressions in \fBglimpse\fP is in general the same as
that for \fBagrep\fP.  The union operation `|', Kleene closure `*',
and parentheses () are all supported.
Currently '+' is not supported.
Regular expressions are currently limited to approximately 30
characters (generally excluding meta characters).  Some options
(\-d, \-w, \-t, \-x, \-D, \-I, \-S) do not 
currently work with regular expressions.
The maximal number of errors for regular expressions that use '*'
or '|' is 4. (See LIMITATIONS.)
.TP
\fBstructured queries\fP
Glimpse supports some form of structured queries using Harvest's SOIF
format.  See STRUCTURED QUERIES below for details.
.SH EXAMPLES
.LP
(Run "glimpse '^glimpse' this-file" to get a list of all examples, some
of which were given earlier.)
.TP
glimpse -F 'haystack.h$' needle
finds all needles in all haystack.h's files.
.TP
glimpse -2 -F html Anestesiology
outputs all occurrences of Anestesiology with two errors in files with
html somewhere in their full name.
.TP
glimpse -l -F '\\.c$' variablename
lists the names of all .c files that contain variablename
(the -l option lists file names rather than output the matched lines).
.TP
glimpse -F 'mail;1993' 'windsurfing;Arizona'
finds all lines containing \fIwindsurfing\fP and \fIArizona\fP in all files
having `mail' and '1993' somewhere in their full name.
.TP
glimpse -F mail 't.j@#uk'
finds all mail addresses (search only files with mail somewhere in
their name) from the uk, where the login name ends with
t.j, where the . stands for any one character. 
(This is very useful to find a login name of someone whose middle name
you don't know.)
.TP
glimpse -F mbox -h -G  . > MBOX
concatenates all files whose name matches `mbox' into one big one.
.SH "SEARCHING IN COMPRESSED FILES"
.LP
Glimpse includes an optional new compression program,
called \fIcast\fP,
which allows glimpse (and agrep) to search the compressed files
without having to decompress them.  The search is actually significantly
faster when the files are compressed.  However, we have not tested
\fIcast\fP as thoroughly as we would have liked, and a mishap in
a compression algorithm can cause loss of data, so we recommend
at this point to use \fIcast\fP very carefully.
We do not support or maintain cast.
(Unless you specifically use \fIcast\fP, the default is to ignore it.)
.SH "GLIMPSEINDEX FILES"
.LP
All files used by glimpse are located at the directory(ies) where 
the index(es) is (are) stored and have .glimpse_ as a prefix.
The first two files (.glimpse_exclude and .glimpse_include) are
optionally supplied by the user.  The other files are built and
read by glimpse.
.LP
.IP "\fB.glimpse_exclude\fR"
contains a list of files that glimpseindex is explicitly told to ignore.  
In general, the syntax of .glimpse_exclude/include is the same as
that of agrep (or any other grep).  The lines in the .glimpse_exclude
file are matched to the file names, and if they match, the files 
are excluded.  Notice that agrep matches to parts of the string!
e.g., agrep /ftp/pub will match /home/ftp/pub and /ftp/pub/whatever.
So, if you want to exclude /ftp/pub/core, you just list
it, as is, in the .glimpse_exclude file.
If you put "/home/ftp/pub/cdrom" in .glimpse_exclude, every file
name that matches that string will be excluded, meaning all files
below it.
You can use ^ to indicate the beginning of a file name, and $ to
indicate the end of one, and you can use * and ? in the usual way.
For example /ftp/*html will exclude /ftp/pub/foo.html, but will
also exclude /home/ftp/pub/html/whatever;  if you want to exclude
files that start with /ftp and end with html use ^/ftp*html$
Notice that putting a * at the beginning or at the end is redundant
(in fact, in this case glimpseindex will remove the * when it 
does the indexing).
No other meta characters are allowed in .glimpse_exclude
(e.g., don't use .* or # or |).
Lines with * or ? must have no more than 30 characters.
Notice that, although the index itself will not be indexed,
the list of file names (.glimpse_filenames) will be indexed
unless it is explicitly listed in .glimpse_exclude.
.IP "\fB.glimpse_filters\fR"
See the description above for the -z option.
.IP "\fB.glimpse_include\fR"
contains a list of files that glimpseindex
is explicitly told to \fIinclude\fP in the index even though they may look
like non-text files.  Symbolic links are followed by glimpseindex 
only if they are specifically included here.
If a file is in both .glimpse_exclude and .glimpse_include it will be
excluded.
.IP "\fB.glimpse_filenames\fP"
contains the list of all indexed file names, one per line.
This is an ASCII file that can also be used with agrep to search
for a file name leading to a fast find command.
For example, 
.br
glimpse 'count#\\.c$' ~/.glimpse_filenames
.br
will output the names of all (indexed) .c files that have 'count' in
their name (including anywhere on the path from the index).
Setting the following alias in the .login file may be useful:
.br
alias findfile 'glimpse -h \!:1 ~/.glimpse_filenames'
.IP ".\fBglimpse_index\fP"
contains the index.  The index consists of lines, each starting with a
word followed by a list of block numbers (unless the -o or -b options 
are used, in which case each word is followed by an offset into 
the file .glimpse_partitions where all pointers are kept).
The block/file numbers are stored in binary form, so this is not an ASCII file.
.IP "\fB.glimpse_messages\fP"
contains the output of the -w option (see above).
.IP "\fB.glimpse_partitions\fP"
contains the partition of the indexed space into blocks
and, when the index is built with the -o or -b options, some part of the
index.  This file is used internally by glimpse and it is
a non-ASCII file.
.IP "\fB.glimpse_statistics\fP"
contains some statistics about the makeup of the index.  Useful for
some advanced applications and customization of glimpse.
.IP "\fB.glimpse_turbo\fP"
An added data structure (used under glimpseindex -o or -b only) 
that helps to speed up
queries significantly for large indexes.  Its size is 0.25MB.
Glimpse will work without it if needed.
.SH "STRUCTURED QUERIES"
Glimpse can search for Boolean combinations of "attribute=value" terms
by using the Harvest SOIF parser library (in glimpse/libtemplate). 
To search this way, the index must be made by using the -s option of
glimpseindex (this can be used in conjunction with other glimpseindex
options). For glimpse and glimpseindex to recognize "structured" files,
they must be in SOIF format. In this format, each value is prefixed by
an attribute-name with the size of the value (in bytes) present in "{}"
after the name of the attribute.  
For example, The following lines are part of an SOIF file:
.br
.nf
type{17}:       Directory-Listing
md5{32}:        3858c73d68616df0ed58a44d306b12ba
.fi
Any string can serve as an attribute name.
Glimpse "pattern;type=Directory-Listing" will search for "pattern"
only in files whose type is "Directory-Listing".
The file itself is considered to be
one "object" and its name/url appears as the first attribute with an
"@" prefix; e.g.,
@FILE { http://xxx... }
The scope of Boolean operations changes from records
(lines) to whole files when structured queries are used in glimpse
(since individual query terms can look at different attributes and they
may not be "covered" by the record/line).  Note that glimpse can only
search for patterns in the value parts of the SOIF file: there are some 
attributes (like the TTL, MD5, etc.) that are interpreted by Harvest's
internal routines.
See http://harvest.cs.colorado.edu/harvest/user-manual/ for more detailed
information of the SOIF format.
.SH "REFERENCES"
.IP 1.
U. Manber and S. Wu,
"GLIMPSE: A Tool to Search Through Entire File Systems,"
\fIUsenix Winter 1994 Technical Conference\fP
(best paper award),
San Francisco (January 1994), pp. 23\-32.
Also, Technical Report #TR 93-34, Dept. of Computer Science,
University of Arizona, October 1993 (a postscript file
is available by anonymous ftp at
ftp://ftp.cs.arizona.edu/reports/1993/TR93-34.ps).
.IP 2.
S. Wu and U. Manber,
"Fast Text Searching Allowing Errors,"
\fICommunications of the ACM\fP
\fB35\fP (October 1992), pp. 83\-91.
.SH "SEE ALSO"
.BR agrep (1),
.BR ed (1),
.BR ex (1),
.BR glimpseindex (1),
.BR glimpseserver (1),
.BR grep (1),
.BR sh (1),
.BR csh (1).
.SH LIMITATIONS
.LP
The index of glimpse is word based.  A pattern that contains more than
one word cannot be found in the index.  The way glimpse overcomes this
weakness is by splitting any multi-word pattern into its set of words
and looking for all of them in the index.
For example, \fBglimpse 'linear programming'\fR will first consult the index
to find all files containing both \fIlinear\fP and \fIprogramming\fP,
and then apply agrep to find the combined pattern.
This is usually an effective solution, but it can be slow for
cases where both words are very common, but their combination is not.
.LP
As was mentioned in the section on PATTERNS above, some characters
serve as meta characters for glimpse and need to be
preceded by '\\' to search for them.  The most common
examples are the characters '.' (which stands for a wild card),
and '*' (the Kleene closure).
So, "glimpse ab.de" will match abcde, but "glimpse ab\\.de"
will not, and "glimpse ab*de" will not match ab*de, but 
"glimpse ab\\*de" will.
The meta character - is translated automatically to a hypen
unless it appears between [] (in which case it denotes a range of
characters).
.LP
The index of glimpse stores all patterns in lower case.
When glimpse searches the index it first converts
all patterns to lower case, finds the appropriate files,
and then searches the actual files using the original
patterns.
So, for example, \fIglimpse ABCXYZ\fR will first find all
files containing abcxyz in any combination of lower and upper
cases, and then searches these files directly, so only the
right cases will be found.
One problem with this approach is discovering misspellings
that are caused by wrong cases.
For example, \fIglimpse -B abcXYZ\fR will first search the
index for the best match to abcxyz (because the pattern is
converted to lower case); it will find that there are matches
with no errors, and will go to those files to search them
directly, this time with the original upper cases. 
If the closest match is, say AbcXYZ, glimpse may miss it,
because it doesn't expect an error.
Another problem is speed.  If you search for "ATT", it will look
at the index for "att".  Unless you use -w to match the whole word,
glimpse may have to search all files containing, for example, "Seattle"
which has "att" in it.
.LP
There is no size limit for simple patterns and simple patterns
within Boolean expressions.
More complicated patterns, such as regular expressions,
are currently limited to approximately 30 characters.
Lines are limited to 1024 characters.
Records are limited to 48K, and may be truncated if they are larger
than that.
The limit of record length can be 
changed by modifying the parameter Max_record in agrep.h.
.LP
Glimpseindex does not index words of size > 64.
.SH BUGS
.LP
In some rare cases, regular expressions using * or # may not match correctly.
.LP
A query that contains no alphanumeric characters is not
recommended (unless glimpse is used as agrep and the file names
are provided).  This is an understatement.
.LP
The notion of "match to the whole word" (the \-w option) can be tricky
sometimes.  For example, glimpse -w 'word$' will not match 'word'
appearing at the end of a line, because the extra '$' makes the pattern
more than just one simple word.
The same thing can happen with ^ and with _.
To be on the safe side,
use the -w option only when the patterns are actual words.
.LP
Please send bug reports or comments to [e-mail].
.SH DIAGNOSTICS
Exit status is 0 if any matches are found,
1 if none, 2 for syntax errors or inaccessible files.
.SH AUTHORS
Udi Manber and Burra Gopal, Department of Computer Science, 
University of Arizona, and Sun Wu, the National Chung-Cheng University,
Taiwan. (Email:  [e-mail])