awk

unix v7是最后一个广泛发布的研究型UNIX版本

.fp 3 G
....TM "78-1271-12, 78-1273-6" 39199 39199-11
.ND "September 1, 1978"
....TR 68
.RP
.	\" macros here
.tr _\(em
.if t .tr ~\(ap
.tr |\(or
.tr *\(**
.de UC
\&\\$3\s-1\\$1\\s0\&\\$2
..
.de IT
.if n .ul
\&\\$3\f2\\$1\fP\|\\$2
..
.de UL
.if n .ul
\&\\$3\f3\\$1\fP\&\\$2
..
.de P1
.DS I 3n
.nf
.if n .ta 5 10 15 20 25 30 35 40 45 50 55 60
.if t .ta .3i .6i .9i 1.2i
.if t .tr -\-'\(fm*\(**
.if t .tr _\(ul
.ft 3
.lg 0
.ss 18
.		\"use first argument as indent if present
..
.de P2
.ps \\n(PS
.vs \\n(VSp
.ft R
.ss 12
.if n .ls 2
.tr --''``^^!!
.if t .tr _\(em
.fi
.lg
.DE
..
.hw semi-colon
.hy 14
.		\"2=not last lines; 4= no -xx; 8=no xx-
.		\"special chars in programs
.de WS
.sp \\$1
..
.	\" end of macros
.TL
Awk \(em A Pattern Scanning and Processing Language
.br
(Second Edition)
.AU "MH 2C-522" 4862
Alfred V. Aho
.AU "MH 2C-518" 6021
Brian W. Kernighan
.AU "MH 2C-514" 7214
Peter J. Weinberger
.AI
.MH
.AB
.IT Awk
is a programming language whose
basic operation
is to search a set of files
for patterns, and to perform specified actions upon lines or fields of lines which
contain instances of those patterns.
.IT Awk
makes certain data selection and transformation operations easy to express;
for example, the
.IT awk
program
.sp
.ce
.ft 3
length > 72
.ft
.sp
prints all input lines whose length exceeds 72 characters;
the program
.ce
.sp
.ft 3
NF % 2 == 0
.ft R
.sp
prints all lines with an even number of fields;
and the program
.ce
.sp
.ft 3
{ $1 = log($1); print }
.ft R
.sp
replaces the first field of each line by its logarithm.
.PP
.IT Awk
patterns may include arbitrary boolean combinations of regular expressions
and of relational operators on strings, numbers, fields, variables, and array elements.
Actions may include the same pattern-matching constructions as in patterns,
as well as
arithmetic and string expressions and assignments,
.UL if-else ,
.UL while ,
.UL for
statements,
and multiple output streams.
.PP
This report contains a user's guide, a discussion of the design and implementation of
.IT awk ,
and some timing statistics.
....It supersedes TM-77-1271-5, dated September 8, 1977.
.AE
.CS 6 1 7 0 1 4
.if n .ls 2
.nr PS 9
.nr VS 11
.NH
Introduction
.if t .2C
.PP
.IT Awk
is a programming language designed to make
many common
information retrieval and text manipulation tasks
easy to state and to perform.
.PP
The basic operation of
.IT awk
is to scan a set of input lines in order,
searching for lines which match any of a set of patterns
which the user has specified.
For each pattern, an action can be specified;
this action will be performed on each line that matches the pattern.
.PP
Readers familiar with the
.UX
program
.IT grep\|
.[
unix program manual
.]
will recognize
the approach, although in
.IT awk
the patterns may be more
general than in
.IT grep ,
and the actions allowed are more involved than merely
printing the matching line.
For example, the
.IT awk
program
.P1
{print $3, $2}
.P2
prints the third and second columns of a table
in that order.
The program
.P1
$2 ~ /A\||B\||C/
.P2
prints all input lines with an A, B, or C in the second field.
The program
.P1
$1 != prev	{ print; prev = $1 }
.P2
prints all lines in which the first field is different
from the previous first field.
.NH 2
Usage
.PP
The command
.P1
awk  program  [files]
.P2
executes the
.IT awk
commands in
the string
.UL program
on the set of named files,
or on the standard input if there are no files.
The statements can also be placed in a file
.UL pfile ,
and executed by the command
.P1
awk  -f pfile  [files]
.P2
.NH 2
Program Structure
.PP
An
.IT awk
program is a sequence of statements of the form:
.P1
.ft I
	pattern	{ action }
	pattern	{ action }
	...
.ft 3
.P2
Each line of input
is matched against
each of the patterns in turn.
For each pattern that matches, the associated action
is executed.
When all the patterns have been tested, the next line
is fetched and the matching starts over.
.PP
Either the pattern or the action may be left out,
but not both.
If there is no action for a pattern,
the matching line is simply
copied to the output.
(Thus a line which matches several patterns can be printed several times.)
If there is no pattern for an action,
then the action is performed for every input line.
A line which matches no pattern is ignored.
.PP
Since patterns and actions are both optional,
actions must be enclosed in braces
to distinguish them from patterns.
.NH 2
Records and Fields
.PP
.IT Awk
input is divided into
``records'' terminated by a record separator.
The default record separator is a newline,
so by default
.IT awk
processes its input a line at a time.
The number of the current record is available in a variable
named
.UL NR .
.PP
Each input record
is considered to be divided into ``fields.''
Fields are normally separated by
white space \(em blanks or tabs \(em
but the input field separator may be changed, as described below.
Fields are referred to as
.UL "$1, $2,"
and so forth,
where
.UL $1
is the first field,
and
.UL $0
is the whole input record itself.
Fields may be assigned to.
The number of fields in the current record
is available in a variable named
.UL NF .
.PP
The variables
.UL FS
and
.UL RS
refer to the input field and record separators;
they may be changed at any time to any single character.
The optional command-line argument
\f3\-F\fIc\fR
may also be used to set
.UL FS
to the character
.IT c .
.PP
If the record separator is empty,
an empty input line is taken as the record separator,
and blanks, tabs and newlines are treated as field separators.
.PP
The variable
.UL FILENAME
contains the name of the current input file.
.NH 2
Printing
.PP
An action may have no pattern,
in which case the action is executed for
all
lines.
The simplest action is to print some or all of a record;
this is accomplished by the
.IT awk
command
.UL print .
The
.IT awk
program
.P1
{ print }
.P2
prints each record, thus copying the input to the output intact.
More useful is to print a field or fields from each record.
For instance, 
.P1
print $2, $1
.P2
prints the first two fields in reverse order.
Items separated by a comma in the print statement will be separated by the current output field separator
when output.
Items not separated by commas will be concatenated,
so
.P1
print $1 $2
.P2
runs the first and second fields together.
.PP
The predefined variables
.UL NF
and
.UL NR
can be used;
for example
.P1
{ print NR, NF, $0 }
.P2
prints each record preceded by the record number and the number of fields.
.PP
Output may be diverted to multiple files;
the program
.P1
{ print $1 >"foo1"; print $2 >"foo2" }
.P2
writes the first field,
.UL $1 ,
on the file
.UL foo1 ,
and the second field on file
.UL foo2 .
The
.UL >>
notation can also be used:
.P1
print $1 >>"foo"
.P2
appends the output to the file
.UL foo .
(In each case,
the output files are
created if necessary.)
The file name can be a variable or a field as well as a constant;
for example,
.P1
print $1 >$2
.P2
uses the contents of field 2 as a file name.
.PP
Naturally there is a limit on the number of output files;
currently it is 10.
.PP
Similarly, output can be piped into another process
(on
.UC UNIX
only); for instance,
.P1
print | "mail bwk"
.P2
mails the output to
.UL bwk .
.PP
The variables
.UL OFS
and
.UL ORS
may be used to change the current
output field separator and output
record separator.
The output record separator is
appended to the output of the
.UL print
statement.
.PP
.IT Awk
also provides the
.UL printf
statement for output formatting:
.P1
printf format expr, expr, ...
.P2
formats the expressions in the list
according to the specification
in
.UL format
and prints them.
For example,
.P1
printf "%8.2f  %10ld\en", $1, $2
.P2
prints 
.UL $1
as a floating point number 8 digits wide,
with two after the decimal point,
and
.UL $2
as a 10-digit long decimal number,
followed by a newline.
No output separators are produced automatically;
you must add them yourself,
as in this example.
The version of
.UL printf
is identical to that used with C.
.[
C programm language prentice hall 1978
.]
.NH 1
Patterns
.PP
A pattern in front of an action acts as a selector
that determines whether the action is to be executed.
A variety of expressions may be used as patterns:
regular expressions,
arithmetic relational expressions,
string-valued expressions,
and arbitrary boolean
combinations of these.
.NH 2
BEGIN and END
.PP
The special pattern
.UL BEGIN
matches the beginning of the input,
before the first record is read.
The pattern
.UL END
matches the end of the input,
after the last record has been processed.
.UL BEGIN
and
.UL END
thus provide a way to gain control before and after processing,
for initialization and wrapup.
.PP
As an example, the field separator
can be set to a colon by
.P1
BEGIN	{ FS = ":" }
.ft I
\&... rest of program ...
.ft 3
.P2
Or the input lines may be counted by
.P1
END  { print NR }
.P2
If
.UL BEGIN
is present, it must be the first pattern;
.UL END
must be the last if used.
.NH 2
Regular Expressions
.PP
The simplest regular expression is a literal string of characters
enclosed in slashes,
like
.P1
/smith/
.P2
This
is actually a complete
.IT awk
program which
will print all lines which contain any occurrence
of the name ``smith''.
If a line contains ``smith''
as part of a larger word,
it will also be printed, as in
.P1
blacksmithing
.P2
.PP
.IT Awk
regular expressions include the regular expression
forms found in
the
.UC UNIX
text editor
.IT ed\|
.[
unix program manual
.]
and
.IT grep
(without back-referencing).
In addition,
.IT awk
allows
parentheses for grouping, | for alternatives,
.UL +
for ``one or more'', and
.UL ?
for ``zero or one'',
all as in
.IT lex .
Character classes
may be abbreviated:
.UL [a\-zA\-Z0\-9]
is the set of all letters and digits.
As an example,
the
.IT awk
program
.P1
/[Aa]ho\||[Ww]einberger\||[Kk]ernighan/
.P2
will print all lines which contain any of the names
``Aho,'' ``Weinberger'' or ``Kernighan,''
whether capitalized or not.
.PP
Regular expressions
(with the extensions listed above)
must be enclosed in slashes,
just as in
.IT ed
and
.IT sed .
Within a regular expression,
blanks and the regular expression
metacharacters are significant.
To turn of the magic meaning
of one of the regular expression characters,
precede it with a backslash.
An example is the pattern
.P1
/\|\e/\^.\^*\e//
.P2
which matches any string of characters
enclosed in slashes.
.PP
One can also specify that any field or variable
matches
a regular expression (or does not match it) with the operators
.UL ~
and
.UL !~ .
The program
.P1
$1 ~ /[jJ]ohn/
.P2
prints all lines where the first field matches ``john'' or ``John.''
Notice that this will also match ``Johnson'', ``St. Johnsbury'', and so on.
To restrict it to exactly
.UL [jJ]ohn ,
use
.P1
$1 ~ /^[jJ]ohn$/
.P2
The caret ^ refers to the beginning
of a line or field;
the dollar sign
.UL $
refers to the end.
.NH 2
Relational Expressions
.PP
An
.IT awk
pattern can be a relational expression
involving the usual relational operators
.UL < ,
.UL <= ,
.UL == ,
.UL != ,
.UL >= ,
and
.UL > .
An example is
.P1
$2 > $1 + 100
.P2
which selects lines where the second field
is at least 100 greater than the first field.
Similarly,
.P1
NF % 2 == 0
.P2
prints lines with an even number of fields.
.PP
In relational tests, if neither operand is numeric,
a string comparison is made;
otherwise it is numeric.
Thus,
.P1
$1 >= "s"
.P2
selects lines that begin with an
.UL s ,
.UL t ,
.UL u ,
etc.
In the absence of any other information,
fields are treated as strings, so
the program
.P1
$1 > $2
.P2
will perform a string comparison.
.NH 2
Combinations of Patterns
.PP
A pattern can be any boolean combination of patterns,
using the operators
.UL \||\||
(or),
.UL &&
(and), and
.UL !
(not).
For example,
.P1
$1 >= "s" && $1 < "t" && $1 != "smith"
.P2
selects lines where the first field begins with ``s'', but is not ``smith''.
.UL &&
and
.UL \||\||
guarantee that their operands
will be evaluated
from left to right;
evaluation stops as soon as the truth or falsehood
is determined.
.NH 2
Pattern Ranges
.PP
The ``pattern'' that selects an action may also
consist of two patterns separated by a comma, as in
.P1
pat1, pat2	{ ... }
.P2
In this case, the action is performed for each line between
an occurrence of
.UL pat1
and the next occurrence of
.UL pat2
(inclusive).
For example,
.P1
/start/, /stop/
.P2
prints all lines between
.UL start
and
.UL stop ,
while
.P1
NR == 100, NR == 200 { ... }
.P2
does the action for lines 100 through 200
of the input.
.NH 1
Actions
.PP
An
.IT awk
action is a sequence of action statements
terminated by newlines or semicolons.
These action statements can be used to do a variety of
bookkeeping and string manipulating tasks.
.NH 2
Built-in Functions
.PP
.IT Awk
provides a ``length'' function
to compute the length of a string of characters.
This program prints each record,
preceded by its length:
.P1
{print length, $0}
.P2
.UL length
by itself is a ``pseudo-variable'' which
yields the length of the current record;
.UL length(argument)
is a function which yields the length of its argument,
as in
the equivalent
.P1
{print length($0), $0}
.P2
The argument may be any expression.
.PP
.IT Awk
also
provides the arithmetic functions
.UL sqrt ,
.UL log ,
.UL exp ,
and
.UL int ,
for
square root,
base