awk

unix v7是最后一个广泛发布的研究型UNIX版本

.IT e
logarithm,
exponential,
and integer part of their respective arguments.
.PP
The name of one of these built-in functions,
without argument or parentheses,
stands for the value of the function on the
whole record.
The program
.P1
length < 10 || length > 20
.P2
prints lines whose length
is less than 10 or greater
than 20.
.PP
The function
.UL substr(s,\ m,\ n)
produces the substring of
.UL s
that begins at position
.UL m
(origin 1)
and is at most
.UL n
characters long.
If
.UL n
is omitted, the substring goes to the end of
.UL s .
The function
.UL index(s1,\ s2)
returns the position where the string
.UL s2
occurs in
.UL s1 ,
or zero if it does not.
.PP
The function
.UL sprintf(f,\ e1,\ e2,\ ...)
produces the value of the expressions
.UL e1 ,
.UL e2 ,
etc.,
in the
.UL printf
format specified by
.UL f .
Thus, for example,
.P1
x = sprintf("%8.2f %10ld", $1, $2)
.P2
sets
.UL x
to the string produced by formatting
the values of
.UL $1
and
.UL $2 .
.NH 2
Variables, Expressions, and Assignments
.PP
.IT Awk
variables take on numeric (floating point)
or string values according to context.
For example, in
.P1
x = 1
.P2
.UL x
is clearly a number, while in
.P1
x = "smith"
.P2
it is clearly a string.
Strings are converted to numbers and
vice versa whenever context demands it.
For instance,
.P1
x = "3" + "4"
.P2
assigns 7 to
.UL x .
Strings which cannot be interpreted
as numbers in a numerical context
will generally have numeric value zero,
but it is unwise to count on this behavior.
.PP
By default, variables (other than built-ins) are initialized to the null string,
which has numerical value zero;
this eliminates the need for most
.UL BEGIN
sections.
For example, the sums of the first two fields can be computed by
.P1
	{ s1 += $1; s2 += $2 }
END	{ print s1, s2 }
.P2
.PP
Arithmetic is done internally in floating point.
The arithmetic operators are
.UL + ,
.UL \- ,
.UL \(** ,
.UL / ,
and
.UL %
(mod).
The C increment
.UL ++
and
decrement
.UL \-\-
operators are also available,
and so are the assignment operators
.UL += ,
.UL \-= ,
.UL *= ,
.UL /= ,
and
.UL %= .
These operators may all be used in expressions.
.NH 2
Field Variables
.PP
Fields in
.IT awk
share essentially all of the properties of variables _
they may be used in arithmetic or string operations,
and may be assigned to.
Thus one can
replace the first field with a sequence number like this:
.P1
{ $1 = NR; print }
.P2
or
accumulate two fields into a third, like this:
.P1
{ $1 = $2 + $3; print $0 }
.P2
or assign a string to a field:
.P1
{ if ($3 > 1000)
	$3 = "too big"
  print
}
.P2
which replaces the third field by ``too big'' when it is,
and in any case prints the record.
.PP
Field references may be numerical expressions,
as in
.P1
{ print $i, $(i+1), $(i+n) }
.P2
Whether a field is deemed numeric or string depends on context;
in ambiguous cases like
.P1
if ($1 == $2) ...
.P2
fields are treated as strings.
.PP
Each input line is split into fields automatically as necessary.
It is also possible to split any variable or string
into fields:
.P1
n = split(s, array, sep)
.P2
splits the
the string
.UL s
into
.UL array[1] ,
\&...,
.UL array[n] .
The number of elements found is returned.
If the
.UL sep
argument is provided, it is used as the field separator;
otherwise
.UL FS
is used as the separator.
.NH 2
String Concatenation
.PP
Strings may be concatenated.
For example
.P1
length($1 $2 $3)
.P2
returns the length of the first three fields.
Or in a
.UL print
statement,
.P1
print $1 " is " $2
.P2
prints
the two fields separated by `` is ''.
Variables and numeric expressions may also appear in concatenations.
.NH 2
Arrays
.PP
Array elements are not declared;
they spring into existence by being mentioned.
Subscripts may have
.ul
any
non-null
value, including non-numeric strings.
As an example of a conventional numeric subscript,
the statement
.P1
x[NR] = $0
.P2
assigns the current input record to
the
.UL NR -th
element of the array
.UL x .
In fact, it is possible in principle (though perhaps slow)
to process the entire input in a random order with the
.IT awk
program
.P1
	{ x[NR] = $0 }
END	{ \fI... program ...\fP }
.P2
The first action merely records each input line in
the array
.UL x .
.PP
Array elements may be named by non-numeric values,
which gives
.IT awk
a capability rather like the associative memory of
Snobol tables.
Suppose the input contains fields with values like
.UL apple ,
.UL orange ,
etc.
Then the program
.P1
/apple/	{ x["apple"]++ }
/orange/	{ x["orange"]++ }
END		{ print x["apple"], x["orange"] }
.P2
increments counts for the named array elements,
and prints them at the end of the input.
.NH 2
Flow-of-Control Statements
.PP
.IT Awk
provides the basic flow-of-control statements
.UL if-else ,
.UL while ,
.UL for ,
and statement grouping with braces, as in C.
We showed the
.UL if
statement in section 3.3 without describing it.
The condition in parentheses is evaluated;
if it is true, the statement following the
.UL if
is done.
The
.UL else
part is optional.
.PP
The
.UL while
statement is exactly like that of C.
For example, to print all input fields one per line,
.P1
i = 1
while (i <= NF) {
	print $i
	++i
}
.P2
.PP
The
.UL for
statement is also exactly that of C:
.P1
for (i = 1; i <= NF; i++)
	print $i
.P2
does the same job as the
.UL while
statement above.
.PP
There is an alternate form of the
.UL for
statement which is suited for accessing the
elements of an associative array:
.P1
for (i in array)
	\fIstatement\f3
.P2
does
.ul
statement
with 
.UL i
set in turn to each element of
.UL array .
The elements are accessed in an apparently random order.
Chaos will ensue if 
.UL i
is altered, or if any new elements are
accessed during the loop.
.PP
The expression in the condition part of an
.UL if ,
.UL while
or
.UL for
can include relational operators like
.UL < ,
.UL <= ,
.UL > ,
.UL >= ,
.UL ==
(``is equal to''),
and
.UL !=
(``not equal to'');
regular expression matches with the match operators
.UL ~
and
.UL !~ ;
the logical operators
.UL \||\|| ,
.UL && ,
and
.UL ! ;
and of course parentheses for grouping.
.PP
The
.UL break
statement causes an immediate exit
from an enclosing
.UL while
or
.UL for ;
the
.UL continue
statement
causes the next iteration to begin.
.PP
The statement
.UL next
causes
.IT awk
to skip immediately to
the next record and begin scanning the patterns from the top.
The statement
.UL exit
causes the program to behave as if the end of the input
had occurred.
.PP
Comments may be placed in
.IT awk
programs:
they begin with the character
.UL #
and end with the end of the line,
as in
.P1
print x, y	# this is a comment
.P2
.NH
Design
.PP
The
.UX
system
already provides several programs that
operate by passing input through a
selection mechanism.
.IT Grep ,
the first and simplest, merely prints all lines which
match a single specified pattern.
.IT Egrep
provides more general patterns, i.e., regular expressions
in full generality;
.IT fgrep
searches for a set of keywords with a particularly fast algorithm.
.IT Sed\|
.[
unix programm manual
.]
provides most of the editing facilities of
the editor
.IT ed  ,
applied to a stream of input.
None of these programs provides
numeric capabilities,
logical relations,
or variables.
.PP
.IT Lex\|
.[
lesk lexical analyzer cstr
.]
provides general regular expression recognition capabilities,
and, by serving as a C program generator,
is essentially open-ended in its capabilities.
The use of
.IT lex ,
however, requires a knowledge of C programming,
and a
.IT lex
program must be compiled and loaded before use,
which discourages its use for one-shot applications.
.PP
.IT Awk
is an attempt
to fill in another part of the matrix of possibilities.
It
provides general regular expression capabilities
and an implicit input/output loop.
But it also provides convenient numeric processing,
variables,
more general selection,
and control flow in the actions.
It
does not require compilation or a knowledge of C.
Finally,
.IT awk
provides
a convenient way to access fields within lines;
it is unique in this respect.
.PP
.IT Awk
also tries to integrate strings and numbers
completely,
by treating all quantities as both string and numeric,
deciding which representation is appropriate
as late as possible.
In most cases the user can simply ignore the differences.
.PP
Most of the effort in developing
.I awk
went into deciding what
.I awk
should or should not do
(for instance, it doesn't do string substitution)
and what the syntax should be
(no explicit operator for concatenation)
rather
than on writing or debugging the code.
We have tried
to make the syntax powerful
but easy to use and well adapted
to scanning files.
For example,
the absence of declarations and implicit initializations,
while probably a bad idea for a general-purpose programming language,
is desirable in a language
that is meant to be used for tiny programs
that may even be composed on the command line.
.PP
In practice,
.IT awk
usage seems to fall into two broad categories.
One is what might be called ``report generation'' \(em
processing an input to extract counts,
sums, sub-totals, etc.
This also includes the writing of trivial
data validation programs,
such as verifying that a field contains only numeric information
or that certain delimiters are properly balanced.
The combination of textual and numeric processing is invaluable here.
.PP
A second area of use is as a data transformer,
converting data from the form produced by one program
into that expected by another.
The simplest examples merely select fields, perhaps with rearrangements.
.NH
Implementation
.PP
The actual implementation of
.IT awk
uses the language development tools available
on the
.UC UNIX
operating system.
The grammar is specified with
.IT yacc ;
.[
yacc johnson cstr
.]
the lexical analysis is done by
.IT lex ;
the regular expression recognizers are
deterministic finite automata
constructed directly from the expressions.
An
.IT awk
program is translated into a 
parse tree which is then directly executed
by a simple interpreter.
.PP
.IT Awk
was designed for ease of use rather than processing speed;
the delayed evaluation of variable types
and the necessity to break input
into fields makes high speed difficult to achieve in any case.
Nonetheless,
the program has not proven to be unworkably slow.
.PP
Table I below shows the execution (user + system) time
on a PDP-11/70 of
the
.UC UNIX
programs
.IT wc ,
.IT grep ,
.IT egrep ,
.IT fgrep ,
.IT sed ,
.IT lex ,
and
.IT awk
on the following simple tasks:
.IP "\ \ 1."
count the number of lines.
.IP "\ \ 2."
print all lines containing ``doug''.
.IP "\ \ 3."
print all lines containing ``doug'', ``ken'' or ``dmr''.
.IP "\ \ 4."
print the third field of each line.
.IP "\ \ 5."
print the third and second fields of each line, in that order.
.IP "\ \ 6."
append all lines containing ``doug'', ``ken'', and ``dmr''
to files ``jdoug'', ``jken'', and ``jdmr'', respectively.
.IP "\ \ 7."
print each line prefixed by ``line-number\ :\ ''.
.IP "\ \ 8."
sum the fourth column of a table.
.LP
The program
.IT wc
merely counts words, lines and characters in its input;
we have already mentioned the others.
In all cases the input was a file containing
10,000 lines
as created by the
command
.IT "ls \-l" ;
each line has the form
.P1
-rw-rw-rw- 1 ava 123 Oct 15 17:05 xxx
.P2
The total length of this input is
452,960 characters.
Times for
.IT lex
do not include compile or load.
.PP
As might be expected,
.IT awk
is not as fast as the specialized tools
.IT wc ,
.IT sed ,
or the programs in the
.IT grep
family,
but
is faster than the more general tool
.IT lex .
In all cases, the tasks were
about as easy to express as
.IT awk
programs
as programs in these other languages;
tasks involving fields were
considerably easier to express as
.IT awk
programs.
Some of the test programs are shown in
.IT awk ,
.IT sed
and
.IT lex .
.[
$LIST$
.]
.1C
.TS
center;
c c c c c c c c c
c c c c c c c c c
c|n|n|n|n|n|n|n|n|.
				Task
Program	1	2	3	4	5	6	7	8
_
\fIwc\fR	8.6
\fIgrep\fR	11.7	13.1
\fIegrep\fR	6.2	11.5	11.6
\fIfgrep\fR	7.7	13.8	16.1
\fIsed\fR	10.2	11.6	15.8	29.0	30.5	16.1
\fIlex\fR	65.1	150.1	144.2	67.7	70.3	104.0	81.7	92.8
\fIawk\fR	15.0	25.6	29.9	33.3	38.9	46.4	71.4	31.1
_
.TE
.sp
.ce
\fBTable I.\fR  Execution Times of Programs. (Times are in sec.)
.sp 2
.2C
.PP
The programs for some of these jobs are shown below.
The
.IT lex
programs are generally too long to show.
.LP
AWK:
.LP
.P1
1.	END	{print NR}
.P2
.P1
2.	/doug/
.P2
.P1
3.	/ken|doug|dmr/
.P2
.P1
4.	{print $3}
.P2
.P1
5.	{print $3, $2}
.P2
.P1
6.	/ken/	{print >"jken"}
	/doug/	{print >"jdoug"}
	/dmr/	{print >"jdmr"}
.P2
.P1
7.	{print NR ": " $0}
.P2
.P1
8.		{sum = sum + $4}
	END	{print sum}
.P2
.LP
SED:
.LP
.P1
1.	$=
.P2
.P1
2.	/doug/p
.P2
.P1
3.	/doug/p
	/doug/d
	/ken/p
	/ken/d
	/dmr/p
	/dmr/d
.P2
.P1
4.	/[^ ]* [ ]*[^ ]* [ ]*\e([^ ]*\e) .*/s//\e1/p
.P2
.P1
5.	/[^ ]* [ ]*\e([^ ]*\e) [ ]*\e([^ ]*\e) .*/s//\e2 \e1/p
.P2
.P1
6.	/ken/w jken
	/doug/w jdoug
	/dmr/w jdmr
.P2
.LP
LEX:
.LP
.P1
1.	%{
	int i;
	%}
	%%
	\en	i++;
	.	;
	%%
	yywrap() {
		printf("%d\en", i);
	}
.P2
.P1
2.	%%
	^.*doug.*$	printf("%s\en", yytext);
	.	;
	\en	;
.P2