cppinternals.texi

理解和实践操作系统的一本好书

function-like macro expansion.
@end itemize

The @code{cpp_token} structure contains @code{line} and @code{col}
members.  The lexer fills these in with the line and column of the first
character of the token.  Consequently, but maybe unexpectedly, a token
from the replacement list of a macro expansion carries the location of
the token within the @code{#define} directive, because cpplib expands a
macro by returning pointers to the tokens in its replacement list.  The
current implementation of cpplib assigns tokens created from built-in
macros and the @samp{#} and @samp{##} operators the location of the most
recently lexed token.  This is a because they are allocated from the
lexer's token runs, and because of the way the diagnostic routines infer
the appropriate location to report.

The diagnostic routines in cpplib display the location of the most
recently @emph{lexed} token, unless they are passed a specific line and
column to report.  For diagnostics regarding tokens that arise from
macro expansions, it might also be helpful for the user to see the
original location in the macro definition that the token came from.
Since that is exactly the information each token carries, such an
enhancement could be made relatively easily in future.

The stand-alone preprocessor faces a similar problem when determining
the correct line to output the token on: the position attached to a
token is fairly useless if the token came from a macro expansion.  All
tokens on a logical line should be output on its first physical line, so
the token's reported location is also wrong if it is part of a physical
line other than the first.

To solve these issues, cpplib provides a callback that is generated
whenever it lexes a preprocessing token that starts a new logical line
other than a directive.  It passes this token (which may be a
@code{CPP_EOF} token indicating the end of the translation unit) to the
callback routine, which can then use the line and column of this token
to produce correct output.

@section Representation of line numbers

As mentioned above, cpplib stores with each token the line number that
it was lexed on.  In fact, this number is not the number of the line in
the source file, but instead bears more resemblance to the number of the
line in the translation unit.

The preprocessor maintains a monotonic increasing line count, which is
incremented at every new line character (and also at the end of any
buffer that does not end in a new line).  Since a line number of zero is
useful to indicate certain special states and conditions, this variable
starts counting from one.

This variable therefore uniquely enumerates each line in the translation
unit.  With some simple infrastructure, it is straight forward to map
from this to the original source file and line number pair, saving space
whenever line number information needs to be saved.  The code the
implements this mapping lies in the files @file{line-map.c} and
@file{line-map.h}.

Command-line macros and assertions are implemented by pushing a buffer
containing the right hand side of an equivalent @code{#define} or
@code{#assert} directive.  Some built-in macros are handled similarly.
Since these are all processed before the first line of the main input
file, it will typically have an assigned line closer to twenty than to
one.

@node Guard Macros
@unnumbered The Multiple-Include Optimization
@cindex guard macros
@cindex controlling macros
@cindex multiple-include optimization

Header files are often of the form

@smallexample
#ifndef FOO
#define FOO
@dots{}
#endif
@end smallexample

@noindent
to prevent the compiler from processing them more than once.  The
preprocessor notices such header files, so that if the header file
appears in a subsequent @code{#include} directive and @code{FOO} is
defined, then it is ignored and it doesn't preprocess or even re-open
the file a second time.  This is referred to as the @dfn{multiple
include optimization}.

Under what circumstances is such an optimization valid?  If the file
were included a second time, it can only be optimized away if that
inclusion would result in no tokens to return, and no relevant
directives to process.  Therefore the current implementation imposes
requirements and makes some allowances as follows:

@enumerate
@item
There must be no tokens outside the controlling @code{#if}-@code{#endif}
pair, but whitespace and comments are permitted.

@item
There must be no directives outside the controlling directive pair, but
the @dfn{null directive} (a line containing nothing other than a single
@samp{#} and possibly whitespace) is permitted.

@item
The opening directive must be of the form

@smallexample
#ifndef FOO
@end smallexample

or

@smallexample
#if !defined FOO     [equivalently, #if !defined(FOO)]
@end smallexample

@item
In the second form above, the tokens forming the @code{#if} expression
must have come directly from the source file---no macro expansion must
have been involved.  This is because macro definitions can change, and
tracking whether or not a relevant change has been made is not worth the
implementation cost.

@item
There can be no @code{#else} or @code{#elif} directives at the outer
conditional block level, because they would probably contain something
of interest to a subsequent pass.
@end enumerate

First, when pushing a new file on the buffer stack,
@code{_stack_include_file} sets the controlling macro @code{mi_cmacro} to
@code{NULL}, and sets @code{mi_valid} to @code{true}.  This indicates
that the preprocessor has not yet encountered anything that would
invalidate the multiple-include optimization.  As described in the next
few paragraphs, these two variables having these values effectively
indicates top-of-file.

When about to return a token that is not part of a directive,
@code{_cpp_lex_token} sets @code{mi_valid} to @code{false}.  This
enforces the constraint that tokens outside the controlling conditional
block invalidate the optimization.

The @code{do_if}, when appropriate, and @code{do_ifndef} directive
handlers pass the controlling macro to the function
@code{push_conditional}.  cpplib maintains a stack of nested conditional
blocks, and after processing every opening conditional this function
pushes an @code{if_stack} structure onto the stack.  In this structure
it records the controlling macro for the block, provided there is one
and we're at top-of-file (as described above).  If an @code{#elif} or
@code{#else} directive is encountered, the controlling macro for that
block is cleared to @code{NULL}.  Otherwise, it survives until the
@code{#endif} closing the block, upon which @code{do_endif} sets
@code{mi_valid} to true and stores the controlling macro in
@code{mi_cmacro}.

@code{_cpp_handle_directive} clears @code{mi_valid} when processing any
directive other than an opening conditional and the null directive.
With this, and requiring top-of-file to record a controlling macro, and
no @code{#else} or @code{#elif} for it to survive and be copied to
@code{mi_cmacro} by @code{do_endif}, we have enforced the absence of
directives outside the main conditional block for the optimization to be
on.

Note that whilst we are inside the conditional block, @code{mi_valid} is
likely to be reset to @code{false}, but this does not matter since
the closing @code{#endif} restores it to @code{true} if appropriate.

Finally, since @code{_cpp_lex_direct} pops the file off the buffer stack
at @code{EOF} without returning a token, if the @code{#endif} directive
was not followed by any tokens, @code{mi_valid} is @code{true} and
@code{_cpp_pop_file_buffer} remembers the controlling macro associated
with the file.  Subsequent calls to @code{stack_include_file} result in
no buffer being pushed if the controlling macro is defined, effecting
the optimization.

A quick word on how we handle the

@smallexample
#if !defined FOO
@end smallexample

@noindent
case.  @code{_cpp_parse_expr} and @code{parse_defined} take steps to see
whether the three stages @samp{!}, @samp{defined-expression} and
@samp{end-of-directive} occur in order in a @code{#if} expression.  If
so, they return the guard macro to @code{do_if} in the variable
@code{mi_ind_cmacro}, and otherwise set it to @code{NULL}.
@code{enter_macro_context} sets @code{mi_valid} to false, so if a macro
was expanded whilst parsing any part of the expression, then the
top-of-file test in @code{push_conditional} fails and the optimization
is turned off.

@node Files
@unnumbered File Handling
@cindex files

Fairly obviously, the file handling code of cpplib resides in the file
@file{files.c}.  It takes care of the details of file searching,
opening, reading and caching, for both the main source file and all the
headers it recursively includes.

The basic strategy is to minimize the number of system calls.  On many
systems, the basic @code{open ()} and @code{fstat ()} system calls can
be quite expensive.  For every @code{#include}-d file, we need to try
all the directories in the search path until we find a match.  Some
projects, such as glibc, pass twenty or thirty include paths on the
command line, so this can rapidly become time consuming.

For a header file we have not encountered before we have little choice
but to do this.  However, it is often the case that the same headers are
repeatedly included, and in these cases we try to avoid repeating the
filesystem queries whilst searching for the correct file.

For each file we try to open, we store the constructed path in a splay
tree.  This path first undergoes simplification by the function
@code{_cpp_simplify_pathname}.  For example,
@file{/usr/include/bits/../foo.h} is simplified to
@file{/usr/include/foo.h} before we enter it in the splay tree and try
to @code{open ()} the file.  CPP will then find subsequent uses of
@file{foo.h}, even as @file{/usr/include/foo.h}, in the splay tree and
save system calls.

Further, it is likely the file contents have also been cached, saving a
@code{read ()} system call.  We don't bother caching the contents of
header files that are re-inclusion protected, and whose re-inclusion
macro is defined when we leave the header file for the first time.  If
the host supports it, we try to map suitably large files into memory,
rather than reading them in directly.

The include paths are internally stored on a null-terminated
singly-linked list, starting with the @code{"header.h"} directory search
chain, which then links into the @code{<header.h>} directory chain.

Files included with the @code{<foo.h>} syntax start the lookup directly
in the second half of this chain.  However, files included with the
@code{"foo.h"} syntax start at the beginning of the chain, but with one
extra directory prepended.  This is the directory of the current file;
the one containing the @code{#include} directive.  Prepending this
directory on a per-file basis is handled by the function
@code{search_from}.

Note that a header included with a directory component, such as
@code{#include "mydir/foo.h"} and opened as
@file{/usr/local/include/mydir/foo.h}, will have the complete path minus
the basename @samp{foo.h} as the current directory.

Enough information is stored in the splay tree that CPP can immediately
tell whether it can skip the header file because of the multiple include
optimization, whether the file didn't exist or couldn't be opened for
some reason, or whether the header was flagged not to be re-used, as it
is with the obsolete @code{#import} directive.

For the benefit of MS-DOS filesystems with an 8.3 filename limitation,
CPP offers the ability to treat various include file names as aliases
for the real header files with shorter names.  The map from one to the
other is found in a special file called @samp{header.gcc}, stored in the
command line (or system) include directories to which the mapping
applies.  This may be higher up the directory tree than the full path to
the file minus the base name.

@node Concept Index
@unnumbered Concept Index
@printindex cp

@bye