flex.man

flex编译器的源代码

Version 2.5          Last change: April 1995                   23






FLEX(1)                  USER COMMANDS                    FLEX(1)



     associated with the given file and large enough to hold size
     characters (when in doubt, use YY_BUF_SIZE  for  the  size).
     It  returns  a  YY_BUFFER_STATE  handle,  which  may then be
     passed to other routines (see below).   The  YY_BUFFER_STATE
     type is a pointer to an opaque struct yy_buffer_state struc-
     ture, so you may safely initialize YY_BUFFER_STATE variables
     to  ((YY_BUFFER_STATE) 0) if you wish, and also refer to the
     opaque structure in order to correctly declare input buffers
     in  source files other than that of your scanner.  Note that
     the FILE pointer in the call  to  yy_create_buffer  is  only
     used  as the value of yyin seen by YY_INPUT; if you redefine
     YY_INPUT so it no longer uses yyin, then you can safely pass
     a nil FILE pointer to yy_create_buffer. You select a partic-
     ular buffer to scan from using:

         void yy_switch_to_buffer( YY_BUFFER_STATE new_buffer )

     switches the scanner's input  buffer  so  subsequent  tokens
     will  come  from new_buffer. Note that yy_switch_to_buffer()
     may be used by yywrap() to set things up for continued scan-
     ning, instead of opening a new file and pointing yyin at it.
     Note  also  that  switching   input   sources   via   either
     yy_switch_to_buffer()  or yywrap() does not change the start
     condition.

         void yy_delete_buffer( YY_BUFFER_STATE buffer )

     is used to reclaim the storage associated with a buffer.   (
     buffer  can be nil, in which case the routine does nothing.)
     You can also clear the current contents of a buffer using:

         void yy_flush_buffer( YY_BUFFER_STATE buffer )

     This function discards the buffer's contents,  so  the  next
     time  the scanner attempts to match a token from the buffer,
     it will first fill the buffer anew using YY_INPUT.

     yy_new_buffer() is an alias for yy_create_buffer(), provided
     for  compatibility  with  the  C++ use of new and delete for
     creating and destroying dynamic objects.

     Finally,   the    YY_CURRENT_BUFFER    macro    returns    a
     YY_BUFFER_STATE handle to the current buffer.

     Here is an example of using these  features  for  writing  a
     scanner  which expands include files (the <<EOF>> feature is
     discussed below):

         /* the "incl" state is used for picking up the name
          * of an include file
          */
         %x incl



Version 2.5          Last change: April 1995                   24






FLEX(1)                  USER COMMANDS                    FLEX(1)



         %{
         #define MAX_INCLUDE_DEPTH 10
         YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH];
         int include_stack_ptr = 0;
         %}

         %%
         include             BEGIN(incl);

         [a-z]+              ECHO;
         [^a-z\n]*\n?        ECHO;

         <incl>[ \t]*      /* eat the whitespace */
         <incl>[^ \t\n]+   { /* got the include file name */
                 if ( include_stack_ptr >= MAX_INCLUDE_DEPTH )
                     {
                     fprintf( stderr, "Includes nested too deeply" );
                     exit( 1 );
                     }

                 include_stack[include_stack_ptr++] =
                     YY_CURRENT_BUFFER;

                 yyin = fopen( yytext, "r" );

                 if ( ! yyin )
                     error( ... );

                 yy_switch_to_buffer(
                     yy_create_buffer( yyin, YY_BUF_SIZE ) );

                 BEGIN(INITIAL);
                 }

         <<EOF>> {
                 if ( --include_stack_ptr < 0 )
                     {
                     yyterminate();
                     }

                 else
                     {
                     yy_delete_buffer( YY_CURRENT_BUFFER );
                     yy_switch_to_buffer(
                          include_stack[include_stack_ptr] );
                     }
                 }

     Three routines are available for setting  up  input  buffers
     for  scanning  in-memory  strings  instead of files.  All of
     them create a new input buffer for scanning the string,  and
     return  a  corresponding  YY_BUFFER_STATE  handle (which you



Version 2.5          Last change: April 1995                   25






FLEX(1)                  USER COMMANDS                    FLEX(1)



     should delete with yy_delete_buffer() when  done  with  it).
     They    also    switch    to    the    new    buffer   using
     yy_switch_to_buffer(), so the  next  call  to  yylex()  will
     start scanning the string.

     yy_scan_string(const char *str)
          scans a NUL-terminated string.

     yy_scan_bytes(const char *bytes, int len)
          scans len bytes (including possibly NUL's) starting  at
          location bytes.

     Note that both of these functions create and scan a copy  of
     the  string or bytes.  (This may be desirable, since yylex()
     modifies the contents of the buffer it  is  scanning.)   You
     can avoid the copy by using:

     yy_scan_buffer(char *base, yy_size_t size)
          which scans in place the buffer starting at base,  con-
          sisting of size bytes, the last two bytes of which must
          be YY_END_OF_BUFFER_CHAR (ASCII NUL).  These  last  two
          bytes  are  not  scanned;  thus,  scanning  consists of
          base[0] through base[size-2], inclusive.

          If you fail to set up base in this manner (i.e., forget
          the   final   two  YY_END_OF_BUFFER_CHAR  bytes),  then
          yy_scan_buffer()  returns  a  nil  pointer  instead  of
          creating a new input buffer.

          The type yy_size_t is an integral type to which you can
          cast  an  integer expression reflecting the size of the
          buffer.

END-OF-FILE RULES
     The special rule "<<EOF>>" indicates actions which are to be
     taken  when  an  end-of-file  is  encountered  and  yywrap()
     returns non-zero (i.e., indicates no further files  to  pro-
     cess).  The action must finish by doing one of four things:

     -    assigning yyin to a new input file  (in  previous  ver-
          sions  of  flex,  after doing the assignment you had to
          call the special action YY_NEW_FILE; this is no  longer
          necessary);

     -    executing a return statement;

     -    executing the special yyterminate() action;

     -    or,    switching    to    a    new     buffer     using
          yy_switch_to_buffer() as shown in the example above.





Version 2.5          Last change: April 1995                   26






FLEX(1)                  USER COMMANDS                    FLEX(1)



     <<EOF>> rules may not be used with other patterns; they  may
     only  be  qualified  with a list of start conditions.  If an
     unqualified <<EOF>> rule is given, it applies to  all  start
     conditions  which  do  not already have <<EOF>> actions.  To
     specify an <<EOF>> rule for only the  initial  start  condi-
     tion, use

         <INITIAL><<EOF>>


     These rules are useful for  catching  things  like  unclosed
     comments.  An example:

         %x quote
         %%

         ...other rules for dealing with quotes...

         <quote><<EOF>>   {
                  error( "unterminated quote" );
                  yyterminate();
                  }
         <<EOF>>  {
                  if ( *++filelist )
                      yyin = fopen( *filelist, "r" );
                  else
                     yyterminate();
                  }


MISCELLANEOUS MACROS
     The macro YY_USER_ACTION can be defined to provide an action
     which is always executed prior to the matched rule's action.
     For example, it could be #define'd to call a routine to con-
     vert  yytext to lower-case.  When YY_USER_ACTION is invoked,
     the variable yy_act gives the number  of  the  matched  rule
     (rules  are  numbered starting with 1).  Suppose you want to
     profile how often each of your rules is matched.   The  fol-
     lowing would do the trick:

         #define YY_USER_ACTION ++ctr[yy_act]

     where ctr is an array to hold the counts for  the  different
     rules.   Note  that  the  macro YY_NUM_RULES gives the total
     number of rules (including the default rule, even if you use
     -s), so a correct declaration for ctr is:

         int ctr[YY_NUM_RULES];


     The macro YY_USER_INIT may be defined to provide  an  action
     which  is  always executed before the first scan (and before



Version 2.5          Last change: April 1995                   27






FLEX(1)                  USER COMMANDS                    FLEX(1)



     the scanner's internal initializations are done).  For exam-
     ple,  it  could  be used to call a routine to read in a data
     table or open a logging file.

     The macro yy_set_interactive(is_interactive) can be used  to
     control  whether  the  current buffer is considered interac-
     tive. An interactive buffer is processed  more  slowly,  but
     must  be  used  when  the  scanner's  input source is indeed
     interactive to avoid problems due to waiting to fill buffers
     (see the discussion of the -I flag below).  A non-zero value
     in the macro invocation marks the buffer as  interactive,  a
     zero  value as non-interactive.  Note that use of this macro
     overrides  %option  always-interactive  or  %option   never-
     interactive  (see Options below).  yy_set_interactive() must
     be invoked prior to beginning to scan the buffer that is (or
     is not) to be considered interactive.

     The macro yy_set_bol(at_bol) can be used to control  whether
     the  current  buffer's  scanning  context for the next token
     match is done as though at the beginning of a line.  A  non-
     zero macro argument makes rules anchored with

     The macro YY_AT_BOL() returns true if the next token scanned
     from  the  current  buffer will have '^' rules active, false
     otherwise.

     In the generated scanner, the actions are  all  gathered  in
     one  large  switch  statement  and separated using YY_BREAK,
     which may be redefined.  By default, it is simply a "break",
     to  separate  each  rule's action from the following rule's.
     Redefining  YY_BREAK  allows,  for  example,  C++  users  to
     #define  YY_BREAK  to  do  nothing (while being very careful
     that every rule ends with a "break" or a "return"!) to avoid
     suffering  from unreachable statement warnings where because
     a rule's action ends with "return", the YY_BREAK is inacces-
     sible.

VALUES AVAILABLE TO THE USER
     This section summarizes the various values available to  the
     user in the rule actions.

     -    char *yytext holds the text of the current  token.   It
          may  be  modified but not lengthened (you cannot append
          characters to the end).

          If the special directive %array appears  in  the  first
          section  of  the  scanner  description,  then yytext is
          instead declared char yytext[YYLMAX], where YYLMAX is a
          macro  definition  that  you  can redefine in the first
          section if you don't like the default value  (generally
          8KB).    Using   %array   results  in  somewhat  slower
          scanners, but the value of  yytext  becomes  immune  to



Version 2.5          Last change: April 1995                   28






FLEX(1)                  USER COMMANDS                    FLEX(1)



          calls to input() and unput(), which potentially destroy
          its value when yytext  is  a  character  pointer.   The
          opposite of %array is %pointer, which is the default.

          You cannot  use  %array  when  generating  C++  scanner
          classes (the -+ flag).

     -    int yyleng holds the length of the current token.

     -    FILE *yyin is the file  which  by  default  flex  reads
          from.   It  may  be  redefined  but doing so only makes
          sense before scanning begins or after an EOF  has  been
          encountered.  Changing it in the midst of scanning will
          have unexpected results since flex buffers  its  input;
          use  yyrestart()  instead.   Once  scanning  terminates
          because an end-of-file has been seen,  you  can  assign
          yyin  at  the  new input file and then call the scanner
          again to continue scanning.

     -    void yyrestart( FILE *new_file ) may be called to point
          yyin at the new input file.  The switch-over to the new
          file is immediate (any previously buffered-up input  is
          lost).   Note  that calling yyrestart() with yyin as an
          argument thus throws away the current input buffer  and
          continues scanning the same input file.

     -    FILE *yyout is the file to which ECHO actions are done.
          It can be reassigned by the user.

     -    YY_CURRENT_BUFFER returns a YY_BUFFER_STATE  handle  to
          the current buffer.

     -    YY_START returns an integer value corresponding to  the
          current start condition.  You can subsequently use