flex.1

操作系统设计与实现源码

FLEX(1)                   Minix Programmer's Manual                    FLEX(1)


SUMMARY OF SPECIAL ACTIONS
     In addition to arbitrary C code, the following can appear in actions:

     -    ECHO copies yytext to the scanner's output.

     -    BEGIN followed by the name of a start condition places  the  scanner
          in the corresponding start condition.

     -    REJECT directs the scanner to proceed on to the "second  best"  rule
          which  matched  the  input  (or  a prefix of the input).  yytext and
          yyleng are set up appropriately.  Note that REJECT is a particularly
          expensive feature in terms scanner performance; if it is used in any
          of the scanner's actions it will slow  down  all  of  the  scanner's
          matching.   Furthermore,  REJECT  cannot  be  used with the -f or -F
          options.

          Note also that unlike the other special actions, REJECT is a branch;
          code immediately following it in the action will not be executed.

     -    yymore() tells the scanner that the next time it matches a rule, the
          corresponding  token  should  be  appended onto the current value of
          yytext rather than replacing it.

     -    yyless(n) returns all but the first  n  characters  of  the  current
          token  back  to  the input stream, where they will be rescanned when
          the scanner looks  for  the  next  match.   yytext  and  yyleng  are
          adjusted appropriately (e.g., yyleng will now be equal to n ).

     -    unput(c) puts the character c back onto the input stream.   It  will
          be the next character scanned.

     -    input() reads the next character from the input stream (this routine
          is called yyinput() if the scanner is compiled using C++).

     -    yyterminate() can be used in  lieu  of  a  return  statement  in  an
          action.   It terminates the scanner and returns a 0 to the scanner's
          caller, indicating "all done".

          By default, yyterminate() is also  called  when  an  end-of-file  is
          encountered.  It is a macro and may be redefined.

     -    YY_NEW_FILE is an action available only in <<EOF>> rules.  It  means
          "Okay, I've set up a new input file, continue scanning".

     -    yy_create_buffer( file, size ) takes a FILE pointer and  an  integer
          size.  It  returns  a  YY_BUFFER_STATE  handle to a new input buffer
          large enough to accomodate size characters and associated  with  the
          given file.  When in doubt, use YY_BUF_SIZE for the size.




                                 26 May 1990                                 6



FLEX(1)                   Minix Programmer's Manual                    FLEX(1)


     -    yy_switch_to_buffer( new_buffer ) switches the scanner's  processing
          to  scan  for  tokens  from  the  given  buffer,  which  must  be  a
          YY_BUFFER_STATE.

     -    yy_delete_buffer( buffer ) deletes the given buffer.

VALUES AVAILABLE TO THE USER

     -    char *yytext holds the text of the current token.   It  may  not  be
          modified.

     -    int yyleng holds the length of the current token.   It  may  not  be
          modified.

     -    FILE *yyin is the file which by default flex reads from.  It may  be
          redefined  but  doing  so  only  makes sense before scanning begins.
          Changing it in the middle of scanning will have  unexpected  results
          since  flex  buffers its input.  Once scanning terminates because an
          end-of-file has been seen, void yyrestart( FILE *new_file )  may  be
          called to point yyin at the new input file.

     -    FILE *yyout is the file to which ECHO actions are done.  It  can  be
          reassigned by the user.

     -    YY_CURRENT_BUFFER returns a YY_BUFFER_STATE handle  to  the  current
          buffer.

MACROS THE USER CAN REDEFINE

     -    YY_DECL controls how the scanning routine is declared.  By  default,
          it  is  "int  yylex()",  or,  if  prototypes  are  being  used, "int
          yylex(void)".  This definition may  be  changed  by  redefining  the
          "YY_DECL"  macro.   Note  that if you give arguments to the scanning
          routine using a K&R-style/non-prototyped function  declaration,  you
          must terminate the definition with a semi-colon (;).

     -    The nature of how the scanner gets its input can  be  controlled  by
          redefining  the  YY_INPUT  macro.   YY_INPUT's  calling  sequence is
          "YY_INPUT(buf,result,max_size)".  Its  action  is  to  place  up  to
          max_size  characters  in  the  character array buf and return in the
          integer variable result either the number of characters read or  the
          constant  YY_NULL  (0 on Unix systems) to indicate EOF.  The default
          YY_INPUT reads  from  the  global  file-pointer  "yyin".   A  sample
          redefinition  of  YY_INPUT  (in the definitions section of the input
          file):

              %{
              #undef YY_INPUT
              #define YY_INPUT(buf,result,max_size) \
                  { \


                                 26 May 1990                                 7



FLEX(1)                   Minix Programmer's Manual                    FLEX(1)


                  int c = getchar(); \
                  result = (c == EOF) ? YY_NULL : (buf[0] = c, 1); \
                  }
              %}


     -    When the scanner receives an end-of-file indication  from  YY_INPUT,
          it  then  checks  the  yywrap() function.  If yywrap() returns false
          (zero), then it is assumed that the function has gone ahead and  set
          up  yyin to point to another input file, and scanning continues.  If
          it returns true (non-zero), then the scanner terminates, returning 0
          to its caller.

          The default yywrap() always returns 1.  Presently,  to  redefine  it
          you  must first "#undef yywrap", as it is currently implemented as a
          macro.  It is likely that yywrap() will soon  be  defined  to  be  a
          function rather than a macro.

     -    YY_USER_ACTION can be redefined to provide an action which is always
          executed prior to the matched rule's action.

     -    The macro YY_USER_INIT may be redefined to provide an  action  which
          is always executed before the first scan.

     -    In the generated scanner, the actions are all gathered in one  large
          switch   statement  and  separated  using  YY_BREAK,  which  may  be
          redefined.  By default, it is simply a  "break",  to  separate  each
          rule's action from the following rule's.

FILES

     flex.skel
          skeleton scanner.

     lex.yy.c
          generated scanner (called lexyy.c on some systems).

     lex.backtrack
          backtracking  information  for  -b  flag  (called  lex.bck  on  some
          systems).

     -lfl library with which to link the scanners.

SEE ALSO

     flexdoc(1), lex(1), yacc(1), sed(1), awk(1).

     M. E. Lesk and E. Schmidt, LEX - Lexical Analyzer Generator




                                 26 May 1990                                 8



FLEX(1)                   Minix Programmer's Manual                    FLEX(1)


DIAGNOSTICS
     reject_used_but_not_detected undefined or

     yymore_used_but_not_detected  undefined  -  These  errors  can  occur  at
     compile time.  They indicate that the scanner uses REJECT or yymore() but
     that flex failed to notice the fact, meaning that flex scanned the  first
     two  sections looking for occurrences of these actions and failed to find
     any, but somehow you snuck some in (via a #include  file,  for  example).
     Make  an explicit reference to the action in your flex input file.  (Note
     that previously flex supported a %used/%unused mechanism for dealing with
     this  problem;  this  feature  is still supported but now deprecated, and
     will go away soon unless the author  hears  from  people  who  can  argue
     compellingly that they need it.)

     flex scanner jammed - a scanner compiled with -s has encountered an input
     string which wasn't matched by any of its rules.

     flex input buffer overflowed - a  scanner  rule  matched  a  string  long
     enough  to  overflow  the  scanner's  internal  input buffer (16K bytes -
     controlled by YY_BUF_MAX in "flex.skel").

     scanner  requires  -8  flag  -  Your   scanner   specification   includes
     recognizing  8-bit  characters  and  you did not specify the -8 flag (and
     your site has not installed flex with -8 as the default).

     fatal flex scanner internal error--end of buffer missed - This can  occur
     in  an  scanner  which  is reentered after a long-jump has jumped out (or
     over) the scanner's activation frame.   Before  reentering  the  scanner,
     use:

         yyrestart( yyin );


     too many %t classes! - You managed to put every single character into its
     own  %t  class.   flex  requires  that  at least one of the classes share
     characters.

AUTHOR
     Vern Paxson, with the help of many ideas and much  inspiration  from  Van
     Jacobson.  Original version by Jef Poskanzer.

     See flexdoc(1) for additional credits and the address  to  send  comments
     to.

DEFICIENCIES / BUGS

     Some trailing context patterns cannot be properly  matched  and  generate
     warning  messages  ("Dangerous  trailing  context").   These are patterns
     where the ending of the first part of the rule matches the  beginning  of
     the second part, such as "zx*/xy*", where the 'x*' matches the 'x' at the


                                 26 May 1990                                 9



FLEX(1)                   Minix Programmer's Manual                    FLEX(1)


     beginning of the trailing context.  (Note that  the  POSIX  draft  states
     that the text matched by such patterns is undefined.)

     For some trailing context rules, parts which  are  actually  fixed-length
     are  not  recognized  as  such, leading to the abovementioned performance
     loss.  In particular, parts using '|'  or  {n}  (such  as  "foo{3}")  are
     always considered variable-length.

     Combining trailing context with the special  '|'  action  can  result  in
     fixed  trailing  context  being  turned  into the more expensive variable
     trailing context.  For example, this happens in the following example:

         %%
         abc      |
         xyz/def


     Use of unput() invalidates yytext and yyleng.

     Use of unput() to push back more text than was matched can result in  the
     pushed-back  text  matching a beginning-of-line ('^') rule even though it
     didn't come at the beginning of the line (though this is rare!).

     Pattern-matching of NUL's is substantially  slower  than  matching  other
     characters.

     flex does not generate correct #line directives for code internal to  the
     scanner; thus, bugs in flex.skel yield bogus line numbers.

     Due to both buffering of input and read-ahead, you cannot intermix  calls
     to  <stdio.h>  routines, such as, for example, getchar(), with flex rules
     and expect it to work.  Call input() instead.

     The total table entries listed by the -v  flag  excludes  the  number  of
     table entries needed to determine what rule has been matched.  The number
     of entries is equal to the number of DFA states if the scanner  does  not
     use REJECT, and somewhat greater than the number of states if it does.

     REJECT cannot be used with the -f or -F options.

     Some of the macros, such as yywrap(), may in the future become  functions
     which live in the -lfl library.  This will doubtless break a lot of code,
     but may be required for POSIX-compliance.

     The flex internal algorithms need documentation.







                                26 May 1990                                 10