news

Flex词法/语法分析器源码

	- The -c flag now has its full glorious POSIX interpretation (do
	  nothing), rather than being interpreted as an old-style -C flag.

	- Scanners generated by flex now include two #define's giving
	  the major and minor version numbers (YY_FLEX_MAJOR_VERSION,
	  YY_FLEX_MINOR_VERSION).  These can then be tested to see
	  whether certain flex features are available.

	- Scanners generated using -l lex compatibility now have the symbol
	  YY_FLEX_LEX_COMPAT #define'd.

	- When initializing (i.e., yy_init is non-zero on entry to yylex()),
	  generated scanners now set yy_init to zero before executing
	  YY_USER_INIT.  This means that you can set yy_init back to a
	  non-zero value in YY_USER_INIT if you need the scanner to be
	  reinitialized on the next call.

	- You can now use "#line" directives in the first section of your
	  scanner specification.

	- When generating full-table scanners (-Cf), flex now puts braces
	  around each row of the 2-d array initialization, to silence warnings
	  on over-zealous compilers.

	- Improved support for MS-DOS.  The flex sources have been successfully
	  built, unmodified, for Borland 4.02 (all that's required is a
	  Borland Makefile and config.h file, which are supplied in
	  MISC/Borland - contributed by Terrence O Kane).

	- Improved support for Macintosh using Think C - the sources should
	  build for this platform "out of the box".  Contributed by Scott
	  Hofmann.

	- Improved support for VMS, in MISC/VMS/, contributed by Pat Rankin.

	- Support for the Amiga, in MISC/Amiga/, contributed by Andreas
	  Scherer.  Note that the contributed files were developed for
	  flex 2.4 and have not been tested with flex 2.5.

	- Some notes on support for the NeXT, in MISC/NeXT, contributed
	  by Raf Schietekat.

	- The MISC/ directory now includes a preformatted version of flex.1
	  in flex.man, and pre-yacc'd versions of parse.y in parse.{c,h}.

	- The flex.1 and flexdoc.1 manual pages have been merged.  There
	  is now just one document, flex.1, which includes an overview
	  at the beginning to help you find the section you need.

	- Documentation now clarifies that start conditions persist across
	  switches to new input files or different input buffers.  If you
	  want to e.g., return to INITIAL, you must explicitly do so.

	- The "Performance Considerations" section of the manual has been
	  updated.

	- Documented the "yy_act" variable, which when YY_USER_ACTION is
	  invoked holds the number of the matched rule, and added an
	  example of using yy_act to profile how often each rule is matched.

	- Added YY_NUM_RULES, a definition that gives the total number
	  of rules in the file, including the default rule (even if you
	  use -s).

	- Documentation now clarifies that you can pass a nil FILE* pointer
	  to yy_create_buffer() or yyrestart() if you've arrange YY_INPUT
	  to not need yyin.

	- Documentation now clarifies that YY_BUFFER_STATE is a pointer to
	  an opaque "struct yy_buffer_state".

	- Documentation now stresses that you gain the benefits of removing
	  backing-up states only if you remove *all* of them.

	- Documentation now points out that traditional lex allows you
	  to put the action on a separate line from the rule pattern if
	  the pattern has trailing whitespace (ugh!), but flex doesn't
	  support this.

	- A broken example in documentation of the difference between
	  inclusive and exclusive start conditions is now fixed.

	- Usage (-h) report now goes to stdout.

	- Version (-V) info now goes to stdout.

	- More #ifdef chud has been added to the parser in attempt to
	  deal with bison's use of alloca().

	- "make clean" no longer deletes emacs backup files (*~).

	- Some memory leaks have been fixed.

	- A bug was fixed in which dynamically-expanded buffers were
	  reallocated a couple of bytes too small.

	- A bug was fixed which could cause flex to read and write beyond
	  the end of the input buffer.

	- -S will not be going away.


Changes between release 2.4.7 (03Aug94) and release 2.4.6:

	- Fixed serious bug in reading multiple files.

	- Fixed bug in scanning NUL's.

	- Fixed bug in input() returning 8-bit characters.

	- Fixed bug in matching text with embedded NUL's when
	  using %array or lex compatibility.

	- Fixed multiple invocations of YY_USER_ACTION when using '|'
	  continuation action.

	- Minor prototyping fixes.

Changes between release 2.4.6 (04Jan94) and release 2.4.5:

	- Linking with -lfl no longer required if your program includes
	  its own yywrap() and main() functions.  (This change will cause
	  problems if you have a non-ANSI compiler on a system for which
	  sizeof(int) != sizeof(void*) or sizeof(int) != sizeof(size_t).)

	- The use of 'extern "C++"' in FlexLexer.h has been modified to
	  get around an incompatibility with g++'s header files.

Changes between release 2.4.5 (11Dec93) and release 2.4.4:

	- Fixed bug breaking C++ scanners that use REJECT or variable
	  trailing context.

	- Fixed serious input problem for interactive scanners on
	  systems for which char is unsigned.

	- Fixed bug in incorrectly treating '$' operator as variable
	  trailing context.

	- Fixed bug in -CF table representation that could lead to
	  corrupt tables.

	- Fixed fairly benign memory leak.

	- Added `extern "C++"' wrapper to FlexLexer.h header.  This
	  should overcome the g++ 2.5.X problems mentioned in the
	  NEWS for release 2.4.3.

	- Changed #include of FlexLexer.h to use <> instead of "".

	- Added feature to control whether the scanner attempts to
	  refill the input buffer once it's exhausted.  This feature
	  will be documented in the 2.5 release.


Changes between release 2.4.4 (07Dec93) and release 2.4.3:

	- Fixed two serious bugs in scanning 8-bit characters.

	- Fixed bug in YY_USER_ACTION that caused it to be executed
	  inappropriately (on the scanner's own internal actions, and
	  with incorrect yytext/yyleng values).

	- Fixed bug in pointing yyin at a new file and resuming scanning.

	- Portability fix regarding min/max/abs macros conflicting with
	  function definitions in standard header files.

	- Added a virtual LexerError() method to the C++ yyFlexLexer class
	  for reporting error messages instead of always using cerr.

	- Added warning in flexdoc that the C++ scanning class is presently
	  experimental and subject to considerable change between major
	  releases.


Changes between release 2.4.3 (03Dec93) and release 2.4.2:

	- Fixed bug causing fatal scanner messages to fail to print.

	- Fixed things so FlexLexer.h can be included in other C++
	  sources.  One side-effect of this change is that -+ and -CF
	  are now incompatible.

	- libfl.a now supplies private versions of the the <string.h>/
	  <strings.h> string routines needed by flex and the scanners
	  it generates, to enhance portability to some BSD systems.

	- More robust solution to 2.4.2's flexfatal() bug fix.

	- Added ranlib of installed libfl.a.

	- Some lint tweaks.

	- NOTE: problems have been encountered attempting to build flex
	  C++ scanners using g++ version 2.5.X.  The problem is due to an
	  unfortunate heuristic in g++ 2.5.X that attempts to discern between
	  C and C++ headers.  Because FlexLexer.h is installed (by default)
	  in /usr/local/include and not /usr/local/lib/g++-include, g++ 2.5.X
	  decides that it's a C header :-(.  So if you have problems, install
	  the header in /usr/local/lib/g++-include instead.


Changes between release 2.4.2 (01Dec93) and release 2.4.1:

	- Fixed bug in libfl.a referring to non-existent "flexfatal" function.

	- Modified to produce both compress'd and gzip'd tar files for
	  distributions (you probably don't care about this change!).


Changes between release 2.4.1 (30Nov93) and release 2.3.8:

	- The new '-+' flag instructs flex to generate a C++ scanner class
	  (thanks to Kent Williams).  flex writes an implementation of the
	  class defined in FlexLexer.h to lex.yy.cc.  You may include
	  multiple scanner classes in your program using the -P flag.  Note
	  that the scanner class also provides a mechanism for creating
	  reentrant scanners.  The scanner class uses C++ streams for I/O
	  instead of FILE*'s (thanks to Tom Epperly).  If the flex executable's
	  name ends in '+' then the '-+' flag is automatically on, so creating
	  a symlink or copy of "flex" to "flex++" results in a version of
	  flex that can be used exclusively for C++ scanners.

	  Note that without the '-+' flag, flex-generated scanners can still
	  be compiled using C++ compilers, though they use FILE*'s for I/O
	  instead of streams.

	  See the "GENERATING C++ SCANNERS" section of flexdoc for details.

	- The new '-l' flag turns on maximum AT&T lex compatibility.  In
	  particular, -l includes support for "yylineno" and makes yytext
	  be an array instead of a pointer.  It does not, however, do away
	  with all incompatibilities.  See the "INCOMPATIBILITIES WITH LEX
	  AND POSIX" section of flexdoc for details.

	- The new '-P' option specifies a prefix to use other than "yy"
	  for the scanner's globally-visible variables, and for the
	  "lex.yy.c" filename.  Using -P you can link together multiple
	  flex scanners in the same executable.

	- The distribution includes a "texinfo" version of flexdoc.1,
	  contributed by Roland Pesch (thanks also to Marq Kole, who
	  contributed another version).  It has not been brought up to
	  date, but reflects version 2.3.  See MISC/flex.texinfo.

	  The flex distribution will soon include G.T. Nicol's flex
	  manual; he is presently bringing it up-to-date for version 2.4.

	- yywrap() is now a function, and you now *must* link flex scanners
	  with libfl.a.

	- Site-configuration is now done via an autoconf-generated
	  "configure" script contributed by Francois Pinard.

	- Scanners now use fread() (or getc(), if interactive) and not
	  read() for input.  A new "table compression" option, -Cr,
	  overrides this change and causes the scanner to use read()
	  (because read() is a bit faster than fread()).  -f and -F
	  are now equivalent to -Cfr and -CFr; i.e., they imply the
	  -Cr option.

	- In the blessed name of POSIX compliance, flex supports "%array"
	  and "%pointer" directives in the definitions (first) section of
	  the scanner specification.  The former specifies that yytext
	  should be an array (of size YYLMAX), the latter, that it should
	  be a pointer.  The array version of yytext is universally slower
	  than the pointer version, but has the advantage that its contents
	  remain unmodified across calls to input() and unput() (the pointer
	  version of yytext is, still, trashed by such calls).

	  "%array" cannot be used with the '-+' C++ scanner class option.

	- The new '-Ca' option directs flex to trade off memory for
	  natural alignment when generating a scanner's tables.  In
	  particular, table entries that would otherwise be "short"
	  become "long".

	- The new '-h' option produces a summary of the flex flags.

	- The new '-V' option reports the flex version number and exits.

	- The new scanner macro YY_START returns an integer value
	  corresponding to the current start condition.  You can return
	  to that start condition by passing the value to a subsequent
	  "BEGIN" action.  You also can implement "start condition stacks"
	  by storing the values in an integer stack.

	- You can now redefine macros such as YY_INPUT by just #define'ing
	  them to some other value in the first section of the flex input;
	  no need to first #undef them.

	- flex now generates warnings for rules that can't be matched.
	  These warnings can be turned off using the new '-w' flag.  If
	  your scanner uses REJECT then you will not get these warnings.

	- If you specify the '-s' flag but the default rule can be matched,
	  flex now generates a warning.

	- "yyleng" is now a global, and may be modified by the user (though
	  doing so and then using yymore() will yield weird results).

	- Name definitions in the first section of a scanner specification
	  can now include a leading '^' or trailing '$' operator.  In this
	  case, the definition is *not* pushed back inside of parentheses.

	- Scanners with compressed tables are now "interactive" (-I option)
	  by default.  You can suppress this attribute (which makes them
	  run slightly slower) using the new '-B' flag.

	- Flex now generates 8-bit scanners by default, unless you use the
	  -Cf or -CF compression options (-Cfe  and -CFe result in 8-bit
	  scanners).  You can force it to generate a 7-bit scanner using
	  the new '-7' flag.  You can build flex to generate 8-bit scanners
	  for -Cf and -CF, too, by adding -DDEFAULT_CSIZE=256 to CFLAGS
	  in the Makefile.

	- You no longer need to call the scanner routine yyrestart() to
	  inform the scanner that you have switched to a new file after
	  having seen an EOF on the current input file.  Instead, just
	  point yyin at the new file and continue scanning.

	- You no longer need to invoke YY_NEW_FILE in an <<EOF>> action
	  to indicate you wish to continue scanning.  Simply point yyin
	  at a new file.

	- A leading '#' no longer introduces a comment in a flex input.

	- flex no longer considers formfeed ('\f') a whitespace character.

	- %t, I'm happy to report, has been nuked.

	- The '-p' option may be given twice ('-pp') to instruct flex to
	  report minor performance problems as well as major ones.

	- The '-v' verbose output no longer includes start/finish time
	  information.

	- Newlines in flex inputs can optionally include leading or
	  trailing carriage-returns ('\r'), in support of several PC/Mac
	  run-time libraries that automatically include these.

	- A start condition of the form "<*>" makes the following rule
	  active in every start condition, whether exclusive or inclusive.

	- The following items have been corrected in the flex documentation:

		- '-C' table compression options *are* cumulative.

		- You may modify yytext but not lengthen it by appending
		  characters to the end.  Modifying its final character
		  will affect '^' anchoring for the next rule matched
		  if the character is changed to or from a newline.

		- The term "backtracking" has been renamed "backing up",
		  since it is a one-time repositioning and not a repeated
		  search.  What used to be the "lex.backtrack" file is now
		  "lex.backup".

		- Unindented "/* ... */" comments are allowed in the first
		  flex input section, but not in the second.

		- yyless() can only be used in the flex input source, not
		  externally.

		- You can use "yyrestart(yyin)" to throw away the
		  current contents of the input buffer.

		- To write high-speed scanners, attempt to match as much
		  text as possible with each rule.  See MISC/fastwc/README
		  for more information.

		- Using the beginning-of-line operator ('^') is fairly
		  cheap.  Using unput() is expensive.  Using yyless() is
		  cheap.

		- An example of scanning strings with embedded escape
		  sequences has been added.

		- The example of backing-up in flexdoc was erroneous; it
		  has been corrected.

	- A flex scanner's internal buffer now dynamically grows if needed
	  to match large tokens.  Note that growing the buffer presently
	  requires rescanning the (large) token, so consuming a lot of
	  text this way is a slow process.  Also note that presently the
	  buffer does *not* grow if you unput() more text than can fit
	  into the buffer.

	- The MISC/ directory has been reorganized; see MISC/README for
	  details.

	- yyless() can now be used in the third (user action) section
	  of a scanner specification, thanks to Ceriel Jacobs.  yyless()
	  remains a macro and cannot be used outside of the scanner source.

	- The skeleton file is no longer opened at run-time, but instead
	  compiled into a large string array (thanks to John Gilmore and
	  friends at Cygnus).  You can still use the -S flag to point flex
	  at a different skeleton file.

	- flex no longer uses a temporary file to store the scanner's