changes_from_1.31

来自「本工具提供一个词法分析器和语法分析器的集成开发环境」· 31 代码 · 共 523 行 · 第 1/2 页

CHANGES FROM 1.31

This file contains the migration of PCCTS from 1.31 in the order that
changes were made.  1.32b7 is the last beta before full 1.32.
Terence Parr, Parr Research Corporation 1995.


======================================================================
1.32b1
Added Russell Quong to banner, changed banner for output slightly
Fixed it so that you have before / after actions for C++ in class def
Fixed bug in optimizer that made it sometimes forget to set internal
        token pointers.  Only showed up when a {...} was in the "wrong spot".

======================================================================
1.32b2
Added fixes by Dave Seidel for PC compilers in 32 bit mode (config.h
and set.h).

======================================================================
1.32b3
Fixed hideous bug in code generator for wildcard and for ~token op.

from Dave Seidel

   Added pcnames.bat
   1. in antlr/main.c: change strcasecmp() to stricmp()

   2. in dlg/output.c: use DLEXER_C instead on "DLexer.C"

   3. in h/PBlackBox.h: use <iostream.h> instead of <stream.h>

======================================================================
1.32b4
When the -ft option was used, any path prefix screwed up
the gate on the .h files

Fixed yet another bug due to the optimizer.

The exception handling thing was a bit wacko:

a : ( A B )? A B
  | A C
  ;
  exception ...

caused an exception if "A C" was the input.  In other words,
it found that A C didn't match the (A B)? pred and caused
an exception rather than trying the next alt.  All I did
was to change the zzmatch_wsig() macros.

Fixed some problems in gen.c relating to the name of token
class bit sets in the output.

Added the tremendously cool generalized predicate.  For the
moment, I'll give this bried description.

a : <<predicate>>? blah
  | foo
  ;

This implies that (assuming blah and foo are syntactically
ambiguous) "predicate" indicates the semantic validity of
applying "blah".  If "predicate" is false, "foo" is attempted.

Previously, you had to say:

a : <<LA(1)==ID ? predicate : 1>>? ID
  | ID
  ;

Now, you can simply use "predicate" without the ?: operator
if you turn on ANTLR command line option: "-prc on".  This
tells ANTLR to compute that all by itself.  It computes n
tokens of lookahead where LT(n) or LATEXT(n) is the farthest
ahead you look.

If you give a predicate using "-prc on" that is followed
by a construct that can recognize more than one n-sequence,
you will get a warning from ANTLR.  For example,

a : <<isTypeName(LT(1)->getText())>>? (ID|INT)
  ;

This is wrong because the predicate will be applied to INTs
as well as ID's.  You should use this syntax to make
the predicate more specific:

a : (ID)? => <<isTypeName(LT(1)->getText())>>? (ID|INT)
  ;

which says "don't apply the predicate unless ID is the
current lookahead context".

You cannot currently have anything in the "(context)? =>"
except sequences such as:

( LPAREN ID | LPAREN SCOPE )? => <<pred>>?

I haven't tested this THAT much, but it does work for the
C++ grammar.

======================================================================
1.32b5

Added getLine() to the ANTLRTokenBase and DLGBasedToken classes
left line() for backward compatibility.
----
Removed SORCERER_TRANSFORM from the ast.h stuff.
-------
Fixed bug in code gen of ANTLR such that nested syn preds work more
efficiently now.  The ANTLRTokenBuffer was getting very large
with nested predicates.
------
Memory leak is now gone from ANTLRTokenBuf; all tokens are deleted.
For backward compatibility reasons, you have to say parser->deleteTokens()
or mytokenbuffer->deleteTokens() but later it will be the default mode.
Say this after the parser is constructed. E.g.,

    ParserBlackBox<DLGLexer, MyParser, ANTLRToken> p(stdin);
    p.parser()->deleteTokens();
    p.parser()->start_symbol();


==============================
1.32b6

Changed so that deleteTokens() will do a delete ((ANTLRTokenBase *))
on the ptr.  This gets the virtual destructor.

Fixed some weird things in the C++ header files (a few return types).

Made the AST routines correspond to the book and SORCERER stuff.

New token stuff:  See testcpp/14/test.g

ANTLR accepts a #pragma gc_tokens which says
[1]     Generate label = copy(LT(1)) instead of label=LT(1) for
        all labeled token references.
[2]     User now has to define ANTLRTokenPtr (as a class or a typedef
        to just a pointer) as well as the ANTLRToken class itself.
		See the example.

To delete tokens in token buffer, use deleteTokens() message on parser.

        All tokens that fall off the ANTLRTokenBuffer get deleted
        which is what currently happens when deleteTokens() message
        has been sent to token buffer.

We always generate ANTLRTokenPtr instead of 'ANTLRToken *' now.
Then if no pragma set, ANTLR generates a

        class ANTLRToken;
        typedef ANTLRToken *ANTLRTokenPtr;

in each file.

Made a warning for x:rule_ref <<$x>>; still no warning for $i's, however.
class BB {

a : x:b y:A <<$x
$y>>
  ;

b : B;

}
generates
Antlr parser generator   Version 1.32b6   1989-1995
test.g, line 3: error: There are no token ptrs for rule references: '$x'

===================
1.32b7:

[With respect to token object garbage collection (GC), 1.32b7
 backtracks from 1.32b6, but results in better and less intrusive GC.
 This is the last beta version before full 1.32.]

BIGGEST CHANGES:

o	The "#pragma gc_tokens" is no longer used.

o	.C files are now .cpp files (hence, makefiles will have to
	be changed; or you can rerun genmk).  This is a good move,
	but causes some backward incompatibility problems.  You can
	avoid this by changing CPP_FILE_SUFFIX to ".C" in pccts/h/config.h.

o	The token object class hierarchy has been flattened to include
	only three classes: ANTLRAbstractToken, ANTLRCommonToken, and
	ANTLRCommonNoRefCountToken.  The common token now does garbage
	collection via ref counting.

o	"Smart" pointers are now used for garbage collection.  That is,
	ANTLRTokenPtr is used instead of "ANTLRToken *".

o	The antlr.1 man page has been cleaned up slightly.

o	The SUN C++ compiler now complains less about C++ support code.

o	Grammars which subclass ANTLRCommonToken must wrap all token
	pointer references in mytoken(token_ptr).  This is the only
	serious backward incompatibility.  See below.


MINOR CHANGES:

--------------------------------------------------------
1	deleteTokens()

The deleteTokens() message to the parser or token buffer has been changed
to one of:

    void noGarbageCollectTokens()   { inputTokens->noGarbageCollectTokens(); }
    void garbageCollectTokens()     { inputTokens->garbageCollectTokens(); }

The token buffer deletes all non-referenced tokens by default now.

--------------------------------------------------------
2	makeToken()

The makeToken() message returns a new type.  The function should look
like:

    virtual ANTLRAbstractToken *makeToken(ANTLRTokenType tt,
                                          ANTLRChar *txt,
                                          int line)
    {
        ANTLRAbstractToken *t = new ANTLRCommonToken(tt,txt);
        t->setLine(line);
        return t;
    }

--------------------------------------------------------
3	TokenType

Changed TokenType-> ANTLRTokenType  (often forces changes in AST defs due
to #[] constructor called to AST(tokentype, string)).

--------------------------------------------------------
4	AST()

You must define AST(ANTLRTokenPtr t) now in your AST class definition.
You might also have to include ATokPtr.h above the definition; e.g.,
if AST is defined in a separate file, such as AST.h, it's a good idea
to include ATOKPTR_H (ATokPtr.h).  For example,

	#include ATOKPTR_H
	class AST : public ASTBase {
	protected:
	    ANTLRTokenPtr token;
	public:
	    AST(ANTLRTokenPtr t) { token = t; }
	    void preorder_action() {
	        char *s = token->getText();
	        printf(" %s", s);
	    }
	};

Note the use of smart pointers rather than "ANTLRToken *".

--------------------------------------------------------
5	SUN C++