changes_from_1.31

来自「本工具提供一个词法分析器和语法分析器的集成开发环境」· 31 代码 · 共 523 行 · 第 1/2 页

From [e-mail] Bob Bailey. Changed ANTLR C++ output
to avoid an error in Sun C++ 3.0.1.  Made "public" return value
structs created to hold multiple return values public.

--------------------------------------------------------
6	genmk

Fixed genmk so that target List.* is not included anymore.  It's
called SList.* anyway.

--------------------------------------------------------
7	\r vs \n

Scott Vorthmann <[e-mail]> fixed antlr.g in ANTLR so that \r
is allowed as the return character as well as \n.

--------------------------------------------------------
8	Exceptions

Bug in exceptions attached to labeled token/tokclass references.  Didn't gen
code for exceptions.  This didn't work:

a : "help" x:ID
  ;
        exception[x]
        catch MismatchedToken : <<printf("eh?\n");>>

Now ANTLR generates (which is kinda big, but necessary):

        if ( !_match_wsig(ID) ) {
                if ( guessing ) goto fail;
                _signal=MismatchedToken;
                switch ( _signal ) {
                case MismatchedToken :
                        printf("eh?\n");
                        _signal = NoSignal;
                        break;
                default :
                        goto _handler;
                }
        }

which implies that you can recover and continue parsing after a missing/bad
token reference.

--------------------------------------------------------
9	genmk

genmk now correctly uses config file for CPP_FILE_SUFFIX stuff.

--------------------------------------------------------
10	general cleanup / PURIFY

Anthony Green <[e-mail]> suggested a bunch of good general
clean up things for the code; he also suggested a few things to
help out the "PURIFY" memory allocation checker.

--------------------------------------------------------
11	$-variable references.

Manuel ORNATO indicated that a $-variable outside of a rule caused
ANTLR to crash.  I fixed this.

12	Tom Moog suggestion

Fail action of semantic predicate needs "{}" envelope.  FIXED.

13	references to LT(1).

I have enclosed all assignments such as:

             _t22 = (ANTLRTokenPtr)LT(1);

in "if ( !guessing )" so that during backtracking the reference count
for token objects is not increased.


TOKEN OBJECT GARBAGE COLLECTION

1	INTRODUCTION

The class ANTLRCommonToken is now garbaged collected through a "smart"
pointer called ANTLRTokenPtr using reference counting.  Any token
object not referenced by your grammar actions is destroyed by the
ANTLRTokenBuffer when it must make room for more token objects.
Referenced tokens are then destroyed in your parser when local
ANTLRTokenPtr objects are deleted.  For example,

a : label:ID ;

would be converted to something like:

void yourclass::a(void)
{
	zzRULE;
	ANTLRTokenPtr label=NULL;	// used to be ANTLRToken *label;
        zzmatch(ID);
        label = (ANTLRTokenPtr)LT(1);
	consume();
	...
}

When the "label" object is destroyed (it's just a pointer to your
input token object LT(1)), it decrements the reference count on the
object created for the ID.  If the count goes to zero, the object
pointed by label is deleted.

To correctly manage the garbage collection, you should use
ANTLRTokenPtr instead of "ANTLRToken *".  Most ANTLR support code
(visible to the user) has been modified to use the smart pointers.

***************************************************************
Remember that any local objects that you create are not deleted when a
lonjmp() is executed.  Unfortunately, the syntactic predicates (...)?
use setjmp()/longjmp().  There are some situations when a few tokens
will "leak".
***************************************************************

2	DETAILS

o	The default is to perform token object garbage collection.
	You may use parser->noGarbageCollectTokens() to turn off
	garbage collection.


o	The type ANTLRTokenPtr is always defined now (automatically).
	If you do not wish to use smart pointers, you will have to
	redefined ANTLRTokenPtr by subclassing, changing the header
	file or changing ANTLR's code generation (easy enough to
	do in gen.c).

o	If you don't use ParserBlackBox, the new initialization sequence is:

	    ANTLRTokenPtr aToken = new ANTLRToken;
	    scan.setToken(mytoken(aToken));

	where mytoken(aToken) gets an ANTLRToken * from the smart pointer.

o	Define C++ preprocessor symbol DBG_REFCOUNTTOKEN to see a bunch of
	debugging stuff for reference counting if you suspect something.


3	WHY DO I HAVE TO TYPECAST ALL MY TOKEN POINTERS NOW??????

If you subclass ANTLRCommonToken and then attempt to refer to one of
your token members via a token pointer in your grammar actions, the
C++ compiler will complain that your token object does not have that
member.  For example, if you used to do this

<<
class ANTLRToken : public ANTLRCommonToken {
        int muck;
	...
};
>>

class Foo {
a : t:ID << t->muck = ...; >> ;
}

Now, you must do change the t->muck reference to:

a : t:ID << mytoken(t)->muck = ...; >> ;

in order to downcast 't' to be an "ANTLRToken *" not the
"ANTLRAbstractToken *" resulting from ANTLRTokenPtr::operator->().
The macro is defined as:

/*
 * Since you cannot redefine operator->() to return one of the user's
 * token object types, we must down cast.  This is a drag.  Here's
 * a macro that helps.  template: "mytoken(a-smart-ptr)->myfield".
 */
#define mytoken(tp) ((ANTLRToken *)(tp.operator->()))

You have to use macro mytoken(grammar-label) now because smart
pointers are not specific to a parser's token objects.  In other
words, the ANTLRTokenPtr class has a pointer to a generic
ANTLRAbstractToken not your ANTLRToken; the ANTLR support code must
use smart pointers too, but be able to work with any kind of
ANTLRToken.  Sorry about this, but it's C++'s fault not mine.  Some
nebulous future version of the C++ compilers should obviate the need
to downcast smart pointers with runtime type checking (and by allowing
different return type of overridden functions).

A way to have backward compatible code is to shut off the token object
garbage collection; i.e., use parser->noGarbageCollectTokens() and
change the definition of ANTLRTokenPtr (that's why you get source code
<wink>).


PARSER EXCEPTION HANDLING

I've noticed some weird stuff with the exception handling.  I intend
to give this top priority for the "book release" of ANTLR.

==========
1.32 Full Release

o	Changed Token class hierarchy to be (Thanks to Tom Moog):

        ANTLRAbstractToken
          ANTLRRefCountToken
             ANTLRCommonToken
          ANTLRNoRefCountCommonToken

o	Added virtual panic() to ANTLRAbstractToken.  Made ANTLRParser::panic()
	virtual also.

o	Cleaned up the dup() stuff in AST hierarchy to use shallowCopy() to
	make node copies.  John Farr at Medtronic suggested this.  I.e.,
	if you want to use dup() with either ANTLR or SORCERER or -transform
	mode with SORCERER, you must defined shallowCopy() as:

	virtual PCCTS_AST *shallowCopy()
	{
	    return new AST;
	    p->setDown(NULL);
	    p->setRight(NULL);
	    return p;
	}

	or

	virtual PCCTS_AST *shallowCopy()
	{
	    return new AST(*this);
	}
	
	if you have defined a copy constructor such as

	AST(const AST &t)	// shallow copy constructor
	{
		token = t.token;
		iconst = t.iconst;
		setDown(NULL);
		setRight(NULL);
	}

o	Added a warning with -CC and -gk are used together.  This is broken,
	hence a warning is appropriate.

o	Added warning when #-stuff is used w/o -gt option.

o	Updated MPW installation.

o	"Miller, Philip W." <[e-mail]> suggested
	that genmk be use RENAME_OBJ_FLAG RENAME_EXE_FLAG instead of
	hardcoding "-o" in genmk.c.

o	made all exit() calls use EXIT_SUCCESS or EXIT_FAILURE.

===========================================================================
1.33

EXIT_FAILURE and EXIT_SUCCESS were not always defined.  I had to modify
a bunch of files to use PCCTS_EXIT_XXX, which forces a new version.  Sorry
about that.