changes_from_1.33

本工具提供一个词法分析器和语法分析器的集成开发环境

             resolving power for some lookahead sequences.
    -------------------------------------------------------------------------

      If you use the "-info p" option the output file will contain:

    +----------------------------------------------------------------------+
    |#if 0                                                                 |
    |                                                                      |
    |The following predicates are identical when compared without          |
    |  lookahead context information.  For some ambiguous lookahead        |
    |  sequences they may not have any power to resolve the ambiguity.     |
    |                                                                      |
    |Choice 1: rule0/1  alt 1  line 5  file t10.g                          |
    |                                                                      |
    |  The original predicate for choice 1 with available context          |
    |    information:                                                      |
    |                                                                      |
    |    OR expr                                                           |
    |                                                                      |
    |      pred  <<  Upper>>?                                              |
    |                        depth=k=1  rule rule2  line 14  t10.g         |
    |        set context:                                                  |
    |           ID                                                         |
    |                                                                      |
    |      pred  <<  Lower>>?                                              |
    |                        depth=k=1  rule rule3  line 15  t10.g         |
    |        set context:                                                  |
    |           ID                                                         |
    |                                                                      |
    |  The predicate for choice 1 after expansion (but without context     |
    |    information):                                                     |
    |                                                                      |
    |    OR expr                                                           |
    |                                                                      |
    |      pred  <<  isUpper(LATEXT(1))>>?                                 |
    |                        depth=k=1  rule   line 1  t10.g               |
    |                                                                      |
    |      pred  <<  isLower(LATEXT(1))>>?                                 |
    |                        depth=k=1  rule   line 2  t10.g               |
    |                                                                      |
    |                                                                      |
    |Choice 2: rule0/2  alt 2  line 6  file t10.g                          |
    |                                                                      |
    |  The original predicate for choice 2 with available context          |
    |    information:                                                      |
    |                                                                      |
    |  pred  <<  Alpha>>?                                                  |
    |                    depth=k=1  rule rule0  line 6  t10.g              |
    |    set context:                                                      |
    |       ID                                                             |
    |                                                                      |
    |  The predicate for choice 2 after expansion (but without context     |
    |    information):                                                     |
    |                                                                      |
    |  OR expr                                                             |
    |                                                                      |
    |    pred  <<  isUpper(LATEXT(1))>>?                                   |
    |                      depth=k=1  rule   line 1  t10.g                 |
    |                                                                      |
    |    pred  <<  isLower(LATEXT(1))>>?                                   |
    |                      depth=k=1  rule   line 2  t10.g                 |
    |                                                                      |
    |                                                                      |
    |#endif                                                                |
    +----------------------------------------------------------------------+

      The comparison of the predicates for the two alternatives takes
      place without context information, which means that in some cases
      the predicates will be considered identical even though they operate
      on disjoint lookahead sets.  Consider:

            #pred Alpha

            rule1: <<Alpha>>? ID
                 | <<Alpha>>? Label
                 ;

      Because the comparison of predicates takes place without context
      these will be considered identical.  The reason for comparing
      without context is that otherwise it would be necessary to re-evaluate
      the entire predicate expression for each possible lookahead sequence.
      This would require more code to be written and more CPU time during
      grammar analysis, and it is not yet clear whether anyone will even make
      use of the new #pred facility.

      A temporary workaround might be to use different #pred statements
      for predicates you know have different context.  This would avoid
      extraneous warnings.

      The above example might be termed a "false positive".  Comparison
      without context will also lead to "false negatives".  Consider the
      following example:

            #pred Alpha
            #pred Beta

            rule1: <<Alpha>>? A
                 | rule2
                 ;

            rule2: <<Alpha>>? A
                 | <<Beta>>?  B
                 ;

      The predicate used for alt 2 of rule1 is (Alpha || Beta).  This
      appears to be different than the predicate Alpha used for alt1.
      However, the context of Beta is B.  Thus when the lookahead is A
      Beta will have no resolving power and Alpha will be used for both
      alternatives.  Using the same predicate for both alternatives isn't
      very helpful, but this will not be detected with 1.33MR11.

      To properly handle this the predicate expression would have to be
      evaluated for each distinct lookahead context.

      To determine whether two predicate expressions are identical is
      difficult.  The routine may fail to identify identical predicates.

      The #pred feature also compares predicates to see if a choice between
      alternatives which is resolved by a predicate which makes the second
      choice unreachable.  Consider the following example:

            #pred A         <<A(LATEXT(1)>>?
            #pred B         <<B(LATEXT(1)>>?
            #pred A_or_B    A || B

            r   : s
                | t
                ;
            s   : <<A_or_B>>? ID
                ;
            t   : <<A>>? ID
                ;

        ----------------------------------------------------------------------------
        t11.g, line 5: warning: the predicate used to disambiguate the
               first choice of  rule r
             (file t11.g alt 1 line 5 and alt 2 line 6)
             appears to "cover" the second predicate when compared without context.
             The second predicate may have no resolving power for some lookahead
               sequences.
        ----------------------------------------------------------------------------

#139. (Changed in MR11) Problem with -gp in C++ mode

      The -gp option to add a prefix to rule names did not work in
      C++ mode.  This has been fixed.

      Reported by Alexey Demakov ([e-mail]).

#138. (Changed in MR11) Additional makefiles for non-MSVC++ MS systems

      Sramji Ramanathan ([e-mail]) has supplied makefiles for
      building antlr and dlg with Win95/NT development tools that
      are not based on MSVC5.  They are pccts/antlr/AntlrMS.mak and
      pccts/dlg/DlgMS.mak.

      The first line of the makefiles require a definition of PCCTS_HOME.

      These are in additiion to the AntlrMSVC50.* and DlgMSVC50.*
      supplied by Jeff Vincent ([e-mail]).

#137. (Changed in MR11) Token getType(), getText(), getLine() const members

      --------------------------------------------------------------------
      If you use ANTLRCommonToken this change probably does not affect you.
      --------------------------------------------------------------------

      For a long time it has bothered me that these accessor functions
      in ANTLRAbstractToken were not const member functions.  I have
      refrained from changing them because it require users to modify
      existing token class definitions which are derived directly
      from ANTLRAbstractToken.  I think it is now time.

      For those who are not used to C++, a "const member function" is a
      member function which does not modify its own object - the thing
      to which "this" points. This is quite different from a function
      which does not modify its arguments

      Most token definitions based on ANTLRAbstractToken have something like
      the following in order to create concrete definitions of the pure
      virtual methods in ANTLRAbstractToken:

        class MyToken : public ANTLRAbstractToken {
            ...
            ANTLRTokenType getType() {return _type; }
            int getLine()            {return _line; }
            ANTLRChar * getText()    {return _text; }
            ...
        }

      The required change is simply to put "const" following the function
      prototype in the header (.h file) and the definition file (.cpp if
      it is not inline):

        class MyToken : public ANTLRAbstractToken {
            ...
            ANTLRTokenType getType() const {return _type; }
            int getLine() const            {return _line; }
            ANTLRChar * getText() const    {return _text; }
            ...
        }

      This was originally proposed a long time ago by Bruce
      Guenter ([e-mail]).

#136. (Changed in MR11) Added getLength() to ANTLRCommonToken

      Classes ANTLRCommonToken and ANTLRCommonTokenNoRefCountToken
      now have a member function:

            int getLength() const { return strlen(getText()) }

      Suggested by Sramji Ramanathan ([e-mail]).

#135. (Changed in MR11) Raised antlr's own default ZZLEXBUFSIZE to 8k

#134a. (ansi_mr10.zip)  T.J. Parr's ANSI C grammar made 1.33MR11 compatible

       There is a typographical error in the definition of BITWISEOREQ:

        #token BITWISEOREQ "!=" should be "\|="

       When this change is combined with the bugfix to the follow set cache
       problem (Item #147) and a minor rearrangement of the grammar
       (Item #134b) it becomes a k=1 ck=2 grammar.

#134b. (ansi_mr10.zip)  T.J. Parr's ANSI C grammar made 1.33MR11 compatible

      The following changes were made in the ansi.g grammar (along with
      using -mrhoist on):

        ansi.g
        ======
        void tracein(char *)     ====>    void tracein(const char *)
        void traceout(char *)    ====>    void traceout(const char *)

        <LT(1)->getType()==IDENTIFIER ? isTypeName(LT(1)->getText()) : 1>>?
        ====> <<isTypeName(LT(1)->getText())>>?

        <<(LT(1)->getType()==LPARENTHESIS && LT(2)->getType()==IDENTIFIER) ? \
                        isTypeName(LT(2)->getText()) : 1>>?
        ====> (LPARENTHESIS IDENTIFIER)? => <<isTypeName(LT(2)->getText())>>?

        <<(LT(1)->getType()==LPARENTHESIS && LT(2)->getType()==IDENTIFIER) ? \
                        isTypeName(LT(2)->getText()) : 1>>?
        ====> (LPARENTHESIS IDENTIFIER)? => <<isTypeName(LT(2)->getText())>>?

        added to init(): traceOptionValueDefault=0;
        added to init(): traceOption(-1);

        change rule "statement":

            statement
                :   plain_label_statement
                |   case_label_statement
                |   <<;>> expression SEMICOLON
                |   compound_statement
                |   selection_statement
                |   iteration_statement
                |   jump_statement
                |   SEMICOLON
                ;

            plain_label_statement
                :   IDENTIFIER COLON statement
                ;

            case_label_statement
                :   CASE constant_expression COLON statement
                |   DEFAULT COLON statement
            ;

        support.cpp
        ===========
        void tracein(char *)     ====>    void tracein(const char *)
        void traceout(char *)    ====>    void traceout(const char *)

        added to tracein():  ANTLRParser::tracein(r);  // call superclass method
        added to traceout(): ANTLRParser::traceout(r); // call superclass method

        Makefile
        ========
        added to AFLAGS: -mrhoist on -prc on

#133. (Changed in 1.33MR11) Make trace options public in ANTLRParser

      In checking T.J. Parr's ANSI C grammar for compatibility with
      1.33MR11 discovered that it was inconvenient to have the
      trace facilities with protected access.

#132. (Changed in 1.33MR11) Recognition of identical predicates in alts

      Prior to 1.33MR11, there would be no ambiguity warning when the
      very same predicate was used to disambiguate both alternatives:

        test: ref B
            | ref C
            ;

        ref : <<pred(LATEXT(1)>>? A

      In 1.33MR11 this will cause the warning:

        warning: the predicates used to disambiguate rule test
            (file v98.g alt 1 line 1 and alt 2 line 2)
             are identical and have no resolving power

        -----------------  Note  -----------------

          This is different than the following case

                test: <<pred(LATEXT(1))>>? A B
                    | <<pred(LATEXT(1)>>?  A C
                    ;

          In this case there are two distinct predicates
          which have exactly the same text.  In the first
          example there are two references to the same
          predicate.  The problem represented by this
          grammar will be addressed later.

#131. (Changed in 1.33MR11) Case insensitive command line options

      Command line switches like "-CC" and keywords like "on", "off",
      and "stdin" are no longer case sensitive in antlr, dlg, and sorcerer.

#130. (Changed in 1.33MR11) Changed ANTLR_VERSION to int from string

      The ANTLR_VERSION was not an integer, making it difficult to
      perform conditional compilation based on the antlr version.

      Henceforth, ANTLR_VERSION will be:

            (base_version * 10000) + release number

            thus 1.33MR11 will be: 133*100+11 = 13311

      Suggested by Rainer Janssen ([e-mail])