javaccgrm.html

This a JavaCC documentation.

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod10">regular_expr_production</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>[ <A HREF="#newprod1">lexical_state_list</A> ]</TD>
</TR>
<TR>
<TD></TD><TD></TD>
<TD ALIGN=LEFT VALIGN=BASELINE><A HREF="#prod17">regexpr_kind</A> [ "[" "IGNORE_CASE" "]" ] ":"</TD>
</TR>
<TR>
<TD></TD><TD></TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"{" <A HREF="#prod18">regexpr_spec</A> ( "|" <A HREF="#prod18">regexpr_spec</A> )* "}"</TD>
</TR>
</TABLE>
<P>

A regular expression production is used to define lexical entities
that get processed by the generated token manager.  A detailed description
of how the token manager works is provided in
<A HREF="tokenmanager.html">this minitutorial (click here)</A>.  This
page describes the syntactic aspects of specifying lexical entities,
while <A HREF="tokenmanager.html">the minitutorial</A> describes how
these syntactic constructs tie in with how the token manager actually
works.
<P>
A regular expression production starts with a specification of the
lexical states for which it applies (the
<A HREF="#newprod1">lexical state list</A>).
There is a standard lexical state called "DEFAULT".  If the
<A HREF="#newprod1">lexical state list</A> is omitted, the regular
expression production applies to the lexical state "DEFAULT".
<P>
Following this is a description of what kind of regular expression
production this is (<A HREF="#prod17">see below for what this means</A>).
<P>
After this is an optional "[IGNORE_CASE]".  If this is present, the
regular expression production is case insensitive - it has the same
effect as the
<A HREF="#prod6">IGNORE_CASE</A>
option, except that in this case it applies locally to this regular
expression production.
<P>
This is then followed by a list of regular expression specifications
that describe in more detail the lexical entities of this regular
expression production.

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod12">token_manager_decls</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"<A NAME="TOKEN_MGR_DECLS">TOKEN_MGR_DECLS</A>" ":" <EM>java_block</EM></TD>
</TR>
</TABLE>
<P>

The token manager declarations starts with the reserved word
"TOKEN_MGR_DECLS" followed by a ":" and then a set of Java declarations
and statements (the Java block).  These declarations and statements are
written into the generated token manager and are accessible from within
<A HREF="#prod18">lexical actions</A>.  See
<A HREF="tokenmanager.html">the minitutorial on the token manager</A>
for more details.
<P>
There can only be one token manager declaration in a JavaCC grammar file.

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="newprod1">lexical_state_list</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"&lt;" "*" "&gt;"</TD>
</TR>
<TR>
<TD></TD><TD>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"&lt;" <EM>java_identifier</EM> ( "," <EM>java_identifier</EM> )* "&gt;"</TD>
</TR>
</TABLE>
<P>

The lexical state list describes the set of lexical states for which
the corresponding <A HREF="#prod10">regular expression production</A>
applies.  If this is written as "<*>", the regular expression production
applies to all lexical states.  Otherwise, it applies to all the lexical
states in the identifier list within the angular brackets.

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod17">regexpr_kind</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"TOKEN"</TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"SPECIAL_TOKEN"</TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"SKIP"</TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"MORE"</TD>
</TR>
</TABLE>
<P>

This specifies the kind of
<A HREF="#prod10">regular expression production</A>.
There are four kinds:
<P>
<UL>
<LI>
<STRONG><A NAME="TOKEN">TOKEN</A></STRONG>:
The regular expressions in this regular expression production describe
<EM>tokens</EM> in the grammar.  The token manager creates a
<A HREF="apiroutines.html">Token</A> object for each match of such
a regular expression and returns it to the parser.
<P>
<LI>
<STRONG><A NAME="SPECIAL_TOKEN">SPECIAL_TOKEN</A></STRONG>:
The regular expressions in this regular expression production describe
<EM>special tokens</EM>.  Special tokens are like tokens, except that
they do not have significance during parsing - that is the BNF productions
ignore them.  Special tokens are, however, still passed on to the parser
so that parser actions can access them.  Special tokens are passed
to the parser by linking them to neighboring real tokens using the
field "specialToken" in the <A HREF="apiroutines.html">Token</A>
class.  Special tokens are useful in the processing of lexical entities
such as comments which have no significance to parsing, but still
are an important part of the input file.  See
<A HREF="tokenmanager.html">the minitutorial on the token manager</A>
for more details of special token handling.
<P>
<LI>
<STRONG><A NAME="SKIP">SKIP</A></STRONG>:
Matches to regular expressions in this regular expression production
are simply skipped (ignored) by the token manager.
<P>
<LI>
<STRONG><A NAME="MORE">MORE</A></STRONG>:
Sometimes it is useful to gradually build up a token to be passed on
to the parser.  Matches to this kind of regular expression are stored
in a buffer until the next TOKEN or SPECIAL_TOKEN match.  Then all
the matches in the buffer and the final TOKEN/SPECIAL_TOKEN match
are concatenated together to form one TOKEN/SPECIAL_TOKEN that is
passed on to the parser.  If a match to a SKIP regular expression
follows a sequence of MORE matches, the contents of the buffer is
discarded.
</UL>

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod18">regexpr_spec</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE><A HREF="#prod19">regular_expression</A> [ <EM>java_block</EM> ] [ ":" <EM>java_identifier</EM> ]</TD>
</TR>
</TABLE>
<P>

The regular expression specification begins the actual description
of the lexical entities that are part of this
<A HREF="#prod10">regular expression production</A>.
Each regular expression production may contain any number of
regular expression specifications.
<P>
Each regular expression specification contains a regular expression
followed by a Java block (the lexical action) which is optional.
This is then followed by an identifier of a lexical state (which
is also optional).  Whenever this regular expression is matched,
the lexical action (if any) gets executed, followed by any
<A HREF="#prod6">common token actions</A>.  Then the action depending
on the
<A HREF="#prod17">regular expression production kind</A>
is taken.  Finally, if a lexical state is specified, the token
manager moves to that lexical state for further processing (the
token manager starts initially in the state "DEFAULT").

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod16">expansion_choices</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE><A HREF="#prod20">expansion</A> ( "|" <A HREF="#prod20">expansion</A> )*</TD>
</TR>
</TABLE>
<P>

Expansion choices are written as a list of one or more expansions
separated by "|"s.  The set of legal parses allowed by an expansion
choice is a legal parse of any one of the contained expansions.

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod20">expansion</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>( <A HREF="#prod22">expansion_unit</A> )*</TD>
</TR>
</TABLE>
<P>

An expansion is written as a sequence of expansion units.
A concatenation of legal
parses of the expansion units is a legal parse of the expansion.
<P>
For example, the expansion "{" decls() "}" consists of three expansion
units - "{", decls(), and "}".  A match for the expansion is a concatenation
of the matches for the individual expansion units - in this case, that would
be any string that begins with a "{", ends with a "}", and contains a match
for decls() in between.

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod22">expansion_unit</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>
<TD ALIGN=LEFT VALIGN=BASELINE><A HREF="#prod21">local_lookahead</A></TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE><EM>java_block</EM></TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"(" <A HREF="#prod16">expansion_choices</A> ")" [ "+" | "*" | "?" ]</TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>"[" <A HREF="#prod16">expansion_choices</A> "]"</TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>[ <EM>java_assignment_lhs</EM> "=" ] <A HREF="#prod19">regular_expression</A></TD>
</TR>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>|</TD>
<TD ALIGN=LEFT VALIGN=BASELINE>[ <EM>java_assignment_lhs</EM> "=" ] <EM>java_identifier</EM> "(" <EM>java_expression_list</EM> ")"</TD>
</TR>
</TABLE>
<P>

An expansion unit can be a <A HREF="#prod21">local LOOKAHEAD specification</A>.
This instructs the
generated parser on how to make choices at choice points.  For details
on how LOOKAHEAD specifications work and how to write LOOKAHEAD specifications,
<A HREF="lookahead.html">click here to visit the minitutorial on LOOKAHEAD</A>.
<P>
An expansion unit can be a set of Java declarations and code enclosed
within braces (the Java block).  These are also called <EM>parser
actions</EM>.  This is generated into the method parsing the
non-terminal at the appropriate location.  This block is executed
whenever the parsing process crosses this point successfully.
When JavaCC processes the Java block, it does not perform any detailed
syntax or semantic checking.  Hence it is possible that the Java compiler
will find errors in your actions that have been processed by JavaCC.
<EM>Actions are not executed during
<A HREF="lookahead.html">lookahead evaluation</A>.</EM>
<P>
An expansion unit can be a parenthesized set of one or more
<A HREF="#prod16">expansion choices</A>.  In which case, a legal parse of the expansion
unit is any legal parse of the nested expansion choices.
The parenthesized set of expansion choices can be suffixed (optionally) by:
<UL>
<LI>
<STRONG>"+":</STRONG>
Then any legal parse of the expansion unit is one or more
repetitions of a legal parse of the parenthesized set of
expansion choices.
<LI>
<STRONG>"*":</STRONG>
Then any legal parse of the expansion unit is zero or more
repetitions of a legal parse of the parenthesized set of
expansion choices.
<LI>
<STRONG>"?":</STRONG>
Then a legal parse of the expansion unit is either the
empty token sequence or any legal parse of the nested expansion choices.
An alternate syntax for this construct is to enclose the
expansion choices within brackets "[...]".
</UL>
<P>
An expansion unit can be a <A HREF="#prod19">regular expression</A>.  Then a legal parse
of the expansion unit is any token that matches this regular
expression.  When a regular expression is matched, it creates an
object of type <A HREF="apiroutines.html">Token</A>.  This object
can be accessed by assigning it to a variable by prefixing the
regular expression with "variable =".  In general, you may have any
valid Java assignment left-hand side to the left of the "=".
<EM>This assignment is not performed during
<A HREF="lookahead.html">lookahead evaluation</A>.</EM>
<P>
An expansion unit can be a non-terminal (the last choice in the syntax
above).  In which case, it takes
the form of a method call with the non-terminal name used as the
name of the method.  A successful parse of the non-terminal causes
the parameters placed in the method call to be operated on and a
value returned (in case the non-terminal was not declared to be
of type "void").  The return value can be assigned (optionally) to
a variable by prefixing the regular expression with "variable =".
In general, you may have any
valid Java assignment left-hand side to the left of the "=".
<EM>This assignment is not performed during
<A HREF="lookahead.html">lookahead evaluation</A>.</EM>
Non-terminals may not be used in an expansion in a manner that introduces
left-recursion.  JavaCC checks this for you.

<P>
<HR>
<P>

<TABLE>
<TR>
<TD ALIGN=RIGHT VALIGN=BASELINE><A NAME="prod21">local_lookahead</A></TD>
<TD ALIGN=CENTER VALIGN=BASELINE>::=</TD>