howtousecpp_parser.txt

C Preprocessor,antlr的grammar语言描述,用于学习词法分析,语法分析

How To Use CPP_parser

Contents

1. How to run CPP_parser

2. How to organise the CPP_parser project

3. How to extract data from CPP_parser


(Best viewed with tabs set to 4)

Introduction

	CPP_parser is used to parse C/C++ programs and may be instrumented to
provide additional output for projects based on the analysis of source code
in C/C++ such as for "pretty printing", complexity analysis, and generally 
for any program understanding requirement.

	Here at London South Bank University, UK, it was designed and used for the
source of program data for the REST project which implements a "Ripple Effect"
metric based on a paper by Yau and Collofello (1980) ([e-mail]).

	The parser is generated from a language definition file called CPP_parser.g
using the open source ANTLR parser generation system obtainable from 
http://www.antlr2.org . I am currently (2007) using ANTLR V2.7.3 . 
(See http://www.antlr.org for ANTLR version 3)

1.  How to run the CPP_parser

    This description is for the use of Java in an MSDOS window for parser 
	generation and MSVC (6.0) for compiling and running the parser and analysis
	programs.

	This is because ANTLR is written in Java but we optionally request the parser
	to be written in C++ so as to be able to include data extraction modules
	previously written in C.

	See the next section on how to organise your files to be able to run the 
	system as described below.

1a. Generate parser

	Using MSDOS window set the directory address to where your project is stored
	containing the language definition file, CPP_parser.g. You can set the MSDOS
	window to start at this address whenever you open it.

	Enter the following command,

	.../MyProject>java antlr.Tool CPP_parser.g                 or
	.../MyProject>java antlr.Tool -traceParser CPP_parser.g

	This should create/update the following files (except for any unchanged),

	CPPLexer.cpp
	CPPLexer.hpp
	CPPParser.cpp
	CPPParser.hpp
	STDCTokenTypes.hpp
	STDCTokenTypes.txt

	It may be noted here that there are a number of options which may entered for
	antlr.Tool. To see what these are enter the following command,

	.../MyProject>java antlr.Tool

1b. Compile and build the parser

	This paper assumes you are using the Microsoft Visual C++. I am using V6.0.

	The set up for MSVC is discussed in section 2, "How to organise the 
	CPP_parser project", below.

	Having generated one or more new files by running antrl.Tool MSVC may tell
	you that some files have been modified outside the source editor in which 
	case click yes to accept the change(s).

    Having regenerated the lexer and/or parser click the build button.

    This will provide a report in the results window (which I keep across the
	bottom of the screen) showing the numbers of any errors and warnings
	detected.

	Any errors will have to be researched and corrected before the parser will 
	be linked and and executable constructed.

	Any warnings should be considered except for "warning C4101: 'pe' : 
	unreferenced local variable" in CPPLexer.cpp and CPPParser.cpp which can 
	be accepted. These are due to the way this ANTLR version (2.7.3) generates 
	catch statements.

	The reason for any other warnings should be researched to see if they can
	be eliminated.

1c. Run the parser

    I run the parser in the MSDOS window for flexibility.

	Enter the following command to parse a pre-processed file (*.i) in the
	TestIfiles directory.

	as follows,

	.../MyProject>debug\CPP_parser TestIfiles\Quadratic.i

	To place the DOS output in another file for later examination use,

	.../MyProject>debug\CPP_parser TestIfiles\Quadratic.i > Quadratic.trace

	See section 2, "How to organise the CPP_parser project" for where to keep
	required files.

	This run should normally produce something like this,

------------------------------------------------------
Centre for Systems and Software Engineering
London South Bank University

ANTLR C++ Parsing Project
Version 3.2 - November 2007

MyCode, Program to demonstrate how to use the parser
 by subclassing your application code here in MyCode.cpp
See myCode_function_direct_declarator(const char *q) in 
 MyCode.cpp and CPP_Parser.g
------------------------------------------------------

Parse TestIfiles\Quadratic.i Wed Dec 12 13:26:03 2007


Support exitExternalScope, scope now 0 as required

Run summary in TestIfiles\Quadratic.dat
See list of 3 functions in TestIfiles\Quadratic.lis
Also see optional list in TestIfiles\Quadratic.log

Parse ended

    
	Points to note about this output are as follows,

    Subclassing of application code is discussed in section 3, "How to extract
	data from CPP_parser".

	If scope does not end as 0 a slightly different warning will be produced. 
	This could occur if there is some error in the parsing process in which
	case some other error warning(s) should also have been displayed by the 
	parser.
		
	The files Quadratic.dat, Quadratic.lis and Quadratic.log are "optional" 
	files produced by MyCode as part of a "users" application and are here only
	produced as examples of what is possible. In this case MyCode merely as an 
	example and model to follow extracts the names of all the functions defined
	in Quadratic.cpp (as represented in Quadratic.i).


2.	How to organise the CPP_parser project

	Place CPP_parserV3.2 in your directories as the top level for this project

	This directory should contain,

	Files

		CPP_parser.dsw	(MSVC 6.0 Project Workspace)
		CPP_parser.g
        CPPDictionary.hpp
        CPPSymbol.hpp
        DictEntry.hpp
        Dictionary.cpp
        Dictionary.hpp
        Grammar.txt
	    HowToBuildStaticLibrary.txt
		HowToUseCPP_parser
        Main.cpp
        MyCode.cpp
        MyCode.hpp
        MyReadMe3.2.txt
        NotesScopes.txt
        Quadratic.cpp
		Quadratic.trace
		QuadraticA.trace
        Support.cpp
        Parser.cpp    (from ...\lib\cpp\src as modified)
        LLkParser.cpp (from ...\lib\cpp\src as modified)
        LLkParser.hpp (from ...\lib\cpp\antlr as modified)
		CharScanner.hpp (from ...\lib\cpp\antlr as modified)

	Directories

		Debug	(for executable etc.)

		TestIfiles    (for holding pre-processed files and related output)

			Quadratic.dat
			Quadratic.i   (this file can be used to check your system set up)
			Quadratic.lis
			Quadratic.log
			QuadraticA.dat
			QuadraticA.lis
			QuadraticA.log

	The ANTLR static source library should be held separately in, for example,

	lib on the same level as CPP_parserV3.2 .
	

3. How to extract data from CPP_parser

   One of the main reasons for processing source code with the ANTLR parser
   is to extract specific data from program source code for processing by a 
   user's (ie. your) program.

   To do this the user needs to include action statements in the parser
   definition, CPP_parser.g as follows.


3a.	In order to adapt the use of MyCode examine the following

	Find in CPP_parser.g

	// myCode functions ready for overriding in MyCode subclass
	// Include application code functions here
	virtual void myCode_pre_processing(int, char *[]);
	virtual void myCode_post_processing();
	virtual void myCode_end_of_stmt();
	virtual void myCode_function_direct_declarator(const char *);

	and

		{		
#ifdef MYCODE
		if (definition)
			myCode_function_direct_declarator(q);
#endif MYCODE
		}

	and in Support.cpp

	//	Functions which may be overridden by MyCode subclass

	void CPPParser::
	myCode_pre_processing(int argc,char *argv[])
		{
		}

3b. This is what they are for,

    myCode_pre_processing(int, char *[]) is defined in MyCode.cpp and
	initialises the user's (ie. your) MyCode application.

    myCode_post_processing() is defined in MyCode.cpp and finalises and
	closes down the user's application.

	myCode_end_of_stmt() is defined in MyCode.cpp and is used to clear 
	any unwanted settings at the end of processing each statement.

    myCode_function_direct_declarator(const char *) is defined in MyCode.cpp
	and is an example of how data can be extracted from the ANTLR parser. By 
	placing the statement "if (definition) myCode_function_direct_declarator(q);"
	within the #ifdef/#endif in the production function_direct_declarator in 
	CPP_parser.g the required data (in this case represented by the variable 
	"q") is extracted and can be processed or stored appropriately by the user.

3c. The user (ie. yourself) can take over the use of MyCode.cpp and introduce 
    whatever functions are required in the same way as shown above to extract 
	whatever data is required from (pre-processed) C++ source code.