readme

这个是内存数据库中的一个管理工具

Hi all Monet and DS hackers,

here comes the new version of the testing environment. Only a few details
have changed since my proposal.


- The testing environment consists of the following tools:
  + Mtest.py	a python script to run tests
  + Mprofile.py	a python script to run tests and collect profiling information
		(see below)
  + Mfilter.py	a python script to filter the test output before running Mdiff 
		(see below)
  + Mdiff	a C program to show the differences of two files as HTML document
  + Mapprove.py	a python script to approve recent test output (see below)
  + Mtimeout	a C program to limit the resouces of a program
		(thanks to Tim Ruhl, [e-mail])
  + MkillUsers	a bash script to kill test processes that are orphaned
		(used "fuser" to detect such processes)
  (Mtimeout and MkillUsers are currently available on Unix, only)


- In each directory of the current Monet and/or DS source tree tests may be
  provided in a subdirectory called "Tests". Of course, the tests should
  deal with the part/modules where they are located and they should be
  provided and maintained by the respective developer (see also below).


- Each "Tests" directory must contain a file "All" that contains the names
  of all tests in that directory (one name per line).


- Each test named TST consists of
  + a test script which is ONE of the following:
    * an arbitrary executable (e.g., a shell script)        (TST)           (~,^)
       on Windows, executables must be ".(exe|com)"         (TST.(exe|com)) (~,^)
                   and scripts must be ".(bat|cmd)"         (TST.(bat|cmd)) (~,^)
    * a Python script                                       (TST.py)        (~,^)
    * a MIL script to be executed by Mserver                (TST.milM) (',^,`)
    * a MIL script to be executed by Mserver -single        (TST.milS) (',^,`)
       (-single has been removed, so .milM & .milS are
        treated equally, now)
    * a MIL script to be executed by MapiClient -lmil       (TST.milC) (",^,`)
    * a SQL script to be executed by MapiClient -lsql       (TST.sql)  (",^,`)
    * a XQUERY script to be executed by MapiClient -lxquery (TST.xq)   (",^,`)
   (~) In case the name of an arbitrary executable or Python script is
        suffixed with ".MAPI" (i.e., TST.MAPI[.(exe|com|bat|cmd|py)]),
        Mtest.py starts an Mserver with a MAPI-listerner on MAPIPORT
        in the background before executing the test. The test can then
        connect via MAPI (on MAPIPORT) to that Mserver.
       Similar, with suffix ".SQL", an Mserver with an SQL-listener on
        SQLPORT is started;
       and, with suffix ".XQUERY", an Mserver with an XQUERY-listener on
        XQUERYPORT is started.
       In both cases, Mtest.py stops the Mserver again, once the test has
        finished.
   (') For Mserver, if several files are present named
        TST_sXX.mil(M|S) (XX={00,01,...,99})
        these are executed by subsequently calling Mserver
   (") For MapiClient, if several files are present named
        (1) TST_sXX.(milC|sql|xq) (XX={00,01,...,99})
            these are executed subsequently using the same Mserver
        (2) TST_pXX.(milC|sql|xq) (XX={00,01,...,99})
            these are executed cuncurrently using the same Mserver
   (^) For each test file TST[.*] involved,
        if a file called
	TST[.*].src  exists instead of  TST[.*],
	TST[.*].src  is expected to contain a single line
	giving the original location of the test file to be used.
	E.g.,  src/modules/plain/Tests/arith.milM.src  contains
		$TSTBLDBASE/$TSTDIR/arith.mil
	in order to use the MIL script extracted from
	src/modules/plain/arith.mx  as test script.
   (`) For each test file TST.* involved,
        if a file called
	TST.*.in  exists instead of  TST.*,  all environment variables in
	TST.*.in  are replaced by their current value
	when copying  $TSTSRCDIR/TST.*.in  to  $TSTTRGDIR/TST.* .
  + a MIL script to be used as prelude for Mserver
     (optional)                                             (TST.prelude)  (^,`)
  + a MAL script to be used as prelude for mserver5
     (optional)                                             (TST.prelude5) (^,`)
  + for frontend tests, a MIL script to be used 
     instead of the frontends dbinit script.
     (optional)                                             (TST.dbinit) (^,`)
  + for MapiClient tests, a MIL script to be used as
     prologue for Mserver, i.e., a script that is executed by Mserver after
     dbinit is executed, but before (the first)
     MapiClient connects to Mserver
     (optional)                                             (TST.prologue) (^,`)
  + for SQL tests, a MAL script to be used as
     prologue for mserver5, i.e., a script that is executed by mserver5 after
     "include sql;" is called and before (the first) SQL-MapiClient connects
     to mserver5
     (optional)                                             (TST.prologue5)(^,`)
  + for MapiClient tests, a MIL script to be used as
     epilogue for Mserver, i.e., a script that is executed by Mserver after
     (all) MapiClient(s) have finished
     (optional)                                             (TST.epilogue) (^,`)
  + for MapiClient tests, a MAL script to be used as
     epilogue for mserver5, i.e., a script that is executed by mserver5 after
     (all) MapiClient(s) have finished
     (optional)                                             (TST.epilogue5)(^,`)
  + a file that contains a list of modules (one per line)
     required by the test  (optional)                       (TST.modules)  (^,`)
  + a file that contains a list of BATs (one per line)
     (created by previous tests within the same directory)
     required by the test  (optional)                       (TST.BATs)     (^,`)
  + a set of files used by the arbitrary executable
     (optinal)                                              (TST.*)        (^,`)
  + stable (i.e. correct) versions of stdout and stderr
     of the test                                            (TST.stable.{out,err})
    operating system (OS) specific stable output can be 
    provided by adding the suffix ".<OSname>" (".`uname`"),
    ".<OSname><OSversion>" (".`uanme``uname -r`"), or
    ".<OSname><OSversion>.(64|32)bit" 
    to the respective filename
  + a file that contains a single integer number which is used as a factor
    to extend the default TIMEOUT for this test, only.      (TST.timeout)

  Using this naming conventions, the test environment can automatically
  decide what to do when called as "Mtest.py TST" (see below), i.e. execute
  one of the following:
  * TST[.exe|.com|.bat|.cmd] TST [TST.prelude]
  * python TST.py [TST.prelude]
  * Mserver --dbname=TSTDB [TST.prelude] < TST.mil(M|S)
  * Mserver --dbname=TSTDB --set mapi_port=$MAPIPORT --set xrpc_port=$XRPCPORT --dbinit="module(mapi); mil_start();" [TST.prologue] &
         MapiClient -lmil < TST[_(s|p)XX].milC
     or  TST[.exe|.com|.bat|.cmd] TST [TST.prelude]
     or  python TST.py [TST.prelude]
  * Mserver --dbname=TSTDB --set mapi_port=$MAPIPORT --set xrpc_port=$XRPCPORT --dbinit="module(sql_server);" [TST.prologue] &
         MapiClient -lsql < TST[_(s|p)XX].sql
     or  TST[.exe|.com|.bat|.cmd] TST [TST.prelude]
     or  python TST.py [TST.prelude]
  * Mserver --dbname=TSTDB --set mapi_port=$MAPIPORT --set xrpc_port=$XRPCPORT --dbinit="module(pathfinder);" [TST.prologue] &
         MapiClient -lxquery -oxml < TST[_(s|p)XX].xq
     or  TST[.exe|.com|.bat|.cmd] TST [TST.prelude]
     or  python TST.py [TST.prelude]

  (On Unix, all M<tool>'s are started with "Mtimeout -timeout TIMEOUT
   M<tool> ..." to kill (probably) hanging M<tool>'s after a certain
   timeout. The default TIMEOUT value is 60 seconds, i.e. 1 minute.
   The TIMEOUT can be changed globally using Mtest.py's "-t" option.
   The TIMEOUT for a single test can be extended be a factor provided
   in a file TST.timeout (see above).
   Additionally, MkillUsers is scheduled as at-job to kill orphaned
   processes that Mtimeout cannot reach any more.)

  (Mserver is called with "--config=$MONETDB_CONF --debug=$GDK_DEBUG $setMONETDB_MOD_PATH --set monet_prompt=".
   See below and "Mserver --help" for details.)

  (MapiClient is called with "--port=$MAPIPORT".
   See "MapiClient --help" for details.)


- Mtest.py uses three directory trees based at TSTSRCBASE, TSTBLDBASE, and
  TSTTRGBASE, respectively. These trees have similar purpose as the SOURCE,
  BUILD, and PREFIX trees when configuring and compiling Monet: the original
  tests are found in TSTSRCBASE, tests extracted from .mx or .in files
  during testing are fould in TSTBLDBASE, and Mtest.py writes the test
  output to TSTTRGBASE. The default settings are as follows:
	TSTSRCBASE=$MONETDB_SOURCE  (where you cecked out Monet's source tree)
	TSTBLDBASE=$MONETDB_BUILD   (where you called configure and make)
	TSTTRGBASE=$MONETDB_PREFIX  (where you told configure to put Monet)
  You can over rule any of these defaults by setting the respective
  environment variable, or by giving, e.g., --TSTTRGBASE=/tmp as command
  line option to Mtest.py. 

  IMPORTANT NOTE:

  In any case, all directories must be given as ABSOLUTE PHYSICAL PATHS,
  i.e., starting with "/" on Unix (or "[<drive-letter>:]\" on Windows, and
  not containing any symbolic link(s). 
  The latter is a restiction that might be released in the future, but for
  the time being, using paths that contain symbolic links will cause
  problems with Mtest.py.
  Hence, if you compiled Monet in a path that contains a symbolic link,
  e.g., ~/dev/monet -> /net/myhost/export/scratch1/myname/monet,
  you should either
  a) overrule the initial setting of MONETDB_SOURCE, MONETDB_BUILD,
     and MONETDB_PREFIX which were hardcoded during configure as (e.g.)
	MONETDB_SOURCE=~/dev/monet
	MONETDB_BUILD=~/dev/monet/$OSVER
	MONETDB_PREFIX=~/dev/monet/$OSVER
     by
     either setting the respective environment variables (e.g.) as follows
	MONETDB_SOURCE=/net/myhost/export/scratch1/myname/monet
	MONETDB_BUILD=/net/myhost/export/scratch1/myname/monet/$OSVER
	MONETDB_PREFIX=/net/myhost/export/scratch1/myname/monet/$OSVER
     before running Mtest.py 
     or using the respective command line options to Mtest.py
  or
  b) overrule the default settings of TSTSRCBASE, TSTBLDBASE, and
     TSTTRGBASE (e.g.) as follows
	TSTSRCBASE=/net/myhost/export/scratch1/myname/monet