perst.html

Perst开源实时数据库

state of the database. When transaction commit is performed by master node, slave node
uses this lock to transfer database into the next consistent state. Starting from this moment
read-only transactions at slave node will work with new consistent state of the database.<p>

Perst supports two replications models: static and dynamic. In static model, all
slave nodes have to be defined when application is started. Master node tries to connect
to all specified slave nodes. After establishing all possible connections with active slave nodes
it starts workings, replicating changes to these slave nodes. Content of all statically 
defined slave nodes is considered to be the same as of master node. So when database
is opened, master expects that all connected slave nodes has the same state as its own state
and will send to them only pages notified by application after open.
 If you are going to add new static slave node, 
you should first (before database is opened) manually copy database from the master node to this
new slave node.<p>

In dynamic mode it is possible to add new replication nodes at each moment of time. 
In this case Perst transfer the complete database to the new node, synchronizing it's state with
master node state. When new replication slave node is synchronized with the master node, 
it will act in the same way as static replication node - master will send to both of them 
all updates done by application. Dynamic mode is especially useful for main-memory databases (with infinite
page pool an NullFile stub). In this case size of the database is expected to be not so large and it's 
transfer to new replication node should not take a lot of time.<p> 

As far as replication nodes are also able to execute read-only queries, replication mechanism can be also used 
to improve performance by distributing load between several nodes. To be able to execute read-only transaction
at slave node, application should use per-thread transactions wrapped by 
<code>Storage.bBeginThreadTransaction(TransactionMode.ReplicationSlave)</code>
and <code>Storage.EndThreadTransaction()</code> methods invocations.  It is possible to run many such transactions 
concurrently. Only when master node commits transaction, slave node has to set exclusive lock
blocking all read-only transactions (certainly it has to wait first completion of all currently executed 
read-only transactions, so these transactions should be short enough to avoid blocking of replication mechanism for 
a long time). This exclusive lock is released very fast - immediately after placing new root page in the pool.
Now read-only transaction will see new state of the database.<p>

When application running at master node closes the storage, master sends CLOSE request to all slave 
nodes causing them to close connection with master. In case of abnormal termination of one of slave nodes, 
master continue to work with other slave nodes. If no more slave nodes are online, master continue to work autonomously.
Perst replication mechanism doesn't enforce any particular strategy for nodes recovery.
It is up to the application how to handle situation of master crash and choosing new master node.<p>


<H2> <A NAME = "jsql">JSQL</A></H2>

<H3> <A NAME = "overview">Overview</A></H3>

JSQL is subset of SQL languages, which can be used to select objects 
instances according to selection condition.
JSQL uses notation more popular for object-oriented programming than for relational database. 
Table rows are considered as object instances and the table - as class of these
objects. Unlike SQL, JSQL is oriented on work with objects instead of SQL
tuples. So the result of each query execution  is a set of objects of one 
class. The main differences of JSQL from standard SQL are:<P>

<OL>
<LI> There are no joins of several tables and nested subqueries. Query always
returns set of objects from one table. 
<LI> Standard C# types are used for atomic table columns.
<LI> There are no NULL values, except null references. 
<LI> Arrays can be used as record components. Special 
<B>exists</B> quantor is provided for locating element in arrays. 
<LI> User methods can be defined for table records (objects) as well as
for record components.
<LI> References between objects are supported including user defined reference resolvers.
<LI> As far as query language is deeply integrated with C# language, case
sensitive mode is used for language identifiers as well as for keywords. 
<LI> No implicit conversion of integer and floating types is done to string
representation. If such conversion is need, it should be done explicitly. 
</OL><P>

In Perst there is <code>org.garret.perst.Query</code> class allowing to execute JSQL
queries. To execute JSQL query it is necessary to provide object iterator 
(java.util.Iterator), class of iterated objects and string representing query condition.
It is also possible to specify indices, resolvers, query parameters.
Result of query execution is another iterator which can be used to traverse though selected objects
(object matching specified search criteria).<P>

All Perst collection classes contains <code>select</code> method which allows to filter collection members
using JSQL query. This method is give class of objects stored in the collection (for some collections
such as FieldIndex it is known and should not be specified) and JSQL condition. As the result of its work, 
select returns iterator of selected objects.<P>

<H3> <A NAME = "bnf">JSQL formal grammar</A></H3>

The following rules in BNF-like notation specifies grammar of
JSQL query language search predicate:<P>

<TABLE BORDER ALIGN="center">
<CAPTION>Grammar conventions</CAPTION>
<TR><TH>Example</TH><TH>Meaning</TH></TR>
<TR><TD><I>expression</I></TD><TD>non-terminals</TD></TR>
<TR><TD><B>not</B></TD><TD>terminals</TD></TR>
<TR><TD ALIGN="center">|</TD><TD>disjoint alternatives</TD></TR>
<TR><TD>(<B>not</B>)</TD><TD>optional part</TD></TR>
<TR><TD>{<B>1</B>..<B>9</B>}</TD><TD>repeat zero or more times</TD></TR>
</TABLE><P>

<PRE>
<I>select-condition</I> ::= ( <I>expression</I> ) ( <I>traverse</I> ) ( <I>order</I> )
<I>expression</I> ::= <I>disjunction</I>
<I>disjunction</I> ::= <I>conjunction</I> 
        | <I>conjunction</I> <B>or</B> <I>disjunction</I>
<I>conjunction</I> ::= <I>comparison</I> 
        | <I>comparison</I> <B>and</B> <I>conjunction</I>
<I>comparison</I> ::= <I>operand</I> <B>=</B> <I>operand</I> 
        | <I>operand</I> <B>!=</B> <I>operand</I> 
        | <I>operand</I> <B>&lt;&gt;</B> <I>operand</I> 
        | <I>operand</I> <B>&lt;</B> <I>operand</I> 
        | <I>operand</I> <B>&lt;=</B> <I>operand</I> 
        | <I>operand</I> <B>&gt;</B> <I>operand</I> 
        | <I>operand</I> <B>&gt;=</B> <I>operand</I> 
        | <I>operand</I> (<B>not</B>) <B>like</B> <I>operand</I> 
        | <I>operand</I> (<B>not</B>) <B>like</B> <I>operand</I> <B>escape</B> <I>string</I>
        | <I>operand</I> (<B>not</B>) <B>in</B> <I>operand</I>
        | <I>operand</I> (<B>not</B>) <B>in</B> <I>expressions-list</I>
        | <I>operand</I> (<B>not</B>) <B>between</B> <I>operand</I> <B>and</B> <I>operand</I>
	| <I>operand</I> <B>is</B> (<B>not</B>) <B>null</B>
<I>operand</I> ::= <I>addition</I>
<I>additions</I> ::= <I>multiplication</I> 
        | <I>addition</I> <B>+</B>  <I>multiplication</I>
        | <I>addition</I> <B>||</B> <I>multiplication</I>
        | <I>addition</I> <B>-</B>  <I>multiplication</I>
<I>multiplication</I> ::= <I>power</I> 
        | <I>multiplication</I> <B>*</B> <I>power</I>
        | <I>multiplication</I> <B>/</B> <I>power</I>
<I>power</I> ::= <I>term</I>
        | <I>term</I> <B>^</B> <I>power</I>
<I>term</I> ::= <I>identifier</I> | <I>number</I> | <I>string</I> 
        | <B>true</B> | <B>false</B> | <B>null</B> 
	| <B>current</B> | <B>first</B> | <B>last</B>
	| <B>(</B> expression <B>)</B> 
        | <B>not</B> <I>comparison</I>
	| <B>-</B> term
	| <I>term</I> <B>[</B> expression <B>]</B> 
	| <I>identifier</I> <B>.</B> <I>term</I> 
	| <I>function</I> <I>term</I>
        | <B>count (*)</B> 
        | <B>contains</B> <I>array-field</I> (<B>with</B> expression) (<B>group by</B> <I>identifier</I> <B>having</B> <I>expression</I>)
        | <B>exists</B> <I>identifier</I> <B>:</B> <I>term</I>
<I>function</I> ::= <B>abs</B> | <B>length</B> | <B>lower</B> | <B>upper</B>
        | <B>integer</B> | <B>real</B> | <B>string</B> | 
        | <B>sin</B> | <B>cos</B> | <B>tan</B> | <B>asin</B> | <B>acos</B> | 
        | <B>atan</B> | <B>log</B> | <B>exp</B> | <B>ceil</B> | <B>floor</B> 
        | <B>sum</B> | <B>avg</B> | <B>min</B> | <B>max</B> 
<I>string</I> ::= <B>'</B> { { <I>any-character-except-quote</I> } (<B>''</B>) } <B>'</B>
<I>expressions-list</I> ::= <B>(</B> <I>expression</I> { <B>,</B> <I>expression</I> } <B>)</B>
<I>order</I> ::= <B>order by</B> <I>sort-list</I>
<I>sort-list</I> ::= <I>field-order</I> { <B>,</B> <I>field-order</I> }
<I>field-order</I> ::= <I>field</I> (<B>asc</B> | <B>desc</B>)
<I>field</I> ::= <I>identifier</I> { <B>.</B> <I>identifier</I> }
<I>fields-list</I> ::=  <I>field</I> { <B>,</B> <I>field</I> }
<I>user-function</I> ::= <I>identifier</I>
</PRE><P>

Identifiers are case sensitive, begin with a..z, A..Z, '_' or '$' 
character, contain only a-z, A..Z, 0..9 '_' or '$' characters, and
do not duplicate a SQL reserved words.<P>

<TABLE WIDTH=100%>
<CAPTION>List of reserved words</CAPTION>
<TR><TD>abs</TD><TD>acos</TD><TD>and</TD><TD>asc</TD><TD>asin</TD></TR>
<TR><TD>atan</TD><TD>avg</TD><TD>between</TD><TD>by</TD><TD>contains</TD></TR>
<TR><TD>cos</TD><TD>ceil</TD><TD>count</TD><TD>current</TD><TD>desc</TD></TR>
<TR><TD>escape</TD><TD>exists</TD><TD>exp</TD><TD>false</TD><TD>floor</TD></TR>
<TR><TD>group</TD><TD>having</TD><TD>in</TD><TD>integer</TD><TD>is</TD></TR>
<TR><TD>length</TD><TD>like</TD><TD>log</TD><TD>lower</TD><TD>max</TD></TR>
<TR><TD>min</TD><TD>not</TD><TD>null</TD><TD>or</TD><TD>real</TD></TR>
<TR><TD>sin</TD><TD>string</TD><TD>sum</TD><TD>tan</TD><TD>true</TD></TR>
<TR><TD>upper</TD><TD>with</TD></TR>
</TABLE><P>


JSQL extends ANSI standard SQL operations by supporting bit manipulation 
operations. Operators <code>and</code>/<code>or</code> can be applied not only 
to boolean operands but also to operands of integer type. Result of applying 
<code>and</code>/<code>or</code> operator to integer operands is integer
value with bits set by bit-AND/bit-OR operation. Bits operations can be used
for efficient implementation of small sets. Also rasing to a power
operation <B>^</B> is supported by JSQL for integer and floating point
types.<P> 

<H3><A NAME = "array">Arrays</A></H3>
JSQL is able to work with C# array types.
JSQL provides a set of special constructions for dealing with arrays:<P>

<OL>
<LI>It is possible to get the number of elements in the array by 
<code>length()</code> function. 

<LI>Array elements can be fetched by <code>[]</code> operator.
If index expression is out of array range, then exception will be raised.

<LI>Operator <code>in</code> can be used for checking if array contains
value specified by left operand. This operation can be used only for arrays of
atomic types: with boolean, numeric, reference or string components.

<LI>Iteration through array elements is performed by <code>exists</code> 
operator. Variable specified after <code>exists</code> keyword can be used
as index in arrays in the expression preceded by <code>exists</code>
quantor. This index variable will iterate through all possible array
index values, until value of expression will become <code>true</code> or
index runs out of range. Condition

<PRE>
        exists i: (contract[i].company.location = 'US')
</PRE>

will select all details which are shipped by companies 
located in US, while query

<PRE>
        not exists i: (contract[i].company.location = 'US')
</PRE>

will select all details which are shipped only from companies outside US.<P>

Nested <code>exists</code> clauses are allowed. Using of nested 
<code>exists</code> quantors is equivalent to nested loops using correspondent
index variables. For example query

<PRE>
        exists colon: (exists row: (matrix[colon][row] = 0))
</PRE>

will select all records, containing 0 in elements of <code>matrix</code> 
field, which has type array of array of integer.
This construction is equivalent to the following
two nested loops:

<PRE>
       boolean result = false;
       for (int colon = 0; colon < matrix.length(); colon++) { 
            for (int row = 0; row < matrix[colon].length(); row++) { 
	         if (matrix[colon][row] == 0) { 
                     result = true;
		     break;
                 }
            }
       }
</PRE>

Order of using indices is significant! Result of the following query execution
<PRE>
        <code>exists row: (exists colon: (matrix[colon][row] = 0))</code>
</PRE>

will be completely different with result of previous query. The program can 
simply hang in last case due to infinite loop for empty matrices. 
<LI>It is possible to use the following clause<BR><B>contains</b> <i>array-field</i> (<b>with</b> <i>expression</i>) (<b>group by</b> <i>array-component-field</i> <b>having</b> <I>expression-with-aggregate-functions</I>)<BR>
to select arrays which elements match specified <i>with</i> condition and group elements in array to apply aggregate functions.
<BR>If <b>with</b> expression is specified in <b>contains</b> clause, this construction is equivalent to <b>exists</b> statement
with the same condition with two exceptions:
<UL>
<LI>In <i>with expression</i> it is possible to referee components of array elements without any indices or array field, i.e.