intf.nr

This Source-Navigator, an IDE for C/C++/Fortran/Java/Tcl/PHP/Python and a host of other languages.

                          Introduction

SDS is a library of calls, written in C but  with  interfaces  to
Fortran  and  C++,  whose  primary job is to describe arbitrarily
structured data.  As well as describing existing  collections  of
data, the SDS interface provides ways of organising existing data
and of requesting allocated memory for data which only exists  as
description.

At the moment [and perhaps forever] SDS can describe data  struc-
tures  that it cannot help generate: in particular data with size
and type information embedded in them which are generated by spe-
cial purpose hardware readout systems.

                              Terms

An Sds is a managed collection of objects and their descriptions.
The  objects may exist purely as descriptions or as arrays of one
or more instantiations, and their storage locations may  be  dif-
ferent.  Datasets  exist in two forms: a 'prototyping' stage when
descriptions of objects are being added to the dataset: these ob-
jects  may or may not have memory allocated for them; and an 'as-
sembled' stage when all memory that should be allocated has been.
When  assembled,  the  dataset is "closed" and objects may not be
added or removed, but full access is given to manipulate the data
themselves;  in  the  proto state the data cannot be modified but
the of the dataset may be.

Here, the term refers to a object,  which may range from  'primi-
tive' types such as integer, float and  character to complex data
types such as C structures and structures with embedded size  and
type information. Examples of objects are:

      int i;

      int iarray[5];

      struct foo {
        int    a[20];
        char   adesc[32];
        double asize;
      } bar[20];

      struct Module {
        int   Type;
        int   DataSize;
        short Data[DataSize];
      } Module[NumberOfModules];

This last case shows an array of objects with embedded  size  in-
formation.  Using a simple naming convention described more fully
below, SDS can apply such descriptions to real data  and  present
the user with the size information and object addresses.

Data objects managed by a dataset may  be  stored  with  the  SDS
headers  or  may  be marked 'disjoint': that is, their storage is
not with the headers. In the latter case, SDS may be informed  of
the storage location by the programmer (having retrieved this in-
formation from, for instance, a database) or the knowledge may be
explicitly stored with the header so that loading or attaching to
the object may be done automatically.


                      Sds uses a number of
: that is, opaque values acting as pointers  into  system  tables
which describe each of the components; they are analagous to file
descriptors.

The allows access to type information containing :

- A type code, indicating if the described  data  is  'primitive'
(eg SDS_INT, SDS_DOUBLE), 'primitive' with special interpretation
(eg SDS_UNIX_TIME, which is in fact SDS_LONG but  can  be  inter-
preted in a special fashion), or compound - referring to a struc-
ture. The typeH is the type code for such a compound  data  type,
and accesses a hierarchy of type codes which finally resolve into
primitives.

- For each type code, a multiplicity.

- For each type code, a definition name.

Each object is associated with a typeH, and  subobjects  used  to
describe compuond objects may also need type handles.  Thus, when
Sds describes the C structure:

    struct Fred {
      int aint;
      float bfloat[20];
    } Albert[3];

the typeH will access the information:

      -  one integer, name aint
      -  twenty floats, name bfloat

Both of these names are referred  to  here  as  names.  The  name
'Fred'  is  also  a definition name, but in Sds it is replaced by
the typeH. By contrast, the name 'Albert' is an  name,  which  is
not  part  of  the  type  description  and is accordingly managed
through the (see below).

To formalise data structures which cannot be easily described  by
C,  in  particular  when size and type information is embedded in
the data themselves, naming conventions  are  used.  A  structure
described by Sds (in pseudo-code) as :

    struct Fred {
      float scale_factor;
      short data_count;
      char data[MULTIPLICITY_ONE];
      int checksum;
    } Albert[MULTIPLICITY_TWO];

can be scanned by Sds to pick up the multiplicity of  the  'data'
arrays in each element of the 'Albert' array from embedded infor-
mation. Here, the strategy is to look for an array called  'data'
with  undetermined  length  after  an  [integer,short,byte] field
called  'data_count'  is  encountered.  Sds  can  then  scan  the
described  data  and, if required, store pointer and multiplicity
arrays with the header for subsequent fast access. Typically, the
total storage size will determine the value of Albert's multipli-
city.

The (storeH) allows access to information detailing where an  ob-
ject or subobject is stored:

- The storage name, for example a  filename  or  the  name  of  a
shared memory partition.

- the name of the host which controls the storage

- an indicator of the storage type  -  SDS_FILE,  SDS_SHARED_MEM,
SDS_DATABASE etc.

The sdsH allows access to dataset information: number of objects,
name and timestamp of dataset and object information such as mul-
tiplicities, structures and addresses; the  sdsP  refers  to  da-
tasets  whose  contents  -  number  and  type of objects - may be
changed. Many queries and manipulations may be asked of proto-sds
datasets,  but they will not respond to queries about the placing
of objects they manage: this is to provide protection against ac-
quiring addresses that may change due to Sds internals or copying
of objects. Such protection will be given if  decent  programming
practice  is  followed  but  inappropriate  use of pointers whose
validity is not guaranteed will cause problems. In  general,  the
transformation  from  a proto dataset to an object-accessible da-
taset should be followed by calls re-establishing  the  addresses
of any objects used.

The recordH  and  objectH  allow  access  to  object  information
(alignment, size, name, multiplicity, instantiation names and in-
directly their associated typeH and storeH).

The objectI points to the position of an object within a dataset:
user  objects start at number 1 [object 0 is the dataset directo-
ry, and should not be directly accessed by the user].

In general, sds calls will return a  long integer; a return of zero indicates an
error or warning which may be investigated with  the  error  han-
dling routines detailed below. All handles are invalid if 0.

NOTE: In what follows, 'Level 1' calls  are  those  of  immediate
usefullness  for  basic use of SDS, whereas Level 2 are of secon-
dary interest but still intended for the user. Internal calls are
not described.


                int call_succeeded = sds_init();
Initialise the Sds system tables. Required. Has  effect  only  on
the first call.


int call_succeeded = sds_reinit_enable();

Where a system may require a second initialisation, for  instance
a    realtime    system    after    partial    failure,   calling
sds_reinit_enable() will allow a subsequent call to sds_init() to
take  effect. Note that only the first subsequent sds_init() will
have effect; after that the normal behaviour returns.

int call_succeeded = sds_fprint_def(FILE *stream, objectH);

Print the structural definition of an object (ie names, types and
structure levels) to the named output stream.

int call_succeeded = sds_fprint_object(FILE *stream, objectH);

Print the data contents of an object to the named output  stream.
An attempt is made to give reasonable format.

int version = sds_version();

    Returns a floating point version number of the form 2.3



                    sdsH = sds_open(storeH)

Gets access to an existing dataset. You may now  query  it  about
what  is there and where it is, but you do not yet have access to
the data....

sdsH = sds_attach(storeH, int access_mechanism);

Attach to a dataset; ie you are  getting  access  to  the  actual
stuff,  through  mechanisms  such as local shared memory, mapping
disk, or reflective memory links.

sdsP = sds_create(char *name)

Gets  a  new   dataset   prototype   ready   for   filling   with
records/objects.  The  name given is the dataset name, which must
be unique for the datasets currently accessed by a process.

objectI = sds_add_oh(sdsP, objectH);

Add an object to a dataset prototype; returns the object_index of
the  object  within  the  dataset (Note that an object may be re-
gistered in more than one dataset, in which  case  their  indices
are in general different.)

int call_succeeded = sds_delete_oh(sdsP, objectH);

Delete an object from a dataset prototype.

sdsH = sds_make_public(sdsP, storeH);

This call creates the Sds described by the input  prototype;  the
dataset  is  now  in  principio  accessible  by  other  processes
(although system permissions may regulate this).  The  call  thus
implies  that some copying will be done: at least the Sds headers
and any objects declared in memory must  be  copied  to  the  new
storage  area. Data objects marked 'disjoint' will not be copied.
The sdsH returned now allows access to  all  data  pointers,  but
further manipulation of the dataset structure is illegal.

sdsH = sds_make_private(sdsP);

This call is similar to sds_make_public() in that a transform  is
made  between  a  proto dataset whose structure may be changed to
one where data may be manipulated but structure remains constant.
In  this case however objects in process memory remain there, and
proto objects without allocated memory will have process  storage
created.

sdsP = sds_protoize(sdsH);

Moves a dataset to the prototype state so that the dataset  stuc-
ture  may  be  changed.  Attempts to obtain object addresses from
sdsP or sdsH are now invalid. It cannot be assumed  that  objects
whose addresses were found from the generating sdsH before proti-
sation (yuk) will remain in these locations after further dataset
manipulation.

sdsH = sds_duplicate(sdsH, storeH);

Duplicates the dataset sdsH to new storage.

sdsH = sds_deep_duplicate(sdsH, storeH);

Duplicates the dataset sdsH to new storage, including  all  'dis-
joint' objects.