fea.tme

speech signal process tools

.lo
.SJ "Guidelines for Implementing ESPS Feature File Subtypes"
.AU "Alan Parker"
.RB "Rodney Johnson"
.TM ETM-S-86-25:rap 3.3 1/22/93
.*R
ESPS User's Manual
.*R
J. Shore and R. Parker, Introduction to the Entropic Signal Processing System (ESPS)
.*R
ETM-S-86-13:rap/jtb, Data Files in the Entropic Signal Processing System (ESPS)
.*R
ETM-S-86-14, Entropic Signal Processing System Programming Guidelines
.sh 1 Introduction
.pp
This document describes how to use the ESPS Feature file type
(\fIfea\fR(5\-ESPS)). 
.(f
\(co Copyright 1987-90 Entropic Speech, Inc.; All rights reserved.  
\(co Copyright 1990-93 Entropic Research Lab, Inc.; All rights reserved.  
.)f
ESPS feature files allow the user to create an ESPS data file with named
fields of different data types and sizes.
They can be used to define a new public ESPS file type
(known as a feature-file subtype)
or to implement a private file type for the user's own work.
The private file type
may be a completely new type or an extension of an existing type.
When used in conjunction with the generic header feature
(see \fIadd_genhd\fR(3\-ESPS)),
FEA files provide a very general way for the user to create ESPS data files.
.pp
The general model for using feature files depends on whether the file is
being used for input or output,
but in either case a program must obtain a file header
before doing anything else with a file.
Not only is the header
the first piece of information to be read or written,
it must be referred to by the routines that allocate space for data records
and read, access, and write them.
The
.i common
part and the
.i variable
part of the header in a feature file contain the same information,
including generic header items,
as in other ESPS files
(see
.i ESPS (5\-ESPS)).
The type-specific part of the header
contains the definition of each data field,
including its name, size, dimensions, and location within the record
(see
.i FEA (5\-ESPS)).
.pp
When an output feature file is created,
a new header must be built
unless a copy of an existing header can be suitably modified.
A new feature-file header is created with
.i new_header;
then the data fields are defined with
.i add_fea_fld,
and generic header items are created with the functions
.i add_genhd_* (3\-ESPS).
Details are given in Section 3.
After all the header information is fixed in place,
the header is written out, as with any other kind of ESPS file.
After the header has been written to the file, the program can
allocate storage for a data record
and start filling in data and writing out records.
Methods for dealing with the data are discussed in Sections 4 and 5.
.pp
For input feature files,
life is simpler since the fields have already been defined
(when the file was created).
After reading the header, the program can allocate storage for a data record
and start reading in records and accessing the data in them.
The programmer will usually know what the fields are and their types.   
The sizes of the fields
might be known by definition or through header items.
But in all cases this information is in the FEA header
and available through access functions
(see
.i get_fea_siz (3\-ESPS)).
.pp
If the feature file is a predefined public subtype
(public in the sense that a file type like SD is public),
then documentation of that subtype
will list the fields and give their type and size information.
Usually a
.i "support module"
will be provided:
a data structure and functions for dealing with the subtype more conveniently
than through the basic feature-file facilities.
The functions support initializing headers,
allocating record storage, and reading and writing records.
The use of a typical set of such support functions is illustrated in Section 2.
.pp
Sections 3 through 6 show how the functions of a support module
are implemented in terms of the basic feature-file facilities.
Section 7 shows how a predefined subtype can be extended with additional
fields to meet special needs.
Section 8 describes the documentation and other material that should be
provided when a new feature-file subtype is installed for general use.
.sh 1 "Example \- Use of a Subtype Support Module"
.pp
This document uses as an example
a simplified version of a feature-file subtype FEA_ANA
that was created to hold the results of analysis of speech data.
The subtype is defined to be tagged
and to have four vector fields and a scalar field
as listed in the following table:
.sp .5
.TS
center, box, tab(;);
c | c | c | c 
l | l | l | l.
Name;Size;Rank;Type
=
raw_power;\fImaxraw\fP;1 (vector);float
lpc_power;\fImaxlpc\fP;1 (vector);float
p_pulse_len;\fImaxpulses\fP;1 (vector);float
ref_coeff;MAX(\fIorder_vcd, order_unvcd\fP);1 (vector);float
frame_len;1;0 (scalar);long
.TE
.sp .5
The variables in the
.i Size
column refer to the following generic header items,
which are also part of the FEA_ANA definition.
.sp .5
.TS
center, box, tab(;);
c | c | c
l | l | l.
Name;Size;Type
=
maxraw;1;long
maxlpc;1;long
maxpulses;1;long
order_vcd;1;long
order_unvcd;1;long
.TE
.sp .5
(If you consult the
.i FEA_ANA (5-ESPS)
manual page you will find other fields and header items listed,
but these are enough for an illustration.)
.pp
To use the FEA_ANA file subtype,
we have to create a header with the field definitions if it is a new file.
The basic ESPS utility function
.i new_header
will create the header.
The function
.i init_anafea_hd
from the FEA_ANA support module
will add the necessary field definitions and generic header items.
This function takes as arguments the values that determine the record size.
The following will do the job:
.ip
.nf
hd = new_header(\fB\s-1FT_FEA\fR\s+1);
init_anafea_hd(hd, maxraw, maxlpc, maxpulses, order_vcd, order_unvcd);
.fi
.lp
The functions
.i new_header
and
.i init_anafea_hd
really may return error codes, but I'm ignoring
that here.
Other header initializations should also be done,
such as filling in the comment field and the source files,
but these are not shown, as they are not peculiar to feature files.
Now assuming that the header is complete, we write it out,
.ip
write_header(hd, file);
.lp
just as with any other ESPS file.
.pp
One simple way to use feature-file data records is to get
pointers to the data and then move data between
variables in your program and the feature file record.
But with the pointers
it's even easier to use the memory in the feature-file record
instead of the program variables.
This is one reason C has pointers.
It's convenient to define a data structure to hold all the pointers:
.ip
.nf
\fBstruct\fR anafea
{
    \fBlong\fR *tag;
    \fBfloat\fR *raw_power;
    \fBfloat\fR *lpc_power;
    \fBfloat\fR *p_pulse_len;
    \fBfloat\fR *ref_coeff;
    \fBlong\fR *frame_len;
    \fBstruct\fR fea_data *fea_rec;
}
.fi
.lp
For a public feature subtype,
a structure declaration like this is provided as part of the support module
and included in the program with a line like
.ip
#\fBinclude\fR <esps/anafea.h>
.lp
near the beginning.
Besides pointers for the tag and the 5 feature-file fields,
we have a pointer
.i fea_rec
to a
.i fea_data
structure containing the data values.
This is used by functions
.i get_anafea_rec
and
.i put_anafea_rec
that are introduced further down.
.pp
Now an \fIanafea\fR structure must be allocated.
The support module contains a function
.i allo_anafea_rec
for that purpose.
Assuming a declaration
.ip
\fBstruct\fR anafea *ana;
.lp
the statement
.ip
ana = allo_anafea_rec(hd);
.lp
will allocate the
.i anafea
structure and the
.i fea_data
structure it refers to
and assign the proper pointer values.
.pp
Now, let's use the data.  Note that you have
two ways for knowing the size of an array, such as
.i lpc_power.
One way is to know that it is
.i maxlpc
from the header.
In case you don't know that, or the rules change,
you can get the size of a feature file field with
.i get_fea_siz. 
Assignments as in
.ip
.nf
\fBfor\fR (i=0; i<maxlpc; i++)
    ana\->lpc_power[i] = some_expression;
.fi
.lp
will store values in the pool of memory set up by the
.i allo_anafea_rec
call.
The stored values can be used in expressions as in
.ip
something = ana\->lpc_power[some_index] + something_else;
.lp
Of course, you could be a real C programmer and avoid the use of array
subscripts entirely and just increment the pointer value.
.pp
As for the scalars (\c
.i tag
and
.i frame_len )
you just have to remember that they are pointers.
So to set the
.i frame_len
value do
.ip
.nf
*ana\->frame_len = xx;
.fi
.lp
You can use
.i *ana\->frame_len
in any expression as often as you please.
No need to assign it to another variable.
You can access the tag by
.i *ana\->tag.
.pp
Once values have been assigned to all the fields in the record,
copy the data in the allocated storage to the file by doing
.ip
put_anafea_rec(ana, hd, file);
.lp
Then you are ready to assign new values to the fields,
output another record with
.i put_anafea_rec,
and so on as often as you like.
After the initial call, you never call
.i allo_anafea_rec
again.
.pp
The functions
.i new_header
and
.i init_anafea_hd
are not used for input files.
If you are reading a FEA_ANA file instead of creating one, get the header with
.ip
hd = read_header(file);
.lp
or use
.i eopen (3-ESPSu).
With the header in hand, allocate a record structure just as above:
.ip
ana = allo_anafea_rec(hd);
.lp
Before you can use the data (for an input file) it must be read from the file:
.ip
get_anafea_rec(rec, hd);
.lp
This reads a record from a file into the memory set up by the
.i allo_anafea_rec
call.
Now you can access the data by referring to
.i ana\->lpc_power [\fIi\fP],
.i *ana\->frame_len,
and the other fields.
After processing one record, you can use
.i get_anafea_rec
to read another and repeat.
Since
.i get_anafea_rec
returns \fB\s-1EOF\fR\s+1 on end of file (and a positive integer otherwise),
it is possible to use a loop of the form
.ip
.nf
\fBwhile\fR (get_anafea_rec(ana, hd, file) != \fB\s-1EOF\fR\s+1)
{
    /* Process one record. */
}
.fi
.lp
to process an entire file.
.sh 1 "Creating Headers"
.pp
To use the FEA_ANA file subtype,
we have to create a header with the field definitions if it is a new file.
Here is how to do this with the basic feature-file facilities.
Use
.ip
hd = new_header(\fB\s-1FT_FEA\fR\s+1);
.lp
to create the header.
Then do
.ip
hd\->common.tag = \fB\s-1YES\fR\s+1;
.lp
since FEA_ANA files are supposed to be tagged.
.pp
The basic function for defining fields in feature files is
.i add_fea_fld.
The statement
.ip
add_fea_fld("raw_power", maxraw, 1, (\fBlong\fP *) \fB\s-1NULL, FLOAT,\fR\s+1 (\fBchar\fP **) \fB\s-1NULL,\fR\s+1 hd);
.lp
adds to the header
information defining the size, type, and dimensions of the field
.i raw_power.
Similar invocations of
.i add_fea_fld
are required for the other FEA_ANA fields.
.pp
The functions
.i add_genhd_* (3-ESPS)
add generic header items of various types to ESPS file headers.
The one that adds
.i long
items is
.i add_genhd_l.
The function returns
a pointer to the storage allocated in the header for the item,
so a statement like
.ip
*add_genhd_l("maxraw", (\fBlong\fP *) \fB\s-1NULL\fR\s+1, 1, hd) = maxraw;
.lp
both creates the item and assigns a value to it.
One such statement is needed for each FEA_ANA generic header item.
.pp
The functions
.i new_header
and
.i add_fea_fld
may return error codes that should be checked,
though the checking code is not shown above.
.pp
After the header is complete, but before any records are written,
executing the statement
.ip
write_header(hd, file);
.lp
writes the header to the file.
.pp
The steps above show what is required to create a feature-file header.
Often there will be several programs
that deal with a particular feature-file subtype.
In that case it is worth the effort to create a support module for the subtype
and to include in it a routine like
.i init_anafea_hd
for filling in the header of a feature file
and making it a file of the correct subtype.
The routine creates generic header items as needed and creates the data
record fields.
In general, such a routine takes as arguments
those values that affect the size of the data record.
These values are stored in the file header (as generic header items).
.pp
This routine is only used when creating a new FEA_ANA file for
output.  It is not used when reading a FEA_ANA file.  This function
returns zero on success and non-zero on failure.
It uses a macro
.i ADDFLD
to simplify rewriting the series of 5 invocations of
.i add_fea_fld
and to check the return codes.
.ip
.nf
#\fBdefine\fP ADDFLD(name, size, rank, dimen, type, enums) \\
    {if (add_fea_fld((name), (\fBlong\fP)(size), (rank), (\fBlong\fP*)(dimen), \\
	type, (\fBchar\fP**)(enums), hd) == \-1) \fBreturn\fP 1;}

\fBint\fR
init_anafea_hd(hd, maxraw, maxlpc, maxpulses, order_vcd, order_unvcd)
    \fBstruct\fR header *hd;
    \fBint\fR maxraw, maxlpc, maxpulses, order_vcd, order_unvcd;
{
    /* file is tagged */

    hd\->common.tag = \fB\s-1YES\fR\s+1;

    /* save parameters in the header */

    *add_genhd_l("maxraw", (\fBlong\fP *) \fB\s-1NULL\fR\s+1, 1, hd) = maxraw;
    *add_genhd_l("maxlpc", (\fBlong\fP *) \fB\s-1NULL\fR\s+1, 1, hd) = maxlpc;
    *add_genhd_l("maxpulses", (\fBlong\fP *) \fB\s-1NULL\fR\s+1, 1, hd) = maxpulses;
    *add_genhd_l("order_vcd", (\fBlong\fP *) \fB\s-1NULL\fR\s+1, 1, hd) = order_vcd;
    *add_genhd_l("order_unvcd", (\fBlong\fP *) \fB\s-1NULL\fR\s+1, 1, hd) = order_unvcd;

    /* create the record fields */

    ADDFLD("raw_power", maxraw, 1, \fB\s-1NULL, FLOAT, NULL\fR\s+1)
    ADDFLD("lpc_power", maxlpc, 1, \fB\s-1NULL, FLOAT, NULL\fR\s+1)
    ADDFLD("p_pulse_len", maxpulses, 1, \fB\s-1NULL, FLOAT, NULL\fR\s+1)
    ADDFLD("ref_coeff",
	MAX(order_vcd,  order_unvcd), 1, \fB\s-1NULL, FLOAT, NULL\fR\s+1)
    ADDFLD("frame_len", 1, 0, \fB\s-1NULL, LONG, NULL\fR\s+1)

    hd\->hd.fea\->fea_type = \fB\s-1FEA_ANA\fR\s+1;  /* this type code is assigned when
					    the subtype becomes official */
    \fBreturn\fR 0;
}
.fi
.sh 1 "Allocating Data Records"
.pp
To deal with a data record, two structures are to be allocated:  a
.i fea_data
structure to hold the data and an
.i anafea
structure to hold pointers to the locations in the
.i fea_data
structure where the various fields are stored.
First we'll see how to allocate the structures and set up the pointers
with just the basic feature-file facilities.
Then comes code for a support-module function
.i allo_anafea_rec
that takes care of all these tasks.
.pp
We begin with allocation of the
.i anafea
structure.
Assuming a variable declaration
.ip
\fBstruct\fR anafea *ana;
.lp
the statement
.ip
ana = (\fBstruct\fR anafea *) calloc(1, \fBsizeof\fR *ana);
.lp
will allocate the
structure and assign a pointer to it to
.i ana.
Of course, depending on the needs of the program,
the structure could have simply been declared at compile time.
.pp
The data-record memory is allocated with the basic feature-file routine
.i allo_fea_rec,
which uses information in the file header