feasd.tme

speech signal process tools

.lo
.if t .ds -- \(em\h'-0.5m'\(em
.if n .ds -- ---
.SJ "FEA_SD Files\*(--Feature Files for Sampled Data in ESPS"
.AU "Rodney W. Johnson"
.TM ETM-S-89-49:rwj 1.3 "1/22/93"
.sh 1 "INTRODUCTION"
.pp
Originally ESPS used a special file type for storing sampled signal data.
.(f
\(co Copyright 1993 Entropic Research Lab, Inc. All rights reserved.  
.)f
This file type, the SD (``sampled-data'') file
is one of several specialized file types
that have been replaced with subtypes of the ESPS FEA (``feature'') file type.
The new FEA subtype for sampled data, the FEA_SD file,
is the subject of this note.
See the
.i FEA_SD (5-ESPS)
and
.i FEA (5-ESPS)
manual sections for more details on the FEA_SD file type.
.pp
With the change from SD to FEA_SD,
various new facilities become available for use with sampled-data files.
In addition to the general facilities that apply to all FEA files,
these include complex data types
(now in fact supported for FEA files in general)
and support for multichannel sampled data.
The new facilities are detailed in the next section.
.pp
The third section mentions some user-level programs
that create FEA_SD files or accept them as input.
These include programs for converting between the SD and FEA_SD file types
and programs for multiplexing and demultiplexing multichannel FEA_SD files.
Actually  most programs that formerly created or accepted SD files
now deal in the same way with at least one-channel real FEA_SD files.
The ones mentioned explicitly in this section are the ones
that so far have been significantly extended
to deal with complex or multichannel data.
Others that could usefully apply to complex or multichannel data
will be added to the list.
.pp
Section 4 gives an overview of the format of a FEA_SD file,
the special record structure used for holding FEA_SD sampled data in memory,
and the support functions provided
for setting up FEA_SD file headers and record structures and
reading and writing FEA_SD files.
.pp
In most cases, existing SD files will
.i not
have to be converted to FEA_SD, although of course they could be
converted with the program provided for that purpose.  Programs that
accepted SD files before FEA_SD was introduced will continue to accept
SD files even though they now accept FEA_SD files as well.  Steps
described in the Section 5 have been taken to assure backward
compatibility.  First, they make it easy to convert an existing
program from SD to FEA_SD input without sacrificing the ability to
accept SD files.  Secondly, they make it essentially automatic that
new programs that take FEA_SD input will also accept SD files.
.pp
Section 6 of this note is a guide to conversion of existing programs
so that they will use FEA_SD files instead of or in addition to SD files.
.sh 1 "NEW FACILITIES"
.pp
.sh 2 "General Feature-File Facilities"
.pp
An ESPS FEA file consists of a header followed by a sequence of
records, each of which consists of a number of named ``fields'' that
may hold various types of numeric or nonnumeric data or arrays of such
data.  A field is characterized by its name, data type, and array
dimensions.  In a FEA_SD file there is a single field that holds all
the sampled data.  The field is named "samples", it can have any
numeric data type, and it has one dimension: the number of channels.
There needn't be any other fields.  Each record contains one sample
from each channel; thus in a single-channel file, each record may
comprise merely a single sample.
.pp
FEA files are self-describing: the header of a FEA file contains a
list of its fields with their names, data types, dimensions, and other
information.  Powerful programs such as
.i fea_edit (1-ESPS),
.i select (1-ESPS),
and
.i fea_stats (1-ESPS)
can operate on arbitrary FEA files by using the header information.
These programs all can now accept FEA_SD files;
to them, a FEA_SD file is just another FEA file
that happens to have a field named "samples".
Thus you can now edit an ASCII representation of a sampled-data file
with your favorite text editor,
run complex queries against the contents of such files,
or compute inverse covariance matrices for multichannel sampled data.
.pp
New fields can be added to any FEA file\*(--see
.i add_fea_fld (3-ESPSu)
and
.i addfea (1-ESPS).
FEA_SD files can thus be created that contain other fields than "samples".
When such additional fields are present,
programs that read FEA_SD files will still work,
ignoring the extraneous data.
However, FEA_SD files with extra fields
may be read less efficiently than ``pure'' FEA_SD files,
since the extra fields complicate the record structure.
.sh 2 "Complex Data Types"
.pp
FEA_SD files were introduced into ESPS at the same that complex data types
were introduced for general FEA files.
As a result, sampled-data files with complex data types are now possible.
Formerly just 5 ESPS data types were possible for a sampled-data file:
the 5 real numeric types.
Now there are 5 more.
For every real numeric type there is now a corresponding complex type,
consisting of complex numbers
whose real and imaginary parts are of that real type.
For example the complex type corresponding to the real type
.i short
is defined by the C typedef
.ip
typedef struct {short	real, imag;}    short_cplx;
.lp
If
.i z
is a variable of this type, its real and imaginary parts are
.i z.real
and
.i z.imag,
which are variables of type
.i short.
.pp
The complete list of numeric types is shown in the following table.
A FEA_SD file can be of any one of these types.
.sp
.TS
center, box, tab(;);
c | c
l | l
l | l.
code;type
=
DOUBLE;double
FLOAT;float
LONG;long
SHORT;short
BYTE;signed char
DOUBLE_CPLX;double_cplx
FLOAT_CPLX;float_cplx
LONG_CPLX;long_cplx
SHORT_CPLX;short_cplx
BYTE_CPLX;byte_cplx
.TE
.sp
(The symbols in caps, sometimes loosely referred to as ESPS types,
actually stand for numeric codes defined in a header file.
The lower-case forms are the ones used in writing declarations in C programs.
See
.i FEA (5-ESPS)
for more information.)
.sh 2 "Multichannel Data"
.pp
In a sense the SD file type provided for multichannel data in that
the type-specific part of the SD header had a member
.i nchan
that was supposed to indicate the number of channels.
But no programs were provided to support multichannel data,
and probably no ESPS program ever set
.i nchan
equal to anything but 1 or 0 (which defaults to 1).
In a FEA_SD file, as noted above, the number of channels is the size
(number of items) of the field "samples".
There are now ESPS programs for multiplexing and demultiplexing
(see Section 3.2).
These can form a multichannel FEA_SD file by multiplexing several
single-channel files, extract channels from a multichannel file,
and perform various other similar operation.
.sh 1 "PROGRAMS THAT READ OR WRITE SAMPLED-DATA FILES"
.sh 2 "File Conversion"
.pp
Two programs are available for converting between SD and FEA_SD files.
The program
.i sdtofea (1-ESPS)
reads an SD file and writes a FEA_SD file containing the same data and
(as nearly as possible) the same header information.
The program
.i featosd (1-ESPS)
performs the inverse transformation (again, as nearly as possible).
See the manual sections for detailed information.
.sh 2 "Multiplexing and Demultiplexing"
.pp
The program
.i mux (1-ESPS)
can multiplex several single-channel or multichannel FEA_SD files into
a single FEA_SD file with as many channels as all the input files taken
together.
It can also join pairs of real input channels
into single complex output channels.
.pp
The program
.i demux (1-ESPS)
can extract a subset of the channels from its input file
and either put them in separate single-channel output files
or form a single output file with fewer channels than the input.
It can also split complex input channels into pairs of real output channels.
See the manual sections for these programs for more complete statements
of the exact capabilities of these programs and for examples of their use.
.sh 2 "Other Programs"
.pp
Most ESPS programs that formerly expected SD files as input will now
accept at least single-channel, real FEA_SD files as well.
Most ESPS programs that formerly produced SD files as output will now
at least produce equivalent single-channel, real FEA_SD files instead.
Many ESPS programs can reasonably be extended to perform useful operations
that involve reading or writing complex or multichannel sampled data.
Now that the basic infrastructure for complex and multichannel support
is in place, these extensions are being done.
The programs listed here are just the ones that have at this writing been
extended with support or partial support for complex or multichannel
sampled data.
.pp
.i testsd
will now create single-channel FEA_SD files of any numeric data type,
including the complex ones.
It can generate complex exponentials, which may be damped or frequency-swept,
and complex noise sequences with independent real and imaginary parts.
It can also create a FEA_SD file containing arbitrary complex data given by
numbers in an ASCII text file.
.pp
.i fft
will now compute fast Fourier transforms of single-channel complex input data.
.pp
.i fftinv
will now create single-channel complex output data as results of the inverse
fast Fourier transform.
.pp
.i addsd
and
.i multsd
will add or multiply single-channel FEA_SD files sample by sample.
The input files may have arbitrary numeric types, including complex.
.pp
.i copysd
will handle single-channel FEA_SD files of any real or complex type.
.pp
.i psps
and
.i pplain
will print both real and complex FEA_SD files (as well as other file types).
In
.i psps
output, complex numbers are formatted as bracketed pairs of real numbers.
In
.i pplain
output the real and imaginary parts
are simply printed in order as consecutive real numbers.
.sh 1 "FEA_SD FILE STRUCTURE AND SUPPORT MODULE"
.pp
This section describes the structure of the header of a FEA_SD file
(Subsection 4.1)
and the external format of its records
(Subsection 4.2).
It also describes the support data structure that has been defined for holding
sampled data in memory
(Subsection 4.3)
and the support functions that are provided for working with FEA_SD files
using this structure.
These include functions for initializing a FEA_SD file header,
allocating an instance of the support structure,
reading data from a FEA_SD file into the structure,
and writing data from the structure to a FEA_SD file.
.pp
The support structure permits data to be read and written in blocks,
rather than one sample at a time, provided certain conditions are
met\*(--\c in particular, only the field "samples" may exist in the
file, and the byte order of the data on the external file must match
the native byte order of the machine.  There are provisions for
automatic data-type conversion on input or output, but programs that
do not actually cause conversions to be done do not incur the
overhead.
.pp
Much of the information in this section is extracted from the
.i FEA_SD (5-ESPS)
manual section or (in Subsection 4.4) from the
.i feasd_recs (3-ESPSu)
manual section.
See the manual sections themselves for complete details.
.sh 2 "Header"
.pp
The SD manual section defines a type-specific header structure
with over 20 items,
many of which are relevant to only a few specialized programs.
They are present because the SD file type was designed before
generic header items (see
.i add_genhd (3-ESPS),
.i get_genhd (3-ESPS))
were introduced in ESPS.
In current ESPS, the preferred way of handling such specialized information
is with generic header items, which programs that need them may use and
other programs may ignore.
The FEA_SD specification calls for only two mandatory header items and one
predefined optional item.
.pp
The header of a FEA_SD file may contain the following generic header items.
The first two are always present.
The remaining one is not present in every file,
but if present has the meaning given below.
.pp
.TS
center, box, tab(;);
c | c | c | c
l | l | l | l
l | l | l | l.
Name;Size;Type;Enums
=
start_time;1 or no. of;double;NULL
 ;  channels; ;
record_freq;1;double;NULL
_
max_value;1 or no. of;double;NULL
 ;  channels; ; 
.TE
.pp
The items have the following meanings.
.ip start_time
Time corresponding to the first record in the file\*(--given in seconds.
If
.i start_time
has more than one element, each applies to a separate channel.
Separate starting times might be appropriate if, for example,
samples from the various channels were recorded in rotation,
rather than simultaneously.
.ip record_freq
The sampling frequency in Hz.
.ip max_value
This item is optional.
If it is present, no sample may exceed this value in magnitude.