spspaper.vme

speech signal process tools

.lo
.de SM          \"macro to shrink left margin
.br
.nr $i \\n($i+\\$1i
..
.de RM          \"macro to restore original left margin
.br
.nr $i 0
..
.he 'INTRODUCTION TO ESPS''page %'
.fo 'Version 1.9'\\*(ep'6/18/87'
.nf
.ce 
\s+4\fB\*(ep ENTROPIC PROCESSING, INC.\fR\s-4
.sp 1i
.ce 100
\s+1\fBINTRODUCTION TO THE 

ENTROPIC SIGNAL PROCESSING SYSTEM (ESPS)\fP\s-1
.sp .3i
\fIJohn Shore \fP and \fIAlan Parker\fP
.sp .3i
Entropic Processing, Inc.
Washington Research Laboratory
600 Pennsylvania Ave. S.E., Suite 202
Washington, D.C. 20003
[phone]
.sp .5i
.fi
.sh 1 "INTRODUCTION"
.lp
The Entropic Signal Processing System (ESPS) is a set of interacting signal
processing programs.  It is a third-generation system that was developed
over a three-year period with these main goals:
.(f
\(co Copyright 1987 Entropic Speech, Inc.; All rights reserved.  
.)f
.SM .25i
.ip "\(bu" 3
to provide a useful set of easy-to-use programs for direct
support of such traditional operations as analog I/O, filtering, 
spectrum analysis, quantization, pattern classification, signal
editing, file editing, plotting, etc;
.ip "\(bu" 3
to provide a toolkit that supports the rapid development of new
signal processing programs;
.ip "\(bu" 3
to exploit \s-1UNIX\s+1\(rg\(dg tools and features (e.g., pipes)
.ip "\(bu" 3
to facilitate record keeping in such a way that ESPS files contain a
complete history of the origin of the data in the file and of all
intermediate processing steps;
.ip "\(bu" 3
to allow easy user-extensions to ESPS file types (including the
creation of new file types) in such a way that the extensions are
supported automatically by standard ESPS plotting, printing, editing,
and database programs;
.RM
.(f
\(dg \s-1UNIX\s+1 is a trademark of AT&T Bell Laboratories
.)f
.lp
ESPS currently includes approximately 50 user-level programs (called
from the \s-1UNIX\s+1 shell) for file manipulation, sampled-data I/O,
plotting, and signal processing operations.  Also included is a
library of over 130 functions that support the modification of
existing programs and the development of new programs.  The ESPS
library contains functions that perform signal processing primitives
as well as functions that support the creation and manipulation of
ESPS files.  ESPS provides 5 built-in file types; it also provides the
ability to modify these file types and to create new file types.
.lp
The ESPS library was developed with disciplined software engineering
methods, including information-hiding.  The library functions 
perform extensive error-checking, and a complete LINT library is
included. 
.lp
ESPS main programs and most of the library functions are written in C.
However, the ESPS library also supports \s-1FORTRAN\s+1 so that new
signal processing operations can be written either in C or
\s-1FORTRAN\s+1.
.sh 1 "ESPS DOCUMENTATION"
.lp
The principal documentation for ESPS is the ESPS Users Manual.  This
contains \s-1UNIX\s+1\-style manual pages for all programs, library
functions, and file types.  The ESPS Manual is organized in the usual
\s-1UNIX\s+1 style, with Section 1 describing user-level programs,
Section 3 describing the ESPS library, and Section 5 describing ESPS
file formats.  Section 3 is further divided into two sections, one
for signal processing functions and one for ESPS utility functions.
References to ESPS programs, library functions, and file types
usually include a parenthetical pointer to the section of the ESPS
Manual in which the manual page is located.  For example,
"(3\-\s-1ESPS\s+1)" indicates Section 3 of the ESPS Manual.
.lp
The entire ESPS Manual is available on-line and can be viewed with the
.i eman
(1\-\s-1ESPS\s+1) program, which operates just like the \s-1UNIX\s+1 
.i man
(1) program (including the \fB\-k\fP option to search for programs
whose descriptions contain a given keyword).  
.lp
The ESPS documentation also includes a series of technical memoranda
that explain ESPS data files, ESPS parameter files, and ESPS common
files.  Also provided are ESPS programming guidelines.  
.sh 1 "ESPS FILES"
.lp
ESPS files have a common structure consisting of a variable-length
header followed by data records.  Within a single ESPS file, the
records have a fixed-length that depends on information in the
header.  Record-lengths vary, however, even among files of the same
basic type.  
.lp
The ESPS header is the primary means by which record-keeping
is performed within ESPS.  An ESPS header includes the values of all
important parameters to the program that created the ESPS file.  It
also includes the headers of any files that were inputs to the
program that created the file.  Since the header is a recursive
structure, these included headers themselves may contain the headers
of previous source files.  It follows that most ESPS files contain a
complete history of the origin of the data in the file and all the
intermediate processing steps.  In addition to this formal
record-keeping mechanism, informal comments can be added to any ESPS
file header by means of
.i comment
(1\-\s-1ESPS\s+1).  
.lp
ESPS file headers and data records are viewed in ASCII form by
means of the program
.i psps
(1\-\s-1ESPS\s+1), which includes options for viewing recursively all of the 
embedded headers.  The output from 
.i psps
identifies values from header and data-records fields by name. Within
programs, the fields are also referred to by name.  
.lp
On some systems, ESPS files can be created directly from analog
inputs and played out over digital to analog converters.  Also,
conversion programs are available for ASCII files,
\s+1ILS\s-1\(rg\(dg
.(f
\(dg \s-1\ILS\s+1 is a trademark of Signal Technology, Inc.
.)f
sampled-data files, and \s-1MASSCOMP\s+1\(rg Laboratory Workbench
(LWB)\(dd files.  
.(f
\(dd \s-1MASSCOMP\s+1 and \s-1LWB\s+1 are trademarks of Massachusetts 
Computer Corporation.
.)f
.sh 2 "Built-in File Types
.lp
ESPS contains the following built-in file types:
.SM .25i
.ip "SD" 8
contains sampled data
.ip "SPEC" 8
contains spectra
.ip "FILT" 8
contains digital filters
.ip "SCBK" 8
contains scalar quantization codebooks
.ip "FEA" 8
contains arbitrary features
.RM
.lp
Except for FEA files, which are discussed further below, these
built-in files have a predefined record structure.  That is, their
data records comprise fields with pre-defined names and data
types.  The definitions are suitable for the types of data stored 
in these built-in file types.  Similarly, the headers of the built-in 
types contain pre-defined fields for relevant information.  However, 
any program can add additional header fields through the use of 
generic header items (see Section 6.1.3).  
.lp
Section 5 of the ESPS Manual contains detailed information about what
is stored in the various ESPS file types.
.sh 2 "Feature Files and User-Defined File Types"
.lp
The FEA file type is a general purpose file type that can be used to
store arbitrary information in fixed-length records; optionally, the
records can point to positions in various source files.  Unlike the
case for the other ESPS file types \- the names, sizes, and data types
of the fields in FEA records are not pre-defined.  Because of this,
and because of the ways in which FEA files are supported by standard
ESPS file manipulation and plotting programs, FEA files can be used to
create user-defined ESPS files that have all of the advantages of the
built-in file types.  This can be done by using the ESPS library
whether or not ESPS source is available.
.lp
When a new file type is implemented using FEA files, it is referred to 
as a FEA file sub-type.  Several such implementations are provided
with ESPS, including the following:
.SM .25i
.ip "FEA_ANA"
suitable for speech processing; contains spectral parameters (various
representations), pulse lengths, and powers;
.ip "FEA_VQ"
contains vector quantization codebooks;
.ip "FEA_STAT"
contains statistical information (means, covariances, etc.) suitable
for pattern-classification;
.RM
.lp
By using access functions provided with the ESPS Library, users can
implement their own FEA file sub-types.  They can also extend
existing FEA file types, like those mentioned above, by adding
whatever additional record fields are desired.
.lp
The power of FEA files is enhanced by the existence of ESPS programs like
the following, which work on any FEA file (including user-defined or
user-extended FEA files):
.SM .25i
.ip \fIpsps\fP(1\-\s-1ESPS\s+1)
produces ASCII representation of any ESPS file (including headers), 
with all fields referred to by name;
.ip \fIgenplot\fP(1\-\s-1ESPS\s+1)
plots information from FEA file records in precise, multi-line
format;
.ip \fIfea_edit\fP(1\-\s-1ESPS\s+1)
permits you to edit an ASCII representation of a FEA file using the
editor of your choice; Thus, you can modify the data in an existing 
FEA file or create a new file with arbitrary data.  Both file headers 
and data records can be edited with \fIfea_edit\fP.  
.ip \fIselect\fP(1\-\s-1ESPS\s+1) 
treats ESPS files as a database and allows interactive selection of
records that satisfy user-supplied queries; selected records can be
output to another FEA file; Here are some examples of queries on an
input FEA_ANA file:
.nf

\->select (spec_param[0] > .96) && (frame_type == "VOICED")
	123 records selected

\->select ((spec_param[0] + spec_param[1]) > 1.5) && (frame_len > 100)
	21 records selected

\->select (spec_param[0] > spec_param[1]) && (log(sqrt(raw_power[0])) > 7)
	13 records selected

\->select (mean(raw_power) > 1.75E10) || (raw_power[0] > 1.75E10)
	100 records selected

.fi
Here, "spec_param", "frame_type", "frame_len", and "raw_power" are
the names of fields in a FEA_ANA file.  Queries can be constructed
using simple relational expressions involving the elements in FEA
records, built-in arithmetic operators (*,+,\-,/,^), built in
functions (sin, cos, tan, atan, log, log10, exp, sqrt), and
user-defined functions implemented as external programs.  As
mentioned above, \fIselect\fP works on user-defined and user-extended
FEA files.
.ip \fIfea_stat\fP(1\-\s-1ESPS\s+1)
Computes means, covariances, eigenvalues, eigenvectors, and other
statistics from the records in a FEA file; outputs a FEA_STAT file;
.ip \fIclassify\fP(1\-\s-1ESPS\s+1)
performs classification of FEA file records using information in 
FEA_STAT files;
.RM
.lp
Another useful aspect of FEA files is a mechanism called "segment
labelling", which makes it easy to associate FEA file records with
positions or ranges in sampled-data files.  If a FEA file is declared
to be segment-labelled, then its records automatically include file
name, starting position, and length fields (these need to be filled
in by user programs, but the field support is there).  This mechanism
could be used, for example, to support a database of
phonetically-labelled sampled data files by implementing a FEA file
containing a phoneme code (or string) in each segment-labelled
record.  Then \fIselect\fR(1\-\s-1ESPS\s+1) could be used to process
sets of these files.  For example, the \fIselect\fP command
.nf

	\->select phoneme == "AW" || phoneme == "OW"

.fi
would yield records that provide file names, starting positions, and
lengths for all occurrences of the two named phonemes.  
.sh 1 "ESPS PROGRAMS AND \s-1UNIX\s+1 PIPES"
.lp
So that pipes can be used, most ESPS programs are written so that they
use standard input and output as well as specific files.  As an example,