feasd.doc

speech signal process tools

                         ENTROPIC RESEARCH LABORATORY, INC.


       TECHNICAL MEMORANDUM

       SUBJECT: FEA_SD Files---Feature Files for Sampled Data in ESPS

       ID: ETM-S-89-49:rwj

       VERSION: 1.3

       DATE: 1/22/93

       AUTHOR: Rodney W. Johnson

       1 .  INTRODUCTION

            Originally ESPS used a special file type for storing sampled sig-
       nal data.  This file type, the SD (``sampled-data'') file  is  one  of
       several  specialized  file types that have been replaced with subtypes
       of the ESPS FEA (``feature'') file type.  The new FEA subtype for sam-
       pled  data,  the  FEA_SD  file,  is the subject of this note.  See the
       FEA_SD(5-ESPS) and FEA(5-ESPS) manual sections for more details on the
       FEA_SD file type.

            With the change from SD to FEA_SD, various new facilities  become
       available for use with sampled-data files.  In addition to the general
       facilities that apply to all FEA files,  these  include  complex  data
       types (now in fact supported for FEA files in general) and support for
       multichannel sampled data.  The new facilities  are  detailed  in  the
       next section.

            The third section mentions some user-level programs  that  create
       FEA_SD files or accept them as input.  These include programs for con-
       verting between the SD and FEA_SD file types and programs  for  multi-
       plexing  and demultiplexing multichannel FEA_SD files.  Actually  most
       programs that formerly created or accepted SD files now  deal  in  the
       same  way  with at least one-channel real FEA_SD files.  The ones men-
       tioned explicitly in this section are the ones that so far  have  been
       significantly  extended  to  deal  with  complex or multichannel data.
       Others that could usefully apply to complex or multichannel data  will
       be added to the list.

            Section 4 gives an overview of the format of a FEA_SD  file,  the
       special  record  structure  used  for  holding  FEA_SD sampled data in
       memory, and the support functions provided for setting up FEA_SD  file
       headers and record structures and reading and writing FEA_SD files.


       ____________________
          c
            Copyright 1993 Entropic Research Lab, Inc. All rights reserved.











       ETM-S-89-49:rwj                                                 page 2


            In most cases, existing SD files will not have to be converted to
       FEA_SD,  although  of  course they could be converted with the program
       provided for that purpose.  Programs that  accepted  SD  files  before
       FEA_SD  was  introduced  will  continue to accept SD files even though
       they now accept FEA_SD files as well.  Steps described in the  Section
       5  have been taken to assure backward compatibility.  First, they make
       it easy to convert an existing program from SD to FEA_SD input without
       sacrificing  the  ability  to accept SD files.  Secondly, they make it
       essentially automatic that new programs that take  FEA_SD  input  will
       also accept SD files.

            Section 6 of this note is a guide to conversion of existing  pro-
       grams  so that they will use FEA_SD files instead of or in addition to
       SD files.

       2 .  NEW FACILITIES

       2 .1 .  General Feature-File Facilities

            An ESPS FEA file consists of a header followed by a  sequence  of
       records,  each  of which consists of a number of named ``fields'' that
       may hold various types of numeric or nonnumeric data or arrays of such
       data.   A  field  is  characterized  by its name, data type, and array
       dimensions.  In a FEA_SD file there is a single field that  holds  all
       the  sampled  data.   The  field  is  named "samples", it can have any
       numeric data type, and it has one dimension: the number  of  channels.
       There  needn't  be  any other fields.  Each record contains one sample
       from each channel; thus in a  single-channel  file,  each  record  may
       comprise merely a single sample.

            FEA files are self-describing: the header of a FEA file  contains
       a  list  of  its  fields with their names, data types, dimensions, and
       other  information.   Powerful  programs  such  as   fea_edit(1-ESPS),
       select(1-ESPS),  and  fea_stats(1-ESPS)  can  operate on arbitrary FEA
       files by using the header information.  These  programs  all  can  now
       accept  FEA_SD  files; to them, a FEA_SD file is just another FEA file
       that happens to have a field named "samples".  Thus you can  now  edit
       an ASCII representation of a sampled-data file with your favorite text
       editor, run complex queries against the contents  of  such  files,  or
       compute inverse covariance matrices for multichannel sampled data.

            New fields can be added  to  any  FEA  file---see  add_fea_fld(3-
       ESPSu) and addfea(1-ESPS).  FEA_SD files can thus be created that con-
       tain other fields than "samples".  When  such  additional  fields  are
       present, programs that read FEA_SD files will still work, ignoring the
       extraneous data.  However, FEA_SD files with extra fields may be  read
       less  efficiently  than  ``pure'' FEA_SD files, since the extra fields
       complicate the record structure.

       2 .2 .  Complex Data Types

            FEA_SD files were introduced into ESPS at the same  that  complex
       data  types  were  introduced  for  general  FEA  files.  As a result,


       Version 1.3                      ERL                           1/22/93







       ETM-S-89-49:rwj                                                 page 3


       sampled-data files with complex data types are now possible.  Formerly
       just  5  ESPS data types were possible for a sampled-data file:  the 5
       real numeric types.  Now there are 5 more.   For  every  real  numeric
       type  there is now a corresponding complex type, consisting of complex
       numbers whose real and imaginary parts are of  that  real  type.   For
       example  the  complex  type  corresponding  to  the real type short is
       defined by the C typedef

            typedef struct {short   real, imag;}    short_cplx;

       If z is a variable of this type, its  real  and  imaginary  parts  are
       z.real and z.imag, which are variables of type short.

            The complete list of numeric types  is  shown  in  the  following
       table.  A FEA_SD file can be of any one of these types.


                            ___________________________
                            |___code_____|____type_____|
                            |DOUBLE      | double      |
                            |FLOAT       | float       |
                            |LONG        | long        |
                            |SHORT       | short       |
                            |BYTE        | signed char |
                            |DOUBLE_CPLX | double_cplx |
                            |FLOAT_CPLX  | float_cplx  |
                            |LONG_CPLX   | long_cplx   |
                            |SHORT_CPLX  | short_cplx  |
                            |BYTE_CPLX   | byte_cplx   |
                            |____________|_____________|


       (The symbols in caps, sometimes loosely referred  to  as  ESPS  types,
       actually  stand  for  numeric  codes  defined  in  a header file.  The
       lower-case forms are the ones used in writing declarations in  C  pro-
       grams.  See FEA(5-ESPS) for more information.)

       2 .3 .  Multichannel Data

            In a sense the SD file type provided  for  multichannel  data  in
       that  the  type-specific part of the SD header had a member nchan that
       was supposed to indicate the number of channels.  But no programs were
       provided  to  support  multichannel data, and probably no ESPS program
       ever set nchan equal to anything but 1 or 0 (which defaults to 1).  In
       a  FEA_SD  file,  as  noted  above, the number of channels is the size
       (number of items) of the field "samples".  There are now ESPS programs
       for multiplexing and demultiplexing (see Section 3.2).  These can form
       a multichannel FEA_SD  file  by  multiplexing  several  single-channel
       files,  extract channels from a multichannel file, and perform various
       other similar operation.





       Version 1.3                      ERL                           1/22/93







       ETM-S-89-49:rwj                                                 page 4


       3 .  PROGRAMS THAT READ OR WRITE SAMPLED-DATA FILES

       3 .1 .  File Conversion

            Two programs are available for converting between SD  and  FEA_SD
       files.   The  program  sdtofea(1-ESPS)  reads  an SD file and writes a
       FEA_SD file containing the same data and (as nearly as  possible)  the
       same  header  information.   The  program featosd(1-ESPS) performs the
       inverse transformation (again, as nearly as possible).  See the manual
       sections for detailed information.

       3 .2 .  Multiplexing and Demultiplexing

            The program mux(1-ESPS) can multiplex several  single-channel  or
       multichannel FEA_SD files into a single FEA_SD file with as many chan-
       nels as all the input files taken together.  It can also join pairs of
       real input channels into single complex output channels.

            The program demux(1-ESPS) can extract a subset  of  the  channels
       from  its  input  file  and either put them in separate single-channel
       output files or form a single output file with fewer channels than the
       input.   It  can  also split complex input channels into pairs of real
       output channels.  See the manual sections for these programs for  more
       complete  statements  of  the exact capabilities of these programs and
       for examples of their use.

       3 .3 .  Other Programs

            Most ESPS programs that formerly expected SD files as input  will
       now  accept  at least single-channel, real FEA_SD files as well.  Most
       ESPS programs that formerly produced SD files as output  will  now  at
       least  produce  equivalent  single-channel, real FEA_SD files instead.
       Many ESPS programs can reasonably be extended to perform useful opera-
       tions  that involve reading or writing complex or multichannel sampled
       data.  Now that the basic infrastructure for complex and  multichannel
       support  is  in  place, these extensions are being done.  The programs
       listed here are just the ones that have at this writing been  extended
       with  support  or  partial support for complex or multichannel sampled
       data.

            testsd will now create single-channel FEA_SD files of any numeric
       data  type,  including  the  complex  ones.   It  can generate complex
       exponentials, which may be  damped  or  frequency-swept,  and  complex
       noise  sequences  with  independent  real and imaginary parts.  It can
       also create a FEA_SD file containing arbitrary complex data  given  by
       numbers in an ASCII text file.

            fft will now compute fast Fourier  transforms  of  single-channel
       complex input data.

            fftinv will now create  single-channel  complex  output  data  as
       results of the inverse fast Fourier transform.



       Version 1.3                      ERL                           1/22/93







       ETM-S-89-49:rwj                                                 page 5


            addsd and multsd will add or multiply single-channel FEA_SD files
       sample  by  sample.  The input files may have arbitrary numeric types,
       including complex.

            copysd will handle single-channel FEA_SD files  of  any  real  or
       complex type.

            psps and pplain will print both real and complex FEA_SD files (as
       well  as  other file types).  In psps output, complex numbers are for-
       matted as bracketed pairs of real numbers.  In pplain output the  real
       and  imaginary  parts  are simply printed in order as consecutive real
       numbers.

       4 .  FEA_SD FILE STRUCTURE AND SUPPORT MODULE

            This section describes the structure of the header  of  a  FEA_SD