convert.me

来自「speech signal process tools」· ME 代码 · 共 681 行 · 第 1/2 页

.lo
.de SM          \"macro to shrink left margin
.br
.nr $i \\n($i+\\$1i
..
.de RM          \"macro to restore original left margin
.br
.nr $i 0
..
.de MS		\"macro for main section (sans serif bold)
.ft SB
.nf
\\$1
.sp -.8
.fi
..
.de SS		\"macro for sub-section (sans serif italics)
.ft SI
.nf
\\$1
.sp -.8
.fi
..
.sc
.he 'CONVERSION APPLICATIONS NOTE''page %'
.fo '1.7'ERL8/16/91'
.nf
.ce 
\s+4\fB ENTROPIC RESEARCH LABORATORY, INC.\fR\s-4
.sp 1i
.ce 100
\s+1\fB

ESPS APPLICATIONS NOTE: Converting Data to and from ESPS FEA Files
\fP\s-1
.sp .3i
\fIJohn Shore \fP
.sp .3i
Entropic Research Laboratory, Inc.
600 Pennsylvania Ave. S.E., Suite 202
Washington, D.C. 20003
[phone]
.sp .5i
.fi
.lp
.sh 1 INTRODUCTION
.lp
ESPS and \fIwaves\fP+ are often used to display and manipulate
"foreign data" \- data that is not contained in ESPS files.  In
addition, many users process data using a combination of ESPS and
non-ESPS programs.  This applications note discusses the various ESPS
programs that are useful in this context.  The discussion includes:
.ip \ \ \ \(bu
Converting between ESPS files and pure binary or ASCII data
.ip \ \ \ \(bu
Reading headerless files without converting to ESPS
.ip \ \ \ \(bu
Reading files with foreign headers left in place
.sh 1 "RELEVANT ESPS PROGRAMS"
.lp
There are 10 or so ESPS programs that are the main utilities for
converting back and forth between ESPS files and ASCII or headerless
binary data.  All of these are included in the ESPS subset shipped
with \fIwaves\fP+.  A few other ESPS programs are relevant \- their
main function is not related to data conversion, but they can be
useful in special cases.  We shall mention all of these programs (as
usual, use \fIeman\fP to see the manual page for any of them).  Here's
the complete list (those included with \fIwaves\fP+ are marked with an
asterisk):
.sp .5
.nf
.ta 1.25i

  \fIaddfea\fP* \-	adds a new FEA file field based  on ASCII data

  \fIaddfeahd\fP* \-	adds an ESPS feature file header to non-ESPS binary or ASCII data

  \fIaddgen\fP* \-	adds a generic header item to an existing  nor new ESPS file header

  \fIbhd\fP* \-	behead an ESPS file

  \fIbtosps\fP* \-	convert binary sampled data file to an ESPS FEA_SD file

  \fIcomment\fP* \-	displays or appends to comment field in ESPS file headers

  \fIdemux\fP \-	(demultiplex) extract real or complex channels from a FEA_SD file

  \fIeman\fP* \-	displays or prints ESPS manual pages

  \fIeparam\fP* \-	run an ESPS program with parameter prompts

  \fIesps2mu\fP \-	convert an ESPS sampled data file to a headerless mu-encoded file

  \fIfea2mat\fP* \-	converts FEA to MATLAB .mat 

  \fIfea_edit\fP \-	feature file editor

  \fIfea_element\fP* \-	prints binary format table for ESPS FEA files

  \fIfea_print\fP \-	 print data from FEA records with user-controlled formatting

  \fIfeafunc\fP \-	apply function, gain factor, additive constant, and type change to FEA fields

  \fIfea_deriv\fP \-	derive a new FEA file containing elements from an existing one

  \fIhditem\fP* \-	print an item from an ESPS header

  \fIils_esps\fP* \-	convert an ILS sampled data file to an ESPS file

  \fImat2fea\fP* \-	converts from MATLAB .mat to FEA

  \fImergefea\fP \-	merge two FEA files so that output records contain fields from both

  \fImux\fP \-	multiplex sampled-data files into a single multichannel or complex file

  \fImu2esps\fP* \-	converts mu-encoded data to ESPS FEA_SD file

  \fIpplain\fP* \-	print values from ESPS file in "plain format"

  \fIepsps\fP* \-	print headers and data from ESPS file in "pretty format"

  \fIselect\fP \-	applies arbitrary queries to select records from FEA files

  \fItestsd\fP* \-	make test signal (sine wave, square wave, noise, pulses, ASCII input, etc.)

  \fIxeparam\fP \-	run an ESPS program with X Window parameter prompts

  \fIexprompt\fP \-	interactive ESPS parameter entry in a pop-up window

  \fIexpromptrun\fP \-	interactive ESPS parameter entry and program run
.sp .5
.fi
.sh 1 "STRUCTURE OF ESPS FEA FILES"
.lp
Before discussing particular cases of data conversion, it's helpful to
recall some basic information about the structure of ESPS FEA files.  
.lp
ESPS FEA files are binary files containing an arbitrary length header
followed by an arbitrary number of fixed-length data records.  The
record length varies from file to file (depending on information in
the header), but is constant within a given file.  Each record
contains information in the form of a set of independent "fields" that
can have different dimensions and numeric data types.  The names,
dimensions, and data types for these fields are the same for every
record in the file; this "self-description" information is stored in
the type-specific part of the file header.  A FEA file may optionally
be "tagged", in which case each record contains as its first element a
LONG that points to a position in a distinguished external file
(usually a sampled data file), whose name is stored in the current
file header.
.lp
Normally, the data records in FEA files are written in
\fItype_order\fP \- except for the optional tag (which always comes
first), all doubles are written first, followed by floats, longs,
shorts, and characters (bytes).  (This choice was made for efficiency,
since the records can be read or written in blocks \- one for each data type \-
and don't have to be dealt with a field at a time.)  An alternative is for
the files to be in \fIfield_order\fR, which means that the data fields
in each record appear in the order that the fields were defined.  In
most cases, the user can choose the order by means of the \s-1UNIX\s+1
environment variable FIELD_ORDER. If this variable is defined
(and does not equal \fIoff\fR), then the files written are written in
\fIfield_order\fP instead of \fItype_order\fP.  (The order is stored
in the header, so that ESPS input routines can read files in either
order).
.lp
The structure of a particular FEA file can be determined in two ways.
One way is to obtain a full listing of the file header \- 
use \fIepsps\fP \fB-vD\fP (list file with verbose information from the
header, and no data).  Another way is to run \fIfea_element\fP on the
file, which produces a table giving the structure of each record; for
example: 
.nf
.ft CW
.ta .2i 2i 3i 3.75i
    
	%fea_element test.fea
	Name	Type	Size	Element
	----	----	----	-------
	voiced_fraction	FLOAT	1	1
	raw_power	FLOAT	1	2
	lpc_power	FLOAT	1	3
	p_pulse_len	FLOAT	1	4
	spec_param	FLOAT	10	5
	frame_len	LONG	1	15
	num_pulses	LONG	1	16

.ft LR
.fi
.sh 1 "DEALING WITH BINARY DATA"
.sh 2 "Converting from ESPS Files to Headerless Binary"
.lp
If the ESPS FEA header were were absent, a non-ESPS programs could read 
and process the binary data provided that the structure of the file is
known \- i.e., if the data types data order are known.  To create
such a file is simple \- just behead it: 
.nf
.ft CW

    %bhd test.fea test.nohead

.fi
.ft LR
Furthermore, the structure of the resulting file is documented well by
the output of \fIfea_element\fP (when run on the original ESPS file).
Thus, to export a file for non-ESPS use, just supply the output of
\fIbhd\fP as data and the output of \fIfea_element\fP as documentation.  
.lp
In some cases, the structure of the required non-ESPS file is
pre-determined \- for example, you may write an ESPS program knowing
that the output data (without header) will be used as input to a
particular non-ESPS program.  In this case, by arranging the order in
which the fields are defined (i.e., by arranging the order of calls to
\fIadd_fea_fld\fP (3\-\s-1ESPS\s+1)), and by specifying that the
output file be written in \fIfield_order\fP rather than
\fItype_order\fP, you can produce FEA files that have the desired
binary format.
.sh 2 "Converting from Headerless Binary to ESPS Files"
.lp
If you have a headerless binary file with a known structure that is
compatible with the definition of an ESPS FEA file (i.e., fixed length
records, identical structure per record, etc.), a FEA file can be
created by means of the program \fIaddfeahd\fP.  This program was
designed to complement \fIfea_element\fP, in the sense that
information about the structure of the input data is provided to
\fIaddfeahd\fP in the form of an ASCII table of the same form as
output by \fIfea_element\fP.  For example, if "fields" is an ASCII
file containing the output from \fIfea_element\fP shown above (but
without the first line giving column headings), then
.nf
.ft CW

    %addfeahd fields test.nohead test.newfea

.ft LR
.fi
will produce a FEA file "test.newfea" with the same binary data that
was in "test.fea".
.lp
By default, the output of \fIaddfeahd\fP is written in \fItype_order\fP, which
probably will result in the data order within each record being
different than in the original file.  If you want to preserve the
order, define the \s-1UNIX\s+1 environment variable FIELD_ORDER and
set it to any value other than off, in which case \fIaddfeahd\fP (and
other programs) will produce the file with data in the same order as
the input file \- the resulting FEA file could be converted back into
the original format by removing the ESPS header with \fIbhd\fP.
.sh 3 "Adding Generic Header Items"
.lp
In order to use a FEA file produced by \fIaddfeahd\fP, you may need to
add some generic header items that determine the behavior of
processing programs.  For example, \fIwaves\fP+ bases the time
synchronization of cursors on a generic header item \fIrecord_freq\fP
that contains that number of FEA records per second.  These can be
added using \fIaddgen\fP, for example:
.nf
.ft CW

    %addgen -g record_freq -v 120 -t DOUBLE test.newfea

.ft LR
.fi
You may prefer to use \fIeparam\fP, which results in your being
prompted for the required information:
.nf
.ft CW

    %eparam addgen test.newfea
    Name of new generic header item [item1]: record_freq
    Type of new generic header item [long]: double
    Value of new generic header item [1000]: 120

.ft LR
.fi
If you are running X, then 
.nf
.ft CW

    %xeparam addgen test.newfea


.ft LR
.fi 
will do the same thing but use an X window dialog box for prompting. 
.lp
A third alternative for running \fIaddgen\fP is to prepare 
an ASCII parameter file containing the name, type, and value of the
desired generic (say, in file "parameters"), and then running:
.nf
.ft CW

    %addgen -P parameters test.newfea

.ft LR
.fi
Such a parameter file could be prepared with any text editor, or by
running \fIexprompt\fP on the system default parameter file for
\fIaddgen\fP (usually /usr/esps/lib/Paddgen).  (For more information
about ESPS parameters, see the technical note "Parameter and Common
Files in ESPS".)
.sh 3 "Converting Sampled Data to ESPS"
.lp
The conversion of a headerless binary sampled data file to an ESPS
FEA_SD file can be accomplished as discussed above \- use
"\fIaddfeahd\fP\fB -t\fP FEA_SD" to create the file (the \fB-t\fP is
to set the FEA_SD subtype code), and then \fIaddgen\fP to set the
sampling rate (\fIrecord_freq\fP) and starting time
(\fIstart_time\fP).  This is an important special case, however, so we
support it with a separate program, \fIbtosps\fP.  This program allows
you to specify the sampling rate, the start time, the number of
channels, and the data type of the data, and it produces a
corresponding FEA_SD file.  For example:
.nf
.ft CW

    %btosps -f 16000 -t double -T 1.2 -n 2 -c "data from run 3" sd.bare sd.feasd

.ft LR
or
.ft CW

    %eparam btosps sd.bare sd.feasd
    Enter the sampling rate of the data [8000]: 16000
    Enter the start time of the data [0]: 1.2
    Enter the number of channels [1]: 2
    Enter data type (double, float, long, short, byte, double_cplx,
            float_cplx, long_cplx, short_cplx, byte_cplx) [SHORT]: double
    Enter the number of leading bytes to skip [0]: 
    Enter a comment describing the data origin [ ]: data from run 3

.ft LR
.fi
The required comment is put in the header as an informal starting
point for ESPS record keeping.  
.sh 3 "Skipping non-ESPS Headers"
.lp
If the non-ESPS file has a non-ESPS header (i.e., it isn't a
headerless binary file), the non-ESPS header can be ignored provided
that you know its length \- both \fIaddfeahd\fP and \fIbtosps\fP have
an option (\fB-S\fP) to skip a given number of leading bytes.  
.sh 3 "Combining several non-ESPS Files"