results.txt

It is the Speech recognition software. It is platform independent. To execute the source code,

#
# results.txt
#

#
# NOTE:
#
# In following "ti46" refers to:
#
# "NIST CD-ROM Version of the Texas Instruments-developed 46-Word
# Speaker-Dependent Isolated Word Speech Database"
#
# "tidigits" refers to the:
#
# "NIST CD-ROM Version of the Texas Instruments-developed Studio Quality
# Speaker-Independent Connected-Digit Corpus".
#
 
#
# This file contains summary of benchmarks for Sphinx4. The format of
# this file is designed to it is parseable by report generator
# programs, and yet still readable by humans with out assistance from
# external programs.  The format of the file is as follow:
# Comments start with a '#' followed by a space
#
# SystemName:	the name (or description) of the system where the test
#		 was run.
# Program:	The program tested (typically sphinx3 or sphinx4)
# Date:		The date the test was run
# Test:		The name of the test (ti46 etc)
# Who:		The name of the tester
#
# statistics as C+P from the run.
#
# comments on the test, multiple lines ok, each line starts with -A

# Formatting rules:
#
#  - no spaces before the colons
#  - always have at least one comment - and the comment should be
#     meaningful.
#  - be consistent with the previous entries
#
#    Also, the statistics fields should not be used as comments. I
#    don't think we need to record these fields until we start looking
#    at optimizing them. At that point, don't include them as comments
#    but as full fledged fields. Since we are parsing fields by name,
#    its ok to add new fields, just as long as they have unique field
#    names.
#
#    comment lines begin with '#'


# ===== system descriptions ==========

#                      (kbytes) (mhz)  (MBytes)
# system name 	CPUS 	CACHE 	CLOCK 	MEMORY ARCHITECTURE 	OS
		
system 	argus   2 	4096 	360 	512 	sparcv9 	solaris  
system 	mute 	1 	2048 	296 	128 	sparcv9 	solaris 
system 	mangueira 2 	8192 	750 	1024 	blade1000	solaris
system  sunlabs	8	4096 	336	4096	E3500 		solaris
system 	pharynx	1 	?  	450 	256 	pentium-3	linux 	
system  boteco	1	?  	700	256	pentium-3	MS-Win2000
system  debris	8	?  	750	32768	UltraSPARC-III  solaris-5.9

# ==================================================
SystemName:  	mangueira
Program: 	sphinx4
Date:		5/30/2002
Test:		ti46
Who:		egouvea

   Accuracy: %95.7   Errors: 102  (Rec: 95  Ins: 0  Del: 7)
   Sentences: 2382      Words: 2382   Matches: 2280
   Total Time Audio: 2567.2s  Proc: 1648.9s  Speed: 0.6 X real time
   Mem Total: 19.43 Mb Free: 6.10 Mb Used: 13.33 Mb

- First test collected by Evandro on the reference machine


# ==================================================
SystemName: 	pharynx
Program:	sphinx4
Date:		5/30/2002
Test:		ti46
Who:		egouvea

   Accuracy: %95.7   Errors: 102  (Rec: 95  Ins: 0  Del: 7)
   Sentences: 2382      Words: 2382   Matches: 2280
   Total Time Audio: 2567.2s  Proc: 3845.8s  Speed: 1.5 X real time
   Mem Total: 14.78 Mb Free: 1.42 Mb Used: 13.36 Mb

- run on linux

# ==================================================
SystemName:	argus
Program:	sphinx3
Date:		5/31/2002
Test:		ti46
Who:		pkwok

   Accuracy: %97.103271   Errors: 69
   Sentences: 2382      Words: 2382   Matches: 2313
   Total Timing Audio: 1611.0s  Proc: 932.6s  Speed: 0.578912 X real time
   Memory usage data unavailable

 - Running the TI-46 test set against Sphinx 3 with the
 - isolated digits model data supplied by Rita Singh

# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		5/31/2002
Test:		ti46
Who:		plamere

   Accuracy: %95.1   Errors: 116  (Rec: 109  Ins: 0  Del: 7)
   Sentences: 2382      Words: 2382   Matches: 2266
   Total Time Audio: 2567.2s  Proc: 3074.7s  Speed: 1.2 X real time
   Mem Total: 17.02 Mb Free: 5.12 Mb Used: 11.90 Mb

- after refactor of the ActiveList stuff, using a RelaxedFixActiveList
- Note that there is an error in the list that may be reducing
- accuracy that will be fixed soon

# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		5/31/2002
Test:		ti46_quick
Who:		plamere

   Accuracy: %98.4   Errors: 1  (Rec: 1  Ins: 0  Del: 0)
   Sentences: 62      Words: 62   Matches: 61
   Total Time Audio: 67.3s  Proc: 90.8s  Speed: 1.3 X real time
   Mem Total: 16.35 Mb Free: 6.72 Mb Used: 9.64 Mb

- a quick test to check the reformatting, and also to serve as a
- baseline for ti46_quick


# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		5/31/2002
Test:		ti46_quick
Who:		plamere

   Accuracy: %98.4    Errors: 1  (Rec: 1  Ins: 0  Del: 0)
   Sentences: 62      Words: 62   Matches: 61
   Total Time Audio: 68.9s  Proc: 86.9s  Speed: 1.3 X real time
   Mem Total: 14.27 Mb Free: 7.11 Mb Used: 7.16 Mb
   NIST WORD ACCURACY=  98.387% (   61/   62)  ERRORS=  1.613% (    1/ 62)

- Better calculation of average score in the RelaxedFitActiveList
- purge routine

# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		5/31/2002
Test:		ti46
Who:		plamere

   Accuracy: %96.9    Errors: 75  (Rec: 71  Ins: 0  Del: 4)
   Sentences: 2382      Words: 2382   Matches: 2307
   Total Time Audio: 2628.2s  Proc: 6960.4s  Speed: 2.6 X real time
   Mem Total: 31.38 Mb Free: 15.37 Mb Used: 16.01 Mb

- Increased the activelist target size from 1500 to 4500 to see if we
- could meet the accuracy of sphinx3, without regard to processing time.
- still slightly less accurate than sphinx3, and quite a bit slower


# ==================================================
SystemName:	argus
Program:	sphinx3
Date:		6/3/2002
Test:		ti46
Who:		pkwok

   Accuracy: %97.1   Errors: 69
   Sentences: 2382      Words: 2382   Matches: 2313
   Total Timing Audio: 2571.6s  Proc: 953.9s  Speed: 0.37 X real time
   Memory usage data unavailable

- Fixed audio time problem due to integer division rounding.
- As you can see, the "Total Timing Audio: 2571.6s" is close enough
- to that of Sphinx 4.


# ==================================================
SystemName:     argus
Program:        sphinx4
Date:           6/3/2002
Test:           ti46
Who:            pkwok

   Accuracy: %94.9    Errors: 121  (Rec: 114  Ins: 0  Del: 7)
   Sentences: 2382      Words: 2382   Matches: 2261
   Total Time Audio: 2571.9s  Proc: 3282.7s  Speed: 1.3 X real time
   Mem Total: 18.86 Mb Free: 4.12 Mb Used: 14.73 Mb

- edu.cmu.sphinx3.frontend.keepAudioReference set to true
- Fixed audio time problem: this should be the correct total audio time
- When the AcousticScorer encounters an UTTERANCE_START, it immediately
- reads the next Feature to score it. Its odd that this causes a 0.2%
- drop in accuracy.

# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		6/3/2002
Test:		ti46_quick
Who:		plamere

# Name          Count   CurTime   MinTime   MaxTime   AvgTime   TotTime
batchAudioSourc 426     0.0000s   0.0000s   0.0840s   0.0007s   0.2900s   
FrontEnd        6797    0.0000s   0.0000s   0.7320s   0.0007s   4.4240s   
Preemphasizer   426     0.0000s   0.0000s   0.0560s   0.0002s   0.0900s   
HammingWindow   426     0.0000s   0.0000s   0.1270s   0.0010s   0.4080s   
FFT             6803    0.0000s   0.0000s   0.0750s   0.0003s   2.1030s   
MelFilter       6803    0.0000s   0.0000s   0.0310s   0.0000s   0.2100s   
MelCepstrum     6803    0.0000s   0.0000s   0.0350s   0.0001s   0.3820s   
FeatureExtracto 311     0.0000s   0.0000s   0.0160s   0.0003s   0.0960s   
AM_Load         1       5.9870s   5.9870s   5.9870s   5.9870s   5.9870s   
LingustLoading  1       0.2970s   0.2970s   0.2970s   0.2970s   0.2970s   
Score           6735    0.0000s   0.0000s   0.7430s   0.0045s   30.3560s  
Prune           6673    0.0000s   0.0000s   0.0670s   0.0002s   1.5470s   
Grow            6735    0.0080s   0.0000s   0.3320s   0.0090s   60.6420s  
Decode          62      1.7350s   0.8920s   3.7290s   1.5074s   93.4600s  
# ------------- Summary statistics ----------- 
   Accuracy: %98.4    Errors: 1  (Rec: 1  Ins: 0  Del: 0)
   Sentences: 62      Words: 62   Matches: 61
   Total Time Audio: 67.5s  Proc: 93.5s  Speed: 1.4 X real time
   Mem Total: 16.06 Mb Free: 7.49 Mb Used: 8.57 Mb

- Added timing info. Interesting to see that the bulk of the time is 
- in the Grow and the Score code

# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		6/3/2002
Test:		ti46_quick
Who:		plamere

# ----------------------------- batch-----------------------------------------
# Name          Count   CurTime   MinTime   MaxTime   AvgTime   TotTime   
batchAudioSourc 426     0.0000s   0.0000s   0.0120s   0.0003s   0.1200s   
FrontEnd        6797    0.0020s   0.0000s   0.7080s   0.0006s   3.9720s   
Preemphasizer   426     0.0000s   0.0000s   0.0030s   0.0001s   0.0340s   
HammingWindow   426     0.0000s   0.0000s   0.0080s   0.0006s   0.2350s   
FFT             6803    0.0000s   0.0000s   0.1010s   0.0003s   2.0720s   
MelFilter       6803    0.0000s   0.0000s   0.0320s   0.0000s   0.2480s   
MelCepstrum     6803    0.0000s   0.0000s   0.0710s   0.0001s   0.4380s   
FeatureExtracto 311     0.0000s   0.0000s   0.1110s   0.0006s   0.1920s   
AM_Load         1       7.4380s   7.4380s   7.4380s   7.4380s   7.4380s   
LingustLoading  1       0.1820s   0.1820s   0.1820s   0.1820s   0.1820s   
Score           6735    0.0020s   0.0000s   0.7190s   0.0042s   28.2650s  
Prune           6673    0.0000s   0.0000s   0.1960s   0.0002s   1.2560s   
Grow            6735    0.0050s   0.0000s   0.4670s   0.0063s   42.4650s  
Decode          62      1.7840s   0.6070s   3.3940s   1.1730s   72.7250s  
#  ------------- Summary statistics ----------- 
   Accuracy: %98.4    Errors: 1  (Rec: 1  Ins: 0  Del: 0)
   Sentences: 62      Words: 62   Matches: 61
   Total Time Audio: 67.5s  Proc: 72.7s  Speed: 1.1 X real time
   Mem Total: 14.59 Mb Free: 6.52 Mb Used: 8.07 Mb

- replaced ArrayList iterator with get(index) call
- a micro-optimization, but reduces times significantly

# ==================================================
SystemName:	argus
Program:	sphinx4
Date:		6/3/2002
Test:		ti46
Who:		pkwok