Skip to content

efosler/eesen

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Eesen

Eesen is to simplify the existing complicated, expertise-intensive ASR pipeline into a straightforward sequence learning problem. Acoustic modeling in Eesen involves training a single recurrent neural network (RNN) to model the mapping from speech to text. Eesen abandons the following elements required by the existing ASR pipeline:

  • Hidden Markov models (HMMs)
  • Gaussian mixture models (GMMs)
  • Decision trees and phonetic questions
  • Dictionary, if characters are used as the modeling units
  • ...

Eesen was created by Yajie Miao with inspiration from the Kaldi toolkit. Thank you, Yajie!

Key Components

Eesen contains 4 key components to enable end-to-end ASR:

  • Acoustic Model -- Bi-directional RNNs with LSTM units.
  • Training -- Connectionist temporal classification (CTC) as the training objective.
  • WFST Decoding -- A principled decoding approach based on Weighted Finite-State Transducers (WFSTs), or
  • RNN-LM Decoding -- Decoding based on (character) RNN language models, when using Tensorflow (currently its own branch)

Highlights of Eesen

  • The WFST-based decoding approach can incorporate lexicons and language models into CTC decoding in an effective and efficient way.
  • The RNN-LM decoding approach does not require a fixed lexicon.
  • GPU implementation of LSTM model training and CTC learning, now also using Tensorflow.
  • Multiple utterances are processed in parallel for training speed-up.
  • Fully-fledged example setups to demonstrate end-to-end system building, with both phonemes and characters as labels, following Kaldi recipes and conventions.

Experimental Results

Refer to RESULTS under each example setup.

References

For more information, please refer to the following paper(s):

Yajie Miao, Mohammad Gowayyed, and Florian Metze, "EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding," in Proc. Automatic Speech Recognition and Understanding Workshop (ASRU), Scottsdale, AZ; U.S.A., December 2015. IEEE.

About

The official repository of the Eesen project

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 78.7%
  • Shell 10.5%
  • Perl 5.5%
  • Cuda 3.1%
  • Makefile 0.8%
  • C 0.8%
  • Other 0.6%