Speech Recognition with CMU Sphinx and SRILM language models

Speech recognition that does not take advantage of domain specific language models produces silly results.  The recognition menagerie below was generated with Sphinx 4.  I used the HUB4 acoustic model and an SRILM language model generated from the human annotator transcripts.

  • No LM: thusly islam brown ivan and apelike of yellow
  • LM: a seated small brown ottoman and teapot of yellow
  • True: i see a small brown ottoman and a pot of yellow


  • No LM: m wantonly dearly tightens several nameplates
  • LM: room what appeared the deer with wings pans several nameplates
  • True: what appeared to be deer with wings on it and several name plates

  • No LM: it’s around a hesitant carlton’s fermented intellectual
  • LM: it’s brown a has arms crossed and front entertainment actually
  • True: it’s brown and has its arms crossed in front of it and standing next to a wall

All of these examples are handpicked; while the non-language model case is bad (WER is 91.6%), the simple language model isn’t great (WER is 68.2%).  Also, I’m cheating by training and testing the language model on the same data; this probably accounts for a lot of the improvement in these longer utterances.  Still a lot of work to do!

Tags: , ,

2 Responses to “Speech Recognition with CMU Sphinx and SRILM language models”

  1. dhd Says:

    what is your corpus here? looks like phonetically balanced (and semantically strange) sentences – ASR has a lot of trouble with these in general

    • romanows Says:

      It’s a human-computer task-oriented dialogue called Treasure. The strange terms are “interesting” objects in an environment the speaker navigates; I don’t think there was any particular effort to make the corpus phonetically balanced.

      As for the problems that ASR has with this type of dialogue, my work currently involves combining language models and eye gaze information to improve recognition rates.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: