%======================================================================
%----------------------------------------------------------------------
% XX X
% X
% XX XXX XXX XXX XXX X XXXX
% X X X X X X X X X X X
% X XXXXX XXXXX XXXXX X X XXX
% X X X X X X X X
% XXX XXX XXX XXX XXX X XXXX
%----------------------------------------------------------------------
% SPECIFICATION FOR COMMON IEEE STYLES
%----------------------------------------------------------------------
% Gregory L. Plett, Istv\'{a}n Koll\'{a}r.
%======================================================================
\documentclass[%
%draft,
%submission,
%compressed,
final,
%
%technote,
%internal,
%submitted,
%inpress,
reprint,
%
%titlepage,
notitlepage,
%anonymous,
narroweqnarray,
inline,
twoside,
invited,
]{ieee}
\newcommand{\latexiie}{\LaTeX2{\Large$_\varepsilon$}}
%\usepackage{ieeetsp} % if you want the "trans. sig. pro." style
%\usepackage{ieeetc} % if you want the "trans. comp." style
%\usepackage{ieeeimtc} % if you want the IMTC conference style
% Use the `endfloat' package to move figures and tables to the end
% of the paper. Useful for `submission' mode.
%\usepackage {endfloat}
% Use the `times' package to use Helvetica and Times-Roman fonts
% instead of the standard Computer Modern fonts. Useful for the
% IEEE Computer Society transactions.
%\usepackage{times}
% (Note: If you have the commercial package `mathtime,' (from
% y&y (http://www.yandy.com), it is much better, but the `times'
% package works too). So, if you have it...
%\usepackage {mathtime}
% for any plug-in code... insert it here. For example, the CDC style...
%\usepackage{ieeecdc}
\usepackage{graphicx}
\begin{document}
%----------------------------------------------------------------------
% Title Information, Abstract and Keywords
%----------------------------------------------------------------------
\title[SHARIKA-Malayalam Speech Recognition System]{%
SHARIKA \\ Malayalam Speech Recognition System}
% format author this way for journal articles.
% MAKE SURE THERE ARE NO SPACES BEFORE A \member OR \authorinfo
% COMMAND (this also means `don't break the line before these
% commands).
%\author[Shyam]{Shyam.K},\authorinfo{Shyam is with the Department of Electronics And Communication
% Engineering, MES College of Engineering,Kuttipuram
% Phone: $+$919946787953\,0494-2667303, e-mail: shyam.karanattu@gmail.com}
%}
% format author this way for conference proceedings
\author[SHYAM]{%
Shyam.K\authorinfo{%
Department of Electronics and Communication Engineering,\\
MES College of Engineering,Kuttipuram,pin 679573.\\
Phone:$+$919946787953,\,0494-2667303,email:shyam.karanattu@gmail.com}%s
}
%\and{}and%
%\and{}Istv\'{a}n Koll\'{a}r\member{Fellow}\authorinfo{%
%Department of Measurement and Instrument Engineering,\\
%Technical University of Budapest, 1521 Budapest, Hungary.\\
%Phone: $+$\,36\,1\,463-1774, fax: +\,36\,1\,463-4112,
%email: kollar@mmt.bme.hu}
%}
%\journal{ICIST Trans.\ on Instrum.\ Meas.}
%\titletext{, VOL.\ 46, NO.\ 6, DECEMBER\ 1997}
%\ieeecopyright{0018--9456/97\$10.00 \copyright\ 1997 IEEE}
%\lognumber{xxxxxxx}
%\pubitemident{S 0018--9456(97)09426--6}
%\loginfo{Manuscript received September 27, 1997.}
%\firstpage{1217}
%\confplacedate{Ottawa, Canada, May 19--21, 1997}
\maketitle
\begin{abstract}
Sharika - an Automated Speech Recognition(ASR) system is an initial
initiative in malayalam speech recognition.ASR in its wider perspective
is still a dream project,in malayalam.Though the speech corpus required
for building the models are ready built,we cannot make use of those corpus
as those are given under non-sharable licence.Since ASR in malayalam in
its true sense is a herculean time consuming project,here our endeavour
is to make a pilot project limiting the word count by confining the ASR
to a particular task namely voice control for desktop commands.
\end{abstract}
\begin{keywords}
Automatic Speech Recognition,Sharika,CMUSphinx,Free Software,Malayalam.
\end{keywords}
%----------------------------------------------------------------------
% SECTION I: Introduction
%----------------------------------------------------------------------
\section{Introduction}
\PARstart Among the most ancient human communication systems,
verbal form of communication is the most scientific mode.
When we go deep in to the human history,we can find that
vocal communication was effective far before written methods.
In this most modern age also one cannot underestimate the
effectiveness of speech as a mode of communication.
If our computer can occupy the place of a stenographer,
read news according to our choice,can teach students in their
own mother tongue,it will be a major break through.We leave the
practical applications of speech recognition to the imagination
of the readers as it is very vast.This effort becomes invaluable
for those with physical inabilities.Our endeavour here is to minimize
the digital divide by making computer applications more user
friendly-just by spelling it! that too in our mother tongue.
%----------------------------------------------------------------------
% SECTION II: The Document Life-Cycle
%----------------------------------------------------------------------
\section{MALAYALAM}
Malayalam belongs to the Dravidian family of languages and
is one of the four major languages of this family with a rich
literary tradtion. Malayalam is the official language of the
state of kerala ,India.Its spoken by around 37million people.
Indian language at large and malayalam in particular are very
phonetic in nature.The structure of alphabets in malayalam is
arranged based on the method of sound production.As we pass
through the consonant list ,we can see that its so arranged
that the noice source of consonant sound, moves from inner mouth
to lips.
\begin{figure}
\includegraphics[scale= 0.5width = 60mm]{formant-graph.png}
\caption{Shows the graph of first formant frequency versus second formant frequency for malayalam vowels for various speakers}
\end{figure}
% Screenshot-1.png: 1024x768 pixel, 72dpi, 36.12x27.09 cm, bb=0 0 1024 768
%\end{center}
%\includegraphics[width=60mm]{praat.eps}
According to KeralaPaniniyam the alphabets of malayalam
consists of 16vowels and 37 consonants,though this has been to
slight modifications through ages.The list of vowels is made of
4 short vowels and 3 long vowels in the division of samanakhshara,
2 short vowels and 4 long vowels in the division of sandhyakshara
and two more as à´ à´[aum] and à´ :[ah].The figure below shows an
illustration on how the formant frequencies vary for some of
the vowels.
The division of consonant sounds are even more interesting.
Consonant sounds are produced by partial or complete closure of
upper vocal tract.So taking the famous analogy of method of speech
production with electrical networks,we have the noice sources changing
their place along the vocal tract,as the consonant changes.Interestingly
in malayalam, they are divided on the same basis of the place where the
flow of air is affected.
%----------------------------------------------------------------------
% SECTION III: Specifications
%----------------------------------------------------------------------
\section{Speech Recognition}
This project uses the Sphinx engines developed by Carnegie Mellon
University ( CMU). SPHINX is one of the best and most versatile
recognition systems in the world today.Sphinx is having a BSD derived
license so people can share the software and use it as they wish.But
propreitary software authors can also use sphinx to make propreitary
software due to the famous liberal attitude of BSD license. Sphinx has
many versions which meets different environments in which the system is
employed.They are sphinx2,PocketSphinx sphinx3 and Sphinx4.PocketSphinx
is CMUSphinx's most accurate ASR system while its not as accurate as Sphinx3
or Sphinx4.Sphinx3 is using continuous HMM while PocketSphinx and Sphinx2 are
using semi-continuous HMMs.Sphinx4 is written entirely in JAVA-programming language.
Sphinx train is the acoustical model trainer software in CMUSphinx.
It is a statistical acoustic model trainer using Hidden Markov Models (HMM).
Due to inavailability of a speech corpora, the project is designed to have
a wrapper around sphinxtrain so that people can easily train the acoustic models
for sharika and use those models to do the desktop control.So A new user first trains his
model for sharika and then uses the system.This will greatly improve the accuracy
of the model as a near-perfect user independent model itself requires long-term
project of collection of speech corpora and tuning the models to perfection.
The desktop control part is developed over the GNOME libraries so that
the GNOME users can control their window navigation and general desktop control
using this system.Initially I developed a simple 8 command system which is
now enhanced to above 50 commands.
The sphinx decoder requires two things to perform decoding.
\begin{enumerate}
\item Acoustic model and
\item Language Model
\end{enumerate}
\subsection{Building The Acoustical Model}
Acoustic model is developed using sphinxtrain package.Sphinx train has the training algorithms implemented so that we can configure the system and give the required data i.e. both speech and other configuration files such as dictionary, phonelist and transcripts to build the models in the way we want.The task of defining the phonemes is much simple,thanks to the phonetic nature of malayalam language. Majority of the letters themselves form phonemes.We can remember the fact that almost all written malayalam words unlike english words can only be read in one way which shows how phonetic they are.As the project requires users to train their own models,the speech database made to train the models uses only my sound.Phone list,dictionary file and transcripts specific to this project are made.Users are expected to read out the transcripts as they do the training as per the trainer program.
As the user completes audio recording ,the recorded sounds are processed to get the feature vectors extracted.Mel Frequency Cepstral Coefficients(MFCC) are by far the favorite feature vector for ASR systems.Here we are trying to model the response of ear and have many narrow filter banks below 1000Hz and gradually wider bands above;using mel scale to define the width.Thus a frequency wrapped version of cepstrum is used.Feature vector sequence is now used to train acoustic models.
Continuous HMMs are made first in the training session which is later converted to Sphinx2 models
so that they can be decoded using PocketSphinx
%----------------------------------------------------------------------
\subsection{Building The Language Model}
Being a project with small word count, I used Finite State Grammer as the language model and an fsg file is handwritten for the purpose.A screenshot of which is given in the illustration below.
%\begin{figure}
%\includegraphics[scale=0.25]{fsg.png}
%\caption{Shows the graph of first formant frequency versus second formant frequency for malayalam vowels for various speakers}
%\end{figure}
These models are given to sphinx decoder to do the decoding.Sphinx decoder APIs are used along with the libraries related to desktop environment to perform live decoding.
The trained models being user dependent,also as the no: of words are low,
is having very high performance and is working well even for male voices similar
to that of mine.As the project is using a user dependent model the accuracy measurements
are giving above 90 percent correctness and low word error rates.
\section{Conclusion And Future Work}
ASR systems are very complex and is a very time consuming project.Members of Swathanthra Malayalam Computing are now doing the ground works of building a Free Text Corpora for malayalam which will be in a sharable license.Building a speech corpora is even bigger activity and requiring more time and effort.But once built,those models can be adapted and used for enormous number of applications.As already stated, Speech recognition programs are having varities of applications and will become a revolutionary breakthrough when we consider the low percentage of literacy of our country.A speech-to-speech application which interactes with the user by accepting speech and giving out speech could be one of the best use of speech recognition systems,where the user gets the maximum convinence.
With world-class Free Software ASR systems like Sphinx which is built and through contribution from people all over the world,the task of building speech recognition programs are becoming more and more simpler.Still the bigger task remain for us - the making of a user independent acoustical and language model for malayalam,
\section{Acknowledgement}
I would like to acknowledge the continuous support and help from my parents and friends and the members of SMC.
\begin{thebibliography}{1}
\bibitem{Tutorialn}
L Rabiner
\newblock {\em A Tutorial on Hidden Markov models and
Selected Applications in Speech Recognition , Proc. Of
IEEE, Vol. 77 No. 2, 1989.}
\bibitem{GOLD}
Ben Gold and Nelson Morgan
\newblock { \em Speech And Audio Signal processing,processing and perception of speech and music}
\bibitem{Digital}
L R Rabiner,R W Schafer
\newblock { \em Digital processing of speech signals}
\bibitem{Quatieri}
Thomas F Quatieri
\newblock { \em Speech Signal processing Principles and Practice}
\bibitem{Keralapaniyiam}
\newblock { \em keralapaniniyam and common malayalam grammar books for the detailed reference of malayalam language.}
\bibitem{CMUSPHINX}
\begin{verbatim}
http://cmusphinx.org/
\end{verbatim}
\newblock {\em above is the central link to the complete Sphinx project at CMU}
\bibitem{Subwiki}
\begin{verbatim}
http://sphinx.subwiki.com/
\end{verbatim}
\newblock {\em above is a newly created wiki.}
\bibitem{Fundamentals}
Lawrence Rabiner and Biing-Hwang Juang
\newblock {\em Fundamentals of Speech Recognition}
\end{thebibliography}
\end{document}