6.858/18.428 Machine Learning Lecture 25: Decmeber 12, 1994 Lecturer: Ron Rivest Scribe: Jesse Hull

25.1 Outline

• Learning Hidden Markov Models

• Application to speech recognition

25.2 Description of Hidden Markov Models

In this lecture we describe Hidden Markov Models and give a method for learning them. Their application to speech recognition is also discussed.

Definition 1 A Hidden Markov Model is a 5-tuple {Q, V, π(i), A, B} where

Q = the set of states = {q_{1}, q_{2}, ..., q_{n}} V = the output alphabet = {v_{1}, v_{2}, ..., v_{m}} π(i) = probability of being in state q_{i} at time t = 0 A = transition probabilities = {a_{ij}},

where a_{ij} = P r[entering state q_{j} at time t + 1 | in state q_{i} at time t] B = output probabilites = {b_{j}(k)},

where b_{j}(k) = P r[producing v_{k} at time t | in state q_{j} at time t]

25.3 Example of a Hidden Markov Model

If we let π(q_{1}) = 1 and π(q_{2}) = 0 then the following is a example of a possible transition sequence and output sequence for the Hidden Markov Model in figure 25.1:

25-1

25-2 Lecture 25: Decmeber 12, 1994

[figure: q1; q2; 0.4; 0.6; 0.2; 0.8; H = 2/3; T = 1/3; H = 1/6; T = 5/6]

Figure 25.1: A Hidden Markov Model

0.8 0.8 0.2 0.6 0.4 0.8
q_{1} ⟶ q_{1} ⟶ q_{1} ⟶ q_{2} ⟶ q_{2} ⟶ q_{1} ⟶ q_{1}
⇓  ⇓  ⇓  ⇓  ⇓  ⇓  ⇓
H  H  T  T  T  T  H

We can easily calculate probabilities for the following events.

1. The probability of the above state transition sequence:

P r[q_{1}q_{1}q_{1}q_{2}q_{2}q_{1}q_{1}] = π(q_{1})a_{11}a_{11}a_{12}a_{22}a_{21}a_{11} ≈ 0.025

2. The probability of the above output sequence given the above transition se-

quence:

P r[(HHT T T T H)|(q_{1}q_{1}q_{1}q_{2}q_{2}q_{1}q_{1})] = \frac{2}{3} \frac{2}{3} \frac{1}{3} \frac{5}{6} \frac{5}{6} \frac{1}{3} \frac{2}{3} ≈ 0.23

3. The probability of the above output sequence and the above transition sequence: P r[(HHT T T T H) ∧ (q_{1}q_{1}q_{1}q_{2}q_{2}q_{1}q_{1})] ≈ (0.025)(0.023) ≈ 5.7x10^{−4}

Lecture 25: Decmeber 12, 1994 25-3

25.4 Using Hidden Markov Models to generate speech

Markov models can be used to generate speech by making the output symbols phonemes of speech. A Hidden Markov Model might then capture the pronounciation of a word. By changing the structure and transition probabilies in the Markov model one can modify the legal pronounciation of a word.

25.5 Using Markov Models to recognize words

Suppose one has a set of Hidden Markov Models for a set of words. Then given a speech signal one could ask which model was the most likely to have generated that signal. We need to be able to calculate the probability of a given output sequence for a Hidden Markov Model. So, given a output sequence O_{1}O_{2}...O_{T} we have the following equation:

P r[O_{1}O_{2}...O_{T}] = ∑_{i_{1},i_{2},...,i_{T}} P r[i_{1}, i_{2}, ..., i_{T}]P r[O_{1}O_{2}...O_{T} |i_{1}, i_{2}, ..., i_{T}]

This is simply the probability of a given state transition sequence times the probability of the output sequence given that transition sequence, summed over all possible state transtions. We can use dynamic programming to calculate this probability efficiently.

Definition 2 Let α_{t}(i) = P r[(O_{1}O_{2}...O_{t})∧(i_{t} = q_{i})]

Then we can use the following formulas to calculate α_{t}(i) where N is the number of states in the Hidden Markov Model.

α_{1}(i) = π(i)b_{i}(O_{1})
α_{t+1}(i) = (∑_{j=1}^{N} α_{t}(j)a_{ji}) b_{i}(O_{t+1})

From these equations it should be clear that α_{T} (i) can be computed in O(N^{2}T) time. Furthermore since α_{T} (i) is the probability that we get the correct output and end in state q_{i}, the probability that we get the correct output is simply the sum of the α_{T} (i)’s. Namely,

P r[O_{1}O_{2}...O_{T}] = ∑_{i=1}^{N} α_{T} (i)

Hence we can calculate P r[O_{1}O_{2}...O_{T}] in O(N^{2}T) time.

25.6 Using Markov Models to recognize sentences

Now suppose that we not only have models for recognizing words but we have a model that tells us how words should be put together. Namely we have encoded a gramatical model of speech into our Hidden Markov Model. In this case we may ask two questions:

1. For each t, what state is most likely? (Note that the sequence of states computed by this criterion might be impossible.)

2. What single sequence of states has the largest posterior probability?

To answer the first question we define β_{t}(i), which is essentially the time reversal of α_{t}(i), and we define γ_{t}(i), which is the probability of being in state q_{i} at time t given the observation sequence.

Definition 3 Let β_{t}(i) = P r[(O_{t+1}O_{t+2}...O_{T})|(at time t in state q_{i})]

Definition 4 Let γ_{t}(i) = P r[i_{t} = q_{i}|O_{1}, O_{2}, …, O_{T}]

Then we have the following formulas for β_{t}(i) and γ_{t}(i):

β_{T} (i) = 1
β_{t}(i) = ∑_{j=1}^{N} β_{t+1}(j)a_{ij}b_{j}(O_{t+1})
γ_{t}(i) = \frac{α_{t}(i)β_{t}(i)}{∑_{j=1}^{N} α_{t}(j)β_{t}(j)}

Now we have the answer to question 1: for time t, the state q_{i} which maximizes γ_{t}(i) is the state with highest probability. To answer question 2, we make one further definition.

Definition 5 Let δ_{t}(i) = P r[that the most likely single sequence of states ends with state q_{i}, given observed output]

And the equations for δ_{t}(i) are

δ_{1}(i) = π(i)b_{i}(O_{1})
δ_{t+1}(i) = ( max_{j∈[1,N]} δ_{t}(j)a_{ji}) b_{i}(O_{t+1})

Backtracking the calculation δ_{T} (i) will allow you to calculate the answer to question 2. This method is called the Viterbi Algorithm.

25.7 Learning Hidden Markov Models

We have shown that given a Hidden Markov Model we can

1. produce output

2. see which sequence of states produces the best fit

Now we need to learn how to build a model. We have to find a method for calculating a_{ij}, b_{i}, and π(i). The algorithm that we present is similiar to the back-propagation algorithm of neural nets in that it is susceptible to local minima and needs lots of data to converge. It does, however, have the advantage that in every step of the algorithm it improves (or at least keeps constant) its accuracy on the test data.

The Baum/Welch training algorithm works as follows: It is given O_{1}O_{2}...O_{T} and initial estimates for parameters a_{ij}, b_{i}, and π(i). We define new parameters ā_{ij}, b̄_{i}, and π̄(i). First we define Υ_{t}(i, j), the estimator of the probabilty of a transition from q_{i} to q_{j} at time t given output O_{1}O_{2}...O_{T}.

Υ_{t}(i, j) = \frac{α_{t}(i)a_{ij}b_{j}(O_{t+1})β_{t+1}(j)}{P r[O_{1}O_{2}...O_{T}]}

then π̄(i), ā_{ij}, and b̄_{i} are calculated as:

π̄(i) = γ_{1}(i)
ā_{ij} = \frac{∑_{t=1}^{T −1} Υ_{t}(i, j)}{∑_{t=1}^{T −1} γ_{t}(i)}
= \frac{expected number of transitions from state q_{i}to q_{j}}{expected number of transitions out of state q_{i}}
b̄_{j}(k) = \frac{∑_{t=1;O_{t}=v_{k}}^{T} γ_{t}(j)}{∑_{t=1}^{T} γ_{t}(j)}
= \frac{expected number of times in state q_{j} with observed output v_{k}}{expected number of times of being in state q_{j}}

These values π̄(i), ā_{ij}, and b̄_{i} are used for our estimates of a_{ij}, b_{i}, and π(i) for the next iteration. Using this algorithm we can learn Hidden Markov Models. Baum and Welch showed that every iteration of this algorithm will not make the parameters worse. However, the algorithm must still be given reasonable initial parameters. Furthermore, the algorithm says nothing about how many states our Hidden Markov Model should have or how they should be connected.

References

[1] Kai-Fu Lee. Automatic peec recognition. In Computer Science Research Review, pages 7-16. School of Computer Science, Carnegie Mellon University, 1989.

[2] S. E. Levinson, L. R. Rabiner, and M. M. Sondhi. An introduction to the application of the theory of probabilistic functions of Markov process to automatic speech recognition. In Bell System Technical Journal, 62(4):1035-1074, April 1983.

[3] L. R. Rabiner and B. H. Juang. An introduction to hidden markov models. IEEE ASSP Magzine, 3(1):4-16, January 1986.