Bounding Gate Covariances

Having derived the update equation for gate means, we now turn our attention to the gate covariances. We bound the Q function with logarithms of Gaussians. Maximizing this bound (a sum of log-Gaussians) reduces to the maximum-likelihood estimation of a covariance matrix. The bound for a Q_im sub-component is shown in Equation 14. Once again, we assume the data has been appropriately whitened with respect to the gate's previous parameters (the gate's previous mean is 0 and previous covariance is identity). Equation 15 solves for the log-Gaussian parameters (again $\rho^2 = {\bf x}_i^T {\bf x}_i$).

  

$$\begin{array}{l} Q(\Theta^t,\Theta^{(t-1)})_{im} ~ \geq ~ \log({\... ...a_{xx}^m}^{-1} {\bf c}_{im} - w_{im} \log \vert {\Sigma_{xx}^m}\vert\end{array}$$ (14)

$$\begin{array}{lll} {\bf c}_{im} {\bf c}_{im}^T & = & \frac{1}{2w_... ...f x}_i) } { tr(I) - tr(\Sigma^{-1}) + \log \vert \Sigma^{-1} \vert} \end{array}$$ (15)

The computation for the minimal w_im simplifies to $w_{im}^* = r_i \alpha_m g(\rho)$. The g function is derived and plotted in Figure 2(c). An example of a log-Gaussian bound is shown in Figure 2(d) a sub-component of the Qfunction. Each sub-component corresponds to a single data point as we vary one gate's covariance. All $M \times N$ log-Gaussian bounds are computed (one for each data point and gate combination) and are summed to bound the Q function in its entirety.

To obtain a final answer for the update of the gate covariances $\Sigma_{xx}^m$ we simply maximize the sum of log Gaussians (parametrized by $w^*_{im},k_{im},{\bf c}_{im}$). The update is $\Sigma_{xx}^m = ( \sum w^*_{im} {\bf c}_{im} {{\bf c}_{im}}^T ) ~ (\sum w^*_{im})^{-1}$. This covariance is subsequently unwhitened, inverting the whitening transform applied to the data.