Читать книгу EEG Signal Processing and Machine Learning - Saeid Sanei - Страница 80

Singular value decomposition (SVD) is often used for solving the least‐squares (LS) problem. This is performed by decomposition of the M × M square autocorrelation matrix R into its eigenvalue matrix Λ = diag(λ₁, λ₂, … λ _M ) and an M × M orthogonal matrix of eigenvectors V, i.e. R = VΛV^H , where (.) ^H denotes Hermitian (conjugate transpose) operation. Moreover, if A is an M × M data matrix such that R = A^H A then there exist an M × M orthogonal matrix U, an M × M orthogonal matrix V, and an M × M diagonal matrix ∑ with diagonal elements equal to , such that:

(4.119)

Hence ∑ ² = Λ. The columns of U are called left singular vectors and the rows of V^H are called right singular vectors. If A is rectangular N × M matrix of rank k then U will be N × N and ∑ will be:

(4.120)

where S = diag(σ₁, σ₂, … σ _k ), where σ _i = . For such a matrix the Moore–Penrose pseudo‐inverse is defined as an M × N matrix A^† defined as:

(4.121)

where ∑ ^† is an M × N matrix defined as:

(4.122)

A ^† has a major role in the solutions of least‐squares problems, and S ⁻¹ is a k × k diagonal matrix with elements equal to the reciprocals of the singular values of A, i.e.

(4.123)

In order to see the application of the SVD in solving the LS problem consider the error vector e defined as:

(4.124)

where d is the desired signal vector and Ah is the estimate . To find h we replace A with its SVD in the above equation (Eq. 4.124) and find h, which thereby minimizes the squared Euclidean norm of the error vector, ‖e ²‖. By using the SVD we obtain:

(4.125)

or equivalently

(4.126)

Since U is a unitary matrix, ‖e ²‖ = ‖U^H e ‖². Hence, the vector h that minimizes ‖e ²‖ also minimizes ‖U^H e ‖². Finally, the unique solution as an optimum h (coefficient vector) may be expressed as [43]:

(4.127)

where k is the rank of A. Alternatively, as the optimum least‐squares coefficient vector:

(4.128)

Performing PCA is equivalent to performing an SVD on the covariance matrix. PCA uses the same concept as SVD and orthogonalization to decompose the data into its constituent uncorrelated orthogonal components such that the autocorrelation matrix is diagonalized. Each eigenvector represents a principal component and the individual eigenvalues are numerically related to the variance they capture in the direction of the principal components. In this case the mean squared error (MSE) is simply the sum of the N‐K eigenvalues, i.e.:

(4.129)

PCA is widely used in data decomposition, classification, filtering, and whitening. In filtering applications, the signal and noise subspaces are separated and the data are reconstructed from only the eigenvalues and eigenvectors of the actual signals. PCA is also used for BSS of correlated mixtures if the original sources can be considered statistically uncorrelated.

Figure 4.13 Adaptive estimation of the weight vector w(n).

The PCA problem is then summarized as how to find the weights w in order to minimize the error given the observations only. The LMS algorithm is used here to iteratively minimize the MSE as:

(4.130)

The update rule for the weights is then:

(4.131)

where the error signal e(n) = x(n) − Φ^T (n)w(n), x(n) is the noisy input and n is the iteration index. The step size μ may be selected empirically or adaptively. These weights are then used to reconstruct the sources from the set of orthogonal basis functions. Figure 4.13 shows the overall system for adaptive estimation of the weight vector w using the LMS algorithm.