Читать книгу Industrial Data Analytics for Diagnosis and Prognosis - Yong Chen - Страница 38

Normal distribution is the most commonly used distribution for continuous random variables. Many statistical models and inference methods are based on the univariate or multivariate normal distribution. One advantage of the normal distribution is its mathematical tractability. More importantly, the normal distribution turns out to be a good approximation to the “true” population distribution for many sample statistics and real-world data due to the central limit theorem, which says that the summation of a large number of independent observations from any population with the same mean and variance approximately follows a normal distribution.

Recall that a univariate random variable X with mean μ and variance σ² is normally distributed, which is denoted by X ∼ N (μ, σ²), if it has the probability density function

(3.7)

The multivariate normal distribution is an extension of the univariate normal distribution. If a p-dimensional random vector X follows a multivariate normal distribution with mean vector μ and covariance matrix Σ, the probability density function of X has the form

(3.8)

We denote the p-dimensional normal distribution by Np(μ, Σ).

From (3.8), the density of a p-dimensional normal distribution depends on x through the term (x − μ)^T Σ⁻¹ (x − μ), which is the square of the distance from x to Σ standardized by the covariance matrix. Then it is clear that the set of x values yielding a constant height for the density form an ellipsoid. The set of points with the same height for the density is called a contour. The constant probability density contour of a p-dimensional normal distribution is:

which forms the surface of an ellipsoid centered at μ with standardized distance between x and μ equal to c. And the contour with larger distance c has a smaller height value for the density. It can be shown that the axes of the ellipsoid contours of constant density for the p-dimensional normal distribution are in the directions of the eigenvectors of Σ with lengths proportional to the square roots of the corresponding eigenvalues of Σ.

Example 3.1: Consider a bivariate (p = 2) normally distributed random vector X = (X₁ X₂)T. Suppose the mean vector is μ = (0 0)T and the covariance matrix is

So the variance of both variables is equal to one and the covariance matrix coincides with the correlation matrix. The inverse of the covariance matrix is

and |Σ| = 1 − ρ². Substituting Σ⁻¹ and |Σ| in (3.8), we have

(3.9)

From (3.9), if ρ = 0, the joint density can be written as f(x₁,x₂) = f(x₁)f(x₂), where f(x) is the univariate normal density as given in (3.7), with μ = 0 and σ = 1. So in this case X₁ and X₂ are independent. This result is true for general multivariate normal distribution, as discussed later in this section.

By solving the characteristic equation |Σ − λI| = 0, the two eigenvalues of Σ are λ₁ = 1 + ρ and λ₂ = 1 – ρ. Based on Σv = λv, the corresponding eigenvectors can be obtained as

So the major axis of the ellipse contour of constant density is along the line x₁ = x₂ and the minor axis is orthogonal to the major axis. The larger the correlation coefficient ρ, the more elongated the ellipse contour. As an example, two bivariate normal distributions with ρ = 0 and ρ = 0.75 are shown in Figure 3.1(a) and Figure 3.1(b), respectively. Notice how the presence of correlation causes the probability distribution to concentrate along the line x₁ = x₂. When ρ = 0, it is easy to see that the constant-density contour is a circle, as shown in Figure 3.2(a). For ρ = 0.75, the constant-density contour is an ellipse shown in Figure 3.2(b).

Figure 3.1 Two bivariate normal distributions, (a) ρ = 0 (b) ρ = 0.75

Figure 3.2 Contour plots for the distributions in Figure 3.1