Читать книгу Data Science in Theory and Practice - Maria Cristina Mariani - Страница 22

We begin this section with the definition of ‐algebra.

Definition 2.17 (σ‐algebra) A ‐algebra is a collection of sets of satisfying the following condition:

1 .

2 If then its complement .

3 If is a countable collection of sets in then their union .

Definition 2.18 (Measurable functions) A real‐valued function defined on is called measurable with respect to a sigma algebra in that space if the inverse image of the set , defined as is a set in ‐algebra , for all Borel sets of . Borel sets are sets that are constructed from open or closed sets by repeatedly taking countable unions, countable intersections and relative complement.

Definition 2.19 (Random vector) A random vector is any measurable function defined on the probability space with values in (Table 2.1).

Measurable functions will be discussed in detail in Section 20.5.

Suppose we have a random vector defined on a space . The sigma algebra generated by is the smallest sigma algebra in that contains all the pre images of sets in through . That is

This abstract concept is necessary to make sure that we may calculate any probability related to the random variable .

Any random vector has a distribution function, defined similarly with the one‐dimensional case. Specifically, if the random vector has components , its cumulative distribution function or cdf is defined as:

Associated with a random variable and its cdf is another function, called the probability density function (pdf) or probability mass function (pmf). The terms pdf and pmf refer to the continuous and discrete cases of random variables, respectively.

Table 2.1 Examples of random vectors.

Experiment	Random variable
Toss two dice	= sum of the numbers
Toss a coin 10 times	= sum of tails in 10 tosses

Definition 2.20 (Probability mass function) The pmf of a discrete random variable is given by

Definition 2.21 (Probability density function) The pdf, of a continuous random variable is the function that satisfies

We will discuss these notations in details in Chapter 20.

Using these concepts, we can define the moments of the distribution. In fact, suppose that is any function, then we can calculate the expected value of the random variable when the joint density exists as:

Now we can define the moments of the random vector. The first moment is a vector

The expectation applies to each component in the random vector. Expectations of functions of random vectors are computed just as with univariate random variables. We recall that expectation of a random variable is its average value.

The second moment requires calculating all the combination of the components. The result can be presented in a matrix form. The second central moment can be presented as the covariance matrix.

(2.1)

where we used the transpose matrix notation and since the , the matrix is symmetric.

We note that the covariance matrix is positive semidefinite (nonnegative definite), i.e. for any vector , we have .

Now we explain why the covariance matrix has to be semidefinite. Take any vector . Then the product

(2.2)

is a random variable (one dimensional) and its variance must be nonnegative. This is because in the one‐dimensional case, the variance of a random variable is defined as . We see that the variance is nonnegative for every random variable, and it is equal to zero if and only if the random variable is constant. The expectation of (2.2) is . Then we can write (since for any number , )

Since the variance is always nonnegative, the covariance matrix must be nonnegative definite (or positive semidefinite). We recall that a square symmetric matrix is positive semidefinite if . This difference is in fact important in the context of random variables since you may be able to construct a linear combination which is not always constant but whose variance is equal to zero.

The covariance matrix is discussed in detail in Chapter 3.

We now present examples of multivariate distributions.