Читать книгу Informatics and Machine Learning - Stephen Winters-Hilt - Страница 38

The LLN will now be derived in the classic “weak” form. The “strong” form is derived in the modern mathematical context of Martingales in Appendix C.1.

Let X_k be independent identically distributed (iid) copies of X, and let X be the real number “alphabet.” Let μ = E(X), σ² = Var(X), and denote

From Chebyshev: P(| _N − μ|>k) ≤ Var(_N)/k² = σ²

As N➔∞ get the LLN (weak):

If X_k are iid copies of X, for k = 1,2,…, and X is a real and finite alphabet, and μ = E(X), σ² = Var(X), then: P(|_N − μ| > k)➔0, for any k > 0. Thus, the arithmetic mean of a sequence of iid r.v.s converges to their common expectation. The weak form has convergence “in probability,” while the strong form has convergence “with probability one.”