EECS 126 - Probability and Random Processes - J. Walrand

LIMITS OF RANDOM VARIABLES

·         Key Ideas

·         Distribution

·         Transforms

·         Almost Sure

·         Probability

·         L2

·         Relationships

     Convergence of expectation

·         Criteria

Key Ideas

 

A few other very subtle but useful ideas!  We want to explain what we mean by Xn ® X as n  ® ¥. 

 

Mathematically, Xn and X are functions.  Thus, it takes some care to define the convergence of functions””. 

 

Intuitively also, the meaning of “Xn is close to X” requires some care since these are uncertain quantities.

 

In this section, we clarify the meaning of convergence and we give methods to test it.

Distribution

 

We can say that X and Y are close to each other if their distributions are similar, i.e., if P(X £ x) » P(Y £ x) for all x Î Â.  For example, we could say that X is almost a standard Gaussian random variable.

 

Correspondingly, we define convergence in distribution as follows:

Xn ®D X if P(Xn £ x) ® P(X £ x) as n ® ¥, for all x Î Â.

(More precisely, we want this convergence to hold only for all x such that P(X £ x) is continuous.)

In it in this sense that one can show that many random variables that occur in physical systems are Gaussian or Poisson.

Transforms

 

Transform methods are convenient to show convergence in distribution.  We give an example here. (See the Central Limit Theorem for another example.) For n ³ 1 and p > 0, let X(n, p) be binomial with the parameters (n, p).  That is,

 

P(X(n, p) = m) = C(n, m)pm(1 – p)n – m.

 

We want to show that as p ® 0 and np ® l, one has X(n, p) ®D X where X is Poisson with mean l. We do this by showing that E(zX(n, p)) ® E(zX) for all complex number z.  These expected values are the z-transforms of the probability mass functions of the random variables.  One then invokes a theorem that says that if the z-transforms converge, then so do the probability mass functions.  For now, we do the calculation and we accept the theorem.  Note that X(n, p) is the sum of n i.i.d. random variables that are 1 with probability p and zero otherwise.  If we designate one such generic random variable by V(p), we have

 

E(zX(n, p)) = {E(zY(p))}n = ((1 – p) + pz)n » (1 + lp(z – 1)/n)n » exp{lp (z – 1)}.

 

Also,

E(zX) = Snzn(lp)nexp{- l}/n! = exp{lp (z – 1)}.

 

Almost Sure

 

In some cases, one can show that

Xn ®as X if Xn ® X  as n ® ¥, for almost all w Î W.

For instance, the fraction of coin flips that are heads converges to ½, almost surely.

In Probability

 

It may be that the probability that Xn and X are not close to each other gets smaller and smaller as n increases.  We say that

Xn ®P X if P(|Xn – X| > e) ® 0  as n ® ¥, for all e > 0.

L2

 

Another way to say that X and Y are close to each other is when E(|X – Y|2) is small.  This is the meaning of convergence in L2.  Specifically, we say that

Xn ®L2 X if E|Xn – X|2 ® 0  as n ® ¥, for all e > 0.

Relationships

 

All these notions make sense.  How do they relate to one another?  Here is a summary:

 

Proofs and counterexamples are useful to appreciate the meaning of these definitions.

Convergence of Expectation

 

Assume Xn ®as X. Examples show that it is generally not the case that E(Xn) ® E(X). However, two simple sets of sufficient conditions are known.

 

Theorem

a. Assume Xn ®as X and Xn £ Xn+1 for all n.  Then E(Xn) ® E(X).

a. Assume Xn ®as X and |Xn| £ Y  for all n with E(Y) < ¥.  Then E(Xn) ® E(X).

 

Criteria

 

How does one prove convergence?  Typically:

·        Almost sure: Borel Cantelli

·        Probability: Markov inequality

·        Distribution: Characteristic function

Jean Walrand – January 2000  --- INDEX