1. ECE 109 DISCUSSION 8 1.1. Pairs of Discrete Random Variables. Definition 1.1. Let SX,Y = {(xj , yk ) , j = 1, 2, . . . , k = 1, 2, . . .} be the entire sample space. The joint pmf of (X, Y ) specifying the probabilities of the event {X = xj } ∩ {Y = yk } for some (xj , yk ) ∈ SX,Y is given by pX,Y (xj , yk ) = P [X = xj , Y = yk ] . This joint pmf of (X, Y ) provides the information about the joint behavior of X and Y . Then, X X pX,Y (xj , yk ) = 1. (xj ,yk ) ∈SX,Y

The probability of any event A is the sum of the pmf over the outcomes in A: X X P [(X, Y ) in A] = pX,Y (xj , yk ) . (xj ,yk ) ∈A

Definition 1.2. Suppose there exist two random variables X and Y . When we are interested in the probabilities of events involving each of the random variables, we use the marginal probability mass functions: pX (xj ) = P [X = xj ] which by the theorem on total probability, = P [{X = xj } ∩ {Y = y1 } ∪ {X = xj } ∩ {Y = y2 } ∪ . . .] X = pX,Y (xj , yk ) . k

1.2. The Joint CDF of X and Y . Definition 1.3. The joint cdf of X and Y is defined as FX,Y (xj , yk ) = P [X ≤ xj , Y ≤ yk ] . Property 1.4. The joint cdf satisfies the following properties. (i) The joint cdf is a nondecreasing function of x and y: FX,Y (x1 , y1 ) ≤ FX,Y (x2 , y2 )

if x1 ≤ x2 and y1 ≤ y2 ,

(ii) FX,Y (x1 , −∞) = FX,Y (−∞, y1 ) = 0 FX,Y (∞, ∞) = 1. (iii) The marginal cdfs are given by FX (x1 ) = FX,Y (x1 , ∞) FY (y1 ) = FX,Y (∞, y1 ) . 1

(iv) The joint cdf is continuous from the right and top, that is, lim FX,Y (x, y) = FX,Y (a, y)

x→a+

lim FX,Y (x, y) = FX,Y (x, b) .

y→b+

(iv) P [x1 < X ≤ x2 , y1 < Y ≤ y2 ] = FX,Y (x2 , y2 ) − FX,Y (x2 , y1 ) − FX,Y (x1 , y2 ) + FX,Y (x1 , y1 ) . 1.3. The Joint PDF of Two Continuous Random Variables. Definition 1.5. The probability of any event A is given by Z Z P [X in A] = fX,Y (x, y) dx dy. A

The joint cdf can be obtained in terms of the joint pdf Z a Z b FX,Y (a, b) = fX,Y (x, y) dx dy. −∞

−∞

If X and Y are jointly continuous random variables, the the pdf can be obtained from the cdf by differentiation: fX,Y (x, y) =

∂ 2 FX,Y (x, y) . ∂x ∂y

Also, the probability of a rectangular region is Z

b1

Z

b2

P [a1 < X ≤ b1 , a2 < Y ≤ b2 ] =

fX,Y (x, y) dx dy. a1

a2

The marginal pdf ’s fX (x) and fY (y) are obtained by taking the derivative of the corresponding marginal cdf ’s, FX (x) = FX,Y (x, ∞) and FY (y) = FX,Y (∞, y). Hence, dFX (x) dx  Z x Z ∞ d fX,Y (t, y) dy dt = dx −∞ −∞ which by the fundamental theorem of calculus Z II, fX (x) =

∞

=

fX,Y (x, y) dy. −∞

Similarly, Z

∞

fY (y) =

fX,Y (x, y) dx. −∞

1.4. Independence of Two Random Variables. Remark 1.6. Two jointly discrete random variables X and Y are independent if and only if the joint pmf is equal to the product of the marginal pmf ’s for all x and y. That is, pX,Y (x, y) = pX (x) pY (y) . 2

Two jointly continuous random variables X and Y are independent if and only if the joint pdf is equal to the product of the marginal pdf ’s for all x and y. fX,Y (x, y) = fX (x) fY (y) . Also, two jointly random variables X and Y are independent if and only if the joint cdf is equal to the product of the marginal cdf ’s for all x and y. FX,Y (x, y) = FX (x) FY (y) . If X and Y are independent random variables, then the random variables defined by any pair of functions g (X) and h (Y ) are also independent. That is, P [g (X) = m, h (Y ) = n] = P [g (X) = m] P [h (Y ) = n] . Example 1. The joint cdf of for two random variables X and Y are given by  (1 − e−αx ) 1 − e−βy
, if x ≥ 0, y ≥ 0 FX,Y (x, y) = 0 , otherwise. Then, show (a) X and Y individually have exponential random variables with parameters α and β, respectively, and (b) X and Y are independent. Proof. (a) FX (x) = FX,Y (x, ∞) = 1 − e−αx




for x ≥ 0.

Taking the derivative with respect to x, fX (x) =

dFX (x) = αe−αx dx

for x ≥ 0.

Similarly, we have fY (y) = βe−βy

for y ≥ 0.

Hence, X and Y individually have exponential random variables with parameters α and β, respectively. (b) The joint pdf for two random variables X and Y is ∂ 2 FX,Y (x, y) ∂x ∂y

fX,Y (x, y) =

= αβe−αx e−βy = fX (x) fY (y) . Hence, two random variables X and Y are independent.

3