DR. MOHAMMED MAZHAR UL HAQUE

In the probability theory the Chebyshev’s Inequality & central limit theorem deal with the situations where we want to find the probability distribution of sum of large numbers of random variables in approximately normal condition, Before looking the limit theorems we see some of the inequalities, which provides the bounds for the probabilities if the mean and variance is known.

Markov’s inequality

The Markov’s inequality for the random variable X which takes only positive value for a>0 is

to prove this for a>0 consider

Since

now taking expectation of this inequality we get

the reason is

which gives the Markov’s inequality for a>0 as

Chebyshev’s inequality

 For the finite mean and variance of random variable X the Chebyshev’s inequality for k>0 is

where sigma and mu represents the variance and mean of random variable, to prove this we use the Markov’s inequality as the non negative random variable

for the value of a as constant square, hence

this equation is equivalent to

as clearly

Examples of Markov’s and Chebyshev’s inequalities :

1. If the production of specific item is taken as random variable for the week with mean 50 , find the probability of production exceeding 75 in a week and what would be the probability if the production of a week is between 40 and 60 provided the variance for that week is 25?

Solution: Consider the random variable X for the production of the item for a week then to find the probability of production exceeding 75 we will use Markov’s inequality as

Now the probability for the production in between 40 to 60 with variance 25 we will use Chebyshev’s inequality as

so

this shows the probability for the week if the production is between 40 to 60 is 3/4.

2. Show that the chebyshev’s inequality which provides upper bound to the probability is not particularly nearer to the actual value of the probability.

Solution:

Consider the random variable X is uniformly distributed with mean 5 and variance 25/3 over the interval (0,1) then by the chebyshev’s inequality we can write

but the actual probability will be

which is far from the actual probability likewise if we take the random variable X as normally distributed with mean and variance then Chebyshev’s inequality will be

but the actual probability is

Weak Law of Large Numbers

The weak law for the sequence of random variables will be followed by the result that Chebyshev’s inequality can be used as the tool for proofs for example to prove

if the variance is zero that is the only random variables having variances equal to 0 are those which are constant with probability 1 so by Chebyshev’s inequality for n greater than or equal to 1

as

by the continuity of the probability

which proves the result.

to prove this we assume that variance is also finite for each random variable in the sequence so the expectation and variance

now from the Chebyshev’s inequality the upper bound of the probability as

which for n tending to infinity will be

Central Limit theorem

The central limit theorem is one of the important result in probability theory as it gives the distribution to the sum of large numbers which is approximately normal distribution in addition to the method for finding the approximate probabilities for sums of independent random variables central limit theorem also shows the empirical frequencies of so many natural populations exhibit bell-shaped means normal curves, Before giving the detail explanation of this theorem we use the result

“If the sequence of random variables Z₁,Z₂,…. have the distribution function and moment generating function as F_Zn and M_zn then

Central Limit theorem: For the sequence of identically distributed and independent random variables X₁,X₂,……. each of which having the mean μ and variance σ2 then the distribution of the sum

tends to the standard normal as n tends to infinity for a to be real values

Proof: To prove the result consider the mean as zero and variance as one i.e μ=0 & σ²=1 and the moment generating function for X_i exists and finite valued so the moment generating function for the random variable X_i/√n will be

hene the moment generating function for the sum ΣX_i/√n will be

Now let us take L(t)=logM(t)

so

to show the proof we first show

by showing its equivalent form

since

hence this shows the result for the mean zero and variance 1, and this same result follows for the general case also by taking

and for each a we have

Example of the Central Limit theorem

To calculate the distance in light year of a star from the lab of an astronomer, he is using some measuring techniques but because of change in atmosphere each time the distance measured is not exact but with some error so to find the exact distance he plans to observe continuously in a sequence and the average of these distances as the estimated distance, If he consider the values of measurement identically distributed and independent random variable with mean d and variance 4 light year, find the number of measurement to do to obtain the 0.5 error in the estimated and actual value?

Solution: Let us consider the measurements as the independent random variables in sequence X₁,X₂,…….X_n so by the Central Limit theorem we can write

which is the approximation into standard normal distribution so the probability will be

so to get the accuracy of the measurement at 95 percent the astronomer should measure n* distances where

so from the normal distribution table we can write it as

which says the measurement should be done for 62 number of times, this also can be observed with the help of Chebyshev’s inequality by taking

so the inequality results in

hence for n=16/0.05=320 which gives certainity that there will be only o.5 percent error in the measurement of the distance of the star from the lab of observations.

2. The number of admitted students in engineering course is Poisson distributed with mean 100, it was decided that if the admitted students are 120 or more the teaching will be in two sections otherwise in one section only, what will be the probability that there will be two sections for the course?

Solution: By following the Poisson distribution the exact solution will be

which is obviously not give the particular numerical value, If we consider the random variable X as the students admitted then by the central limit theorem

which can be

which is the numerical value.

3. Calculate the probability that the sum on ten die when rolled is between 30 and 40 including 30 and 40?

Solution: Here considering the die as X_i for ten values of i. the mean and variance will be

thus following the central limit theorem we can write

which is the required probability.

4. For the uniformly distributed independent random variables X_i on the interval (0,1) what will be the approximation of the probability

Solution: From the Unifrom distribution we know that the mean and variance will be

Now using the central limit theorem we can

thus the summation of the random variable will be 14 percent.

5. Find the probability for the evaluator of the exam to give grades will be 25 exams in starting 450 min if there are 50 exams whose grading time is independent with mean 20 min and standard deviation 4 min.

Solution: Consider the time require to grade the exam by the random variable X_i so the random variable X will be

since this task for 25 exam is withing 450 min so

here using the central limit theorem

which is the required probability.

Central Limit theorem for independent random variables

For the sequence which is not identically distributed but having independent random variables X₁,X₂,……. each of which having the mean μ and variance σ² provided it satisfies

1. each X_i is uniformly bounded
2. sum of the variances is infinite, then

Strong Law of Large Numbers

Strong Law of Large numbers is very crucial concept of the probability theory which says that the average of sequence of commonly distributed random variable with probability one will converge to the mean of that same distribution

Statement: For the sequence of identically distributed and independent random variables X₁,X₂,……. each of which having the finite mean with probability one then

Proof: To prove this consider the mean of each of random variable is zero, and the series

now for this consider power of this as

after taking the expansion of the right hand side terms we have the terms of the form

since these are independents so the mean of these will be

with the help of combination of the pair the expansion of the series now will be

since

so

we get

this suggest the inequality

hence

By the convergence of the series since the probability of each random variable is one so

since

if the mean of each random variable is not equal to zero then with deviation and probability one we can write it as

or

which is required result.

One Sided Chebyshev Inequality

The one sided Chebysheve inequality for the random variable X with mean zero and finite variance if a>0 is

to prove this consider for b>0 let the random variable X as

which gives

so using the Markov’s inequality

which gives the required inequality. for the mean and variance we can write it as

This further can be written as

Example:

Find the upper bound of the probability that the production of the company which is distributed randomly will at least 120, if the production of this certain company is having mean 100 and variance 400.

Solution:

Using the one sided chebyshev inequaility

so this gives the probability of the production within a week atleast 120 is 1/2, now the bound for this probability will be obtained by using Markov’s inequality

which shows the upper bound for the probability.

Example:

Hundred pairs are taken from two hundred persons having hundred men and hundred women find the upper bound of the probability that at most thirty pair will consist a men and a women.

Solution:

Let the random variable X_i as

so the pair can be expressed as

Since every man can equally likely to be pair with remaining people in which hundred are women so the mean

in the same way if i and j are not equal then

as

hence we have

using the chebyshev inequality

which tells that the possibility of pairing 30 men with women is less than six, thus we can improve the bound by using one sided chebyshev inequality

Chernoff Bound

If the moment generating function already known then

as

in the same way we can write for t<0 as

Thus the Chernoff bound can be define as

this inequality stands for all the values of t either positive or negative.

Chernoff bounds for the standard normal random variable

The Chernoff bounds for the standard normal random variable whose moment generating function

is

so minimizing this inequality and right hand side power terms gives for a>0

and for a<0 it is

Chernoff bounds for the Poisson random variable

The Chernoff bounds for the Poisson random variable whose moment generating function

is

so minimizing this inequality and right hand side power terms gives for a>0

and it would be

Example on Chernoff Bounds

In a game if a player is equally likely to either win or lose the game independent of any past score, find the chernoff bound for the probability

Solution: Let X_i denote the winning of the player then the probability will be

for the sequence of n plays let

so the moment generating function will be

here using the expansions of exponential terms

so we have

now applying the property of moment generating function

This gives the inequality

hence

Conclusion:

The inequalities and limit theorem for the large numbers were discussed and the justifiable examples for the bounds of the probabilities were also taken to get the glimpse of the idea, Also the the help of normal, poisson random variable and moment generating function is taken to demonstrate the concept easily, if you require further reading go through below books or for more Article on Probability, please follow our Mathematics pages.

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

Probability is a fundamental concept in mathematics that allows us to quantify uncertainty and make predictions about the likelihood of events occurring. It plays a crucial role in various fields, including statistics, economics, physics, and computer science. In this section, we will explore the definition of probability and its importance in mathematics, as well as the axioms that form the foundation of probability theory.

Definition of Probability and Its Importance in Math

Probability can be defined as a measure of the likelihood of an event occurring. It is represented as a number between 0 and 1, where 0 indicates impossibility and 1 indicates certainty. The concept of probability is essential in mathematics because it helps us analyze and understand uncertain situations.

In real life, we encounter probabilistic situations every day. For example, when flipping a fair coin, we know that the probability of it landing on heads is 0.5. Similarly, when rolling a fair six-sided die, the probability of rolling a specific number, say 3, is 1/6. By understanding and applying probability, we can make informed decisions and assess risks in various scenarios.

Probability theory provides a systematic framework for studying and analyzing uncertain events. It allows us to mathematically model and analyze random phenomena, such as coin flips, dice rolls, and card games. By using probability theory, we can calculate the likelihood of different outcomes, estimate the expected value of random variables, and make predictions based on available data.

Axioms of Probability Theory

To ensure a consistent and coherent approach to probability, mathematicians have established a set of axioms that form the foundation of probability theory. These axioms provide a rigorous framework for defining and manipulating probabilities. Let’s take a closer look at the three axioms of probability:

1. Non-negativity: The probability of any event is always a non-negative number. In other words, the probability of an event cannot be negative.

2. Additivity: For any collection of mutually exclusive events (events that cannot occur simultaneously), the probability of the union of these events is equal to the sum of their individual probabilities. This axiom allows us to calculate the probability of complex events by considering the probabilities of their constituent parts.

3. Normalization: The probability of the entire sample space (the set of all possible outcomes) is equal to 1. This axiom ensures that the total probability of all possible outcomes is always 1, providing a consistent framework for probability calculations.

By adhering to these axioms, we can ensure that our calculations and reasoning about probabilities are logically sound and consistent. These axioms, along with other probability concepts, such as conditional probability, independence, and Bayes’ theorem, form the building blocks of probability theory.

In the upcoming sections, we will delve deeper into probability theory, exploring various probability concepts, examples, exercises, and calculations. By understanding the axioms and principles of probability, we can develop a solid foundation for tackling more complex probability problems and applying probability in real-world scenarios.

Problems on Probability and Its Axioms

Example 1: Restaurant Menu Combinations

Imagine you’re at a restaurant with a diverse menu, offering a variety of appetizers, entrees, and desserts. Let’s say there are 5 appetizers, 10 entrees, and 3 desserts to choose from. How many different combinations of a meal can you create?

To solve this problem, we can use the fundamental principle of counting. The principle states that if there are m ways to do one thing and n ways to do another, then there are m * n ways to do both.

In this case, we can multiply the number of choices for each course: 5 appetizers * 10 entrees * 3 desserts = 150 different combinations of a meal.

Example 2: Probability of Item Purchases

Suppose you’re running an online store and you want to analyze the probability of customers purchasing certain items together. Let’s say you have 100 customers, and you track their purchase history. Out of these customers, 30 have bought item A, 40 have bought item B, and 20 have bought both items A and B. What is the probability that a randomly selected customer has bought either item A or item B?

To solve this problem, we can use the principle of inclusion-exclusion. This principle allows us to calculate the probability of the union of two events by subtracting the probability of their intersection.

First, we calculate the probability of buying item A or item B separately. The probability of buying item A is 30/100 = 0.3, and the probability of buying item B is 40/100 = 0.4.

Next, we calculate the probability of buying both item A and item B. This is given by the intersection of the two events, which is 20/100 = 0.2.

To find the probability of buying either item A or item B, we add the probabilities of buying each item and subtract the probability of buying both items: 0.3 + 0.4 – 0.2 = 0.5.

Therefore, the probability that a randomly selected customer has bought either item A or item B is 0.5.

Example 3: Probability of Card Occurrences

Let’s consider a standard deck of 52 playing cards. What is the probability of drawing a heart or a diamond from the deck?

To solve this problem, we need to determine the number of favorable outcomes (drawing a heart or a diamond) and the total number of possible outcomes (drawing any card from the deck).

There are 13 hearts and 13 diamonds in a deck, so the number of favorable outcomes is 13 + 13 = 26.

The total number of possible outcomes is 52 (since there are 52 cards in a deck).

Therefore, the probability of drawing a heart or a diamond is 26/52 = 0.5.

Example 4: Probability of Temperature Occurrences

Suppose you are interested in predicting the weather for the next day. You have observed that over the past year, the probability of a hot day is 0.3, the probability of a cold day is 0.2, and the probability of a rainy day is 0.4. What is the probability that tomorrow will be either hot or cold, but not rainy?

To solve this problem, we can use the probability addition rule. The rule states that the probability of the union of two mutually exclusive events is the sum of their individual probabilities.

In this case, the events “hot day” and “cold day” are mutually exclusive, meaning they cannot occur at the same time. Therefore, we can simply add their probabilities: 0.3 + 0.2 = 0.5.

Therefore, the probability that tomorrow will be either hot or cold, but not rainy, is 0.5.

Example 5: Probability of Card Denominations and Suits

Consider a standard deck of 52 playing cards. What is the probability of drawing a card that is either a king or a spade?

To solve this problem, we need to determine the number of favorable outcomes (drawing a king or a spade) and the total number of possible outcomes (drawing any card from the deck).

There are 4 kings and 13 spades in a deck, so the number of favorable outcomes is 4 + 13 = 17.

The total number of possible outcomes is 52 (since there are 52 cards in a deck).

Therefore, the probability of drawing a card that is either a king or a spade is 17/52 ≈ 0.327.

Example 6: Probability of Pen Colors

Suppose you have a bag containing 5 red pens, 3 blue pens, and 2 green pens. What is the probability of randomly selecting a red or blue pen from the bag?

To solve this problem, we need to determine the number of favorable outcomes (selecting a red or blue pen) and the total number of possible outcomes (selecting any pen from the bag).

There are 5 red pens and 3 blue pens in the bag, so the number of favorable outcomes is 5 + 3 = 8.

The total number of possible outcomes is 5 + 3 + 2 = 10 (since there are 5 red pens, 3 blue pens, and 2 green pens in the bag).

Therefore, the probability of randomly selecting a red or blue pen from the bag is 8/10 = 0.8.

Example 7: Probability of Committee Formation

Suppose there are 10 people, and you need to form a committee of 3 people. What is the probability that you select 2 men and 1 woman for the committee?

To solve this problem, we need to determine the number of favorable outcomes (selecting 2 men and 1 woman) and the total number of possible outcomes (selecting any 3 people from the group of 10).

First, we calculate the number of ways to select 2 men from a group of 5 men: C(5, 2) = 10.

Next, we calculate the number of ways to select 1 woman from a group of 5 women: C(5, 1) = 5.

To find the total number of favorable outcomes, we multiply the number of ways to select 2 men by the number of ways to select 1 woman: 10 * 5 = 50.

The total number of possible outcomes is the number of ways to select any 3 people from a group of 10: C(10, 3) = 120.

Therefore, the probability of selecting 2 men and 1 woman for the committee is 50/120 ≈ 0.417.

Example 8: Probability of Suit Occurrences in a Card Hand

Consider a standard deck of 52 playing cards. What is the probability of drawing a hand of 5 cards that contains at least one card of each suit (hearts, diamonds, clubs, and spades)?

To solve this problem, we need to determine the number of favorable outcomes (drawing a hand with at least one card of each suit) and the total number of possible outcomes (drawing any hand of 5 cards from the deck).

First, we calculate the number of ways to select one card from each suit: 13 * 13 * 13 * 13 = 285,316.

Next, we calculate the total number of possible outcomes, which is the number of ways to draw any 5 cards from a deck of 52: C(52, 5) = 2,598,960.

Therefore, the probability of drawing a hand of 5 cards that contains at least one card of each suit is 285,316/2,598,960 ≈ 0.11.

Example 9: Probability of choosing the same letter from two words

When it comes to probability, we often encounter interesting problems that challenge our understanding of the subject. Let’s consider an example that involves choosing the same letter from two words.

Suppose we have two words, “apple” and “banana.” We want to determine the probability of randomly selecting the same letter from both words. To solve this problem, we need to break it down into smaller steps.

First, let’s list all the letters in each word:

Word 1: “apple”
Word 2: “banana”

Now, we can calculate the probability of choosing the same letter by considering each letter individually. Let’s go through the process step by step:

1. Selecting a letter from the first word:
2. The word “apple” has five letters, namely ‘a’, ‘p’, ‘p’, ‘l’, and ‘e’.

3. The probability of selecting any particular letter is 1 out of 5, as there are five letters in total.

4. Selecting a letter from the second word:

5. The word “banana” has six letters, namely ‘b’, ‘a’, ‘n’, ‘a’, ‘n’, and ‘a’.

6. Similarly, the probability of selecting any particular letter is 1 out of 6.

7. Calculating the probability of choosing the same letter:

8. Since each letter has an equal chance of being selected from both words, we multiply the probabilities together.
9. The probability of selecting the same letter is (1/5) * (1/6) = 1/30.

Therefore, the probability of choosing the same letter from the words “apple” and “banana” is 1/30.

What are the important properties of conditional expectation and how do they relate to problems on probability and its axioms?

The concept of conditional expectation is a fundamental concept in probability theory, and it has important properties that can help us solve problems related to probability and its axioms. To understand these properties and their relationship to probability problems, it is essential to delve into the Properties of conditional expectation explained. These properties provide insights into how conditional expectations behave and can be used to calculate expectations and probabilities in various scenarios. By understanding these properties, we can bridge the gap between the concept of probability and its axioms and the idea of conditional expectation, enabling us to tackle complex probability problems with confidence.

Frequently Asked Questions

1. What is the importance of probability in math?

Probability is important in math because it allows us to quantify uncertainty and make predictions based on available information. It provides a framework for analyzing and understanding random events and their likelihood of occurrence.

2. How would you define probability and its axioms?

Probability is a measure of the likelihood of an event occurring. It is defined using three axioms:

1. The probability of any event is a non-negative number.
2. The probability of the entire sample space is 1.
3. The probability of the union of mutually exclusive events is equal to the sum of their individual probabilities.

3. What are the three axioms of probability?

The three axioms of probability are:

1. Non-negativity: The probability of any event is a non-negative number.
2. Normalization: The probability of the entire sample space is 1.
3. Additivity: The probability of the union of mutually exclusive events is equal to the sum of their individual probabilities.

4. What are the axioms of expected utility theory?

The axioms of expected utility theory are a set of assumptions that describe how individuals make decisions under uncertainty. They include the axioms of completeness, transitivity, continuity, and independence.

5. What are the axioms of probability theory?

The axioms of probability theory are the fundamental principles that govern the behavior of probabilities. They include the axioms of non-negativity, normalization, and additivity.

6. Can you provide some solved problems on axioms of probability?

Certainly! Here is an example:

Problem: A fair six-sided die is rolled. What is the probability of rolling an even number?

Solution: Since the die is fair, it has six equally likely outcomes: {1, 2, 3, 4, 5, 6}. Out of these, three are even numbers: {2, 4, 6}. Therefore, the probability of rolling an even number is 3/6 = 1/2.

7. Where can I find probability problems and answers?

You can find probability problems and answers in various resources such as textbooks, online math websites, and educational platforms. Additionally, there are specific websites that provide probability problems and solutions, such as Math-Aids Answers.

8. Are there any probability examples available?

Yes, there are many probability examples available. Some common examples include flipping a coin, rolling dice, drawing cards from a deck, and selecting balls from an urn. These examples help illustrate how probability concepts can be applied in different scenarios.

9. What are some probability formulas and rules?

There are several probability formulas and rules that are commonly used, including:

• Addition Rule: P(A or B) = P(A) + P(B) – P(A and B)
• Multiplication Rule: P(A and B) = P(A) * P(B|A)
• Complement Rule: P(A’) = 1 – P(A)
• Conditional Probability: P(A|B) = P(A and B) / P(B)
• Bayes’ Theorem: P(A|B) = P(B|A) * P(A) / P(B)

10. Can you suggest some probability exercises for practice?

Certainly! Here are a few probability exercises you can try:

1. A bag contains 5 red balls and 3 blue balls. What is the probability of drawing a red ball?
2. Two dice are rolled. What is the probability of getting a sum of 7?
3. A deck of cards is shuffled and one card is drawn. What is the probability of drawing a heart?
4. A jar contains 10 red marbles and 5 green marbles. If two marbles are drawn without replacement, what is the probability of getting two red marbles?
5. A spinner is divided into 8 equal sections numbered 1 to 8. What is the probability of landing on an even number?

These exercises will help you practice applying probability concepts and calculations.

For the random variable dependent on one another requires the calculation of conditional probabilities which we already discussed, now we will discuss some more parameters for such random variables or experiments like conditional expectation and conditional variance for different types of random variables.

Conditional Expectation

   The definition of conditional probability mass function of discrete random variable X given Y is

here pY(y)>0 , so the conditional expectation for the discrete random variable X given Y when pY (y)>0 is

in the above expectation probability is the conditional probability.

  In similar way if X and Y are continuous then the conditional probability density function of the random variable X given Y is

where f(x,y) is joint probability density function and for all yf_Y(y)>0 , so the conditional expectation for the random variable X given y will be

for all yf_Y(y)>0.

   As we know that all the properties of probability are applicable to conditional probability same is the case for the conditional expectation, all the properties of mathematical expectation are satisfied by conditional expectation, for example conditional expectation of function of random variable will be

and the sum of random variables in conditional expectation will be

Conditional Expectation for the sum of binomial random variables

    To find conditional expectation of the sum of binomial random variables X and Y with parameters n and p which are independent, we know that X+Y will be also binomial random variable with the parameters 2n and p, so for random variable X given X+Y=m the conditional expectation will be obtained by calculating the probability

since we know that

thus the conditional expectation of X given X+Y=m is

Example:

Find the conditional expectation

if the joint probability density function of continuous random variables X and Y is given as

solution:

To calculate the conditional expectation we require conditional probability density function, so

since for the continuous random variable the conditional expectation is

hence for the given density function the conditional expectation would be

Expectation by conditioning||Expectation by conditional expectation

                We can calculate the mathematical expectation with the help of conditional expectation of X given Y as

for the discrete random variables this will be

which can be obtained as

and for the continuous random we can similarly show

Example:

                A person is trapped in his building underground as the entrance is blocked due to some heavy load fortunately there are three pipelines from which he can come out the first pipe take him safely out after 3 hours, the second after 5 hours and the third pipeline  after 7 hours, If any of these pipeline chosen equally likely by him, then what would be the expected time he will come outside safely.

Solution:

Let X be the random variable that denote the time in hours until the person came out safely and Y denote the pipe he chooses initially, so

since

If the person chooses the second pipe , he spends 5 hous in that but  he come outside with expected time

so the expectation  will be

Expectation of sum of random number of random variables using conditional expectation

                Let N be the random number of random variable and sum of random variables is     then the expectation  

since

as

thus

Correlation of bivariate distribution

If the probability density function of the bivariate random variable X and Y is

where

then the correlation between random variable X and Y for the bivariate distribution with density function is

since correlation is defined as

since the expectation using conditional expectation is

for the normal distribution the conditional  distribution X given Y is having mean

now the expectation of  XY given Y is

this gives

hence

Variance of geometric distribution

    In the geometric distribution let us perform successively independent trials which results in success with probability p , If N represents the time of first success in these succession then the variance of N as by definition will be

Let the random variable Y=1 if the first trial results in success and Y=0 if first trial results in failure, now to find the mathematical expectation here we apply the conditional expectation as

since

if success is in first trial then N=1 and N²=1 if failure occur in first trial , then to get the first success  the total number of trials will have the same distribution as 1 i.e the first trial that results in failure with  plus the necessary number of additional trials,  that is

Thus the expectation will be

since the expectation of geometric distribution is so

hence

and

E

so the variance of geometric distribution will be

Expectation of Minimum of sequence of uniform random variables

   The sequence of uniform random variables U₁, U₂ … .. over the interval (0, 1) and N is defined as

then for the expectation of N, for any x ∈ [0, 1] the value of N

we will set the expectation of N as

to find the expectation we use the definition of conditional expectation on continuous random variable

now conditioning for the first term of the sequence  we have

here we get

the remaining number of uniform random variable is same at the point where the first uniform value is y,in starting and then were going to add uniform random variables until their sum surpassed x − y.

so using this value of expectation the value of integral will be

if we differentiate this equation

and

now integrating this gives

hence

the value of k=1 if x=0 , so

m

and m(1) =e, the expected number of uniform  random variables over the interval (0, 1) that need to be added until their sum surpasses 1, is equal to e

Probability using conditional Expectation || probabilities using conditioning

   We can find the probability also by using conditional expectation like expectation we found with conditional expectation, to get this consider an event and a random variable X as

from the definition of this random variable and expectation clearly

now by conditional expectation in any sense we have

Example:

compute the probability mass function of random variable X , if U is the uniform random variable on the interval (0,1), and consider the conditional distribution of X given U=p as binomial  with parameters n and p.

Solution:

For the value of U the probability by conditioning is

we have the result

so we will get

Example:

what is the probability of X < Y, If X and Y are the continuous random variables with probability density functions f_X and f_Y respectively.

Solution:

By using conditional expectation and conditional probability

as

Example:

Calculate the distribution of sum of continuous independent random variables X and Y.

Solution:

To find the distribution of X+Y we have to find the probability of the sum by using the conditioning as follows

Conclusion:

The conditional Expectation for the discrete and continuous random variable with different examples considering some of the types of these random variables discussed using the independent random variable and the joint distribution in different conditions, Also the expectation and probability how to find using conditional expectation is explained with examples, if you require further reading go through below books or for more Article on Probability, please follow our Mathematics pages.

https://en.wikipedia.org/wiki/probability_distribution

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

Moment generating function    

Moment generating function is very important function which generates the moments of random variable which involve mean, standard deviation and variance etc., so with the help of moment generating function only, we can find basic moments as well as higher moments, In this article we will see moment generating functions for the different discrete and continuous random variables. since the Moment generating function(MGF) is defined with the help of mathematical expectation denoted by M(t) as

and using the definition of expectation for the discrete and continuous random variable this function will be

which by substituting the value of t as zero generates respective moments. These moments we have to collect by differentiating this moment generating function for example for first moment or mean we can obtain by differentiating once as

This gives the hint that differentiation is interchangeable under the expectation and we can write it as

and

if t=0 the above moments will be

and

In general we can say that

hence

Moment generating function of Binomial distribution||Binomial distribution moment generating function||MGF of Binomial distribution||Mean and Variance of Binomial distribution using moment generating function

The Moment generating function for the random variable X which is Binomially distribution will follow the probability function of binomial distribution with the parameters n and p as

which is the result by binomial theorem, now differentiating and putting the value of t=0

which is the mean or first moment of binomial distribution similarly the second moment will be

so the variance of the binomial distribution will be

which is the standard mean and variance of Binomial distribution, similarly the higher moments also we can find using this moment generating function.

Moment generating function of Poisson distribution||Poisson distribution moment generating function||MGF of Poisson distribution||Mean and Variance of Poisson distribution using moment generating function

 If we have the random variable X which is Poisson distributed with parameter Lambda then the moment generating function for this distribution will be

now differentiating this will give

this gives

which gives the mean and variance for the Poisson distribution same which is true

Moment generating function of Exponential distribution||Exponential distribution moment generating function||MGF of Exponential distribution||Mean and Variance of Exponential distribution using moment generating function

                The Moment generating function for the exponential random variable X by following the definition is

here the value of t is less than the parameter lambda, now differentiating this will give

which provides the moments

clearly

Which are the mean and variance of exponential distribution.

Moment generating function of Normal distribution||Normal distribution moment generating function||MGF of Normal distribution||Mean and Variance of Normal distribution using moment generating function

  The Moment generating function for the continuous distributions also same as the discrete one so the moment generating function for the normal distribution with standard probability density function will be

this integration we can solve by adjustment as

since the value of integration is 1. Thus the moment generating function for the standard normal variate will be

from this we can find for any general normal random variable the moment generating function by using the relation

thus

so differetiation gives us

thus

so the variance will be

Moment generating function of Sum of random variables

The Moment generating function of sum of random variables gives important property that it equals the product of moment generating function of respective independent random variables that is for independent random variables X and Y then the moment generating function for the sum of random variable X+Y is

here moment generating functions of each X and Y are independent by the property of mathematical expectation. In the succession we will find the sum of moment generating functions of different distributions.

Sum of Binomial random variables

If the random variables X and Y are distributed by Binomial distribution with the parameters (n,p) and (m,p) respectively then moment generating function of their sum X+Y will be

where the parameters for the sum is (n+m,p).

Sum of Poisson random variables

The distribution for the sum of independent random variables X and Y with respective means which are distributed by Poisson distribution we can find as

Where

is the mean of Poisson random variable X+Y.

Sum of Normal random variables

     Consider the independent normal random variables X and Y with the parameters

then for the sum of random variables X+Y with parameters

so the moment generating function will be

which is moment generating function with additive mean and variance.

Sum of random number of random variables

To find the moment generating function of the sum of random number of random variables let us assume the random variable

where the random variables X₁,X₂, … are sequence of random variables of any type, which are independent and identically distributed then the moment generating function will be

Which gives the moment generating function of Y on differentiation as

hence

in the similar way the differentiation two times will give

which give

thus the variance will be

Example of Chi-square random variable

Calculate the moment generating function of the Chi-squared random variable with n-degree of freedom.

Solution: consider the Chi-squared random variable with the n-degree of freedom for

the sequence of standard normal variables then the moment generating function will be

so it gives

the normal density with mean 0 and variance σ² integrates to 1

which is the required moment generating function of n degree of freedom.

Example of Uniform random variable

Find the moment generating function of random variable X which is binomially distributed with parameters n and p given the conditional random variable Y=p on the interval (0,1)

Solution: To find the moment generating function of random variable X given Y

using the binomial distribution, sin Y is the Uniform random variable on the interval (0,1)

Joint moment generating function

The Joint moment generating function for the n number of random variables X₁,X₂,…,X_n

where t₁,t₂,……t_n are the real numbers, from the joint moment generating function we can find the individual moment generating function as

Theorem: The random variables X₁,X₂,…,X_n are independent if and only if the joint mement generating function

Proof: Let us assume that the given random variables X₁,X₂,…,X_n are independent then

Now assume that the joint moment generating function satisfies the equation

• to prove the random variables X₁,X₂,…,X_n are independent we have the result that the joint moment generating function uniquely gives the joint distribution(this is another important result which requires proof) so we must have joint distribution which shows the random variables are independent, hence the necessary and sufficient condition proved.

Example of Joint Moment generating function

1.Calculate the joint moment generating function of the random variable X+Y and X-Y

Solution : Since the sum of random variables X+Y and subtraction of random variables X-Y are independent as for the independent random variables X and Y so the joint moment generating function for these will be

as this moment generating function determine the joint distribution so from this we can have X+Y and X-Y are independent random variables.

2. Consider for the experiment the number of events counted and uncounted distributed by poisson distribution with probability p and the mean λ, show that the number of counted and uncounted events are independent with respective means λp and λ(1-p).

Solution: We will consider X as the number of events and X_c the number of counted events so the number of uncounted events is X-X_c,the joint moment genrating function will generate moment

and by the moment generating function of binomial distribution

and taking expectation off these will give

Conclusion:

By using the standard definition of moment generating function the moments for the different distributions like binomial, poisson, normal etc were discussed and the sum of these random variables either the discrete or continuous the moment generating function for those and joint moment generating function were obtained with suitable examples , if you require further reading go through below books.

For more articles on Mathematics, please see our Mathematics page.

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH

In this article the conditional Variance and predictions using conditional expectation for the different kind of random variable with some examples we will discuss.

Conditional Variance

The conditional variance of random variable X given Y is defined in similar way as conditional Expectation of random variable X given Y as

(X|Y)=E[(X-E[X|Y])²|Y]

here variance is the conditional expectation of difference between random variable and square of conditional expectation of X given Y when the value of Y is given.

The relation between the conditional variance and conditional expectation is

(X|Y) = E[X²|Y] – (E[X|Y])²

E[(X|Y)] = E[E[X²|Y]] – E[(E[X|Y])²]

= E[X²] – E[(E[X\Y])²]

since E[E[X|Y]] = E[X], we have

(E[X|Y]) = E[(E[X|Y])²] – (E[X])²

this is somehow similar from the relation of unconditional variance and expectation which was

Var(X) = E[X²] – (E[X])²

and we can find the variance with the help of conditional variance as

Var(X) = E[var(X|Y] + var(E[X|Y])

Example of conditional variance

Find the mean and variance of the number of travelers who enters into the bus if the people arrived at bus depot is Poisson distributed with mean λt and the initial bus arrived at bus depot is uniformly distributed over the interval (0,T) independent of people arrived or not.

Solution:

To find the mean and variance let for any time t , Y is the random variable for the time bus arrive and N(t) is the number of arrivals

E[N(Y)|Y = t] = E[N(t)|Y = t]

by the independence of Y and N(t)

=λt

since N(t) is Poisson with mean \lambda t
Hence

E[N(Y)|Y]=λY

so taking expectations gives

E[N(Y)] = λE[Y] = λT/2

To obtain Var(N(Y)), we use the conditional variance formula

thus

(N(Y)|Y) = λY

E[N(Y)|Y] = λY

Hence, from the conditional variance formula,

Var(N(Y)) = E[λY]+(λY)

=λT/² + λ²T²/12

where we have used the fact that Var(Y)=T² / 12.

Variance of a sum of a random number of random variables

consider the sequence of independent and identically distributed random variables X₁,X₂,X₃,………. and another random variable N independent of this sequence, we will find variance of sum of this sequence as

using

which is obvious with the definition of variance and conditional variance for the individual random variable to the sum of sequence of random variables hence

Prediction

In prediction the value of one random variable can be predicted on the basis of observation of another random variable, for prediction of random variable Y if observed random variable is X we use g(X) as the function which tells the predicted value, obviously we try to choose g(X) closed to Y for this the best g is g(X)=E(Y|X) for this we must have to minimize the value of g by using the inequality

This inequality we can get as

However, given X, E[Y|X]-g(X), being a function of X, can be treated as a constant. Thus,

which gives the required inequality

Examples on Prediction

1. It is observed that the height of a person is six feet, what would be the prediction of his sons height after grown up if the height of son which is x inches now is normally distributed with mean x+1 and variance 4.

Solution: let X be the random variable denoting the height of the person and Y be the random variable for the height of son, then the random variable Y is

Y=X+e+1

here e represent the normal random variable independent of random variable X with mean zero and variance four.

so the prediction for the sons height is

so the height of the son will be 73 inches after growth.

2. Consider an example of sending signals from location A and location B, if from location A a signal value s is sent which at location B received by normal distribution with mean s and variance 1 while if the signal S sent at A is normally distributed with mean \mu and variance \sigma^2, how we can predict that the signal value R sent from location A will be received is r at location B?

Solution: The signal values S and R denote here the random variables distributed normally, first we find the conditional density function S given R as

this K is independent of S, now

here also C₁ and C₂ are independent on S, so the value of conditional density function is

C is also independent on s, Thus the signal sent from location A as R and received at location B as r is normal with mean and variance

and the mean square error for this situation is

Linear Predictor

Every time we can not find the joint probability density function even the mean, variance and the correlation between two random variables is known, in such a situation linear predictor of one random variable with respect to another random variable is very helpful which can predict the minimum, so the for the linear predictor of random variable Y with respect to random variable X we take a and b to minimize

Now differentiate partially with respect to a and b we will get

solving these two equations for a nd b we will get

thus minimizing this expectation gives the linear predictor as

where the means are the respective means of random variables X and Y, the error for the linear predictor will be obtained with the expectation of

This error will be nearer to zero if correlation is perfectly positive or perfectly negative that is coefficient of correlation is either +1 or -1.

Conclusion

The conditional variance for the discrete and continuous random variable with different examples were discussed, one of the important application of conditional expectation in prediction is also explained with suitable examples and with best linear predictor, if you require further reading go through below links.

For more post on Mathematics, please refer to our Mathematics Page

A first course in probability by Sheldon Ross

Schaum’s Outlines of Probability and Statistics

An introduction to probability and statistics by ROHATGI and SALEH