What Something Exists in Everyday Life: A Review of Probability and Statistics

Abstract

I. Introduction

II. Literature Review

A. Basic Definition of Probability

Probability is observed as randomness and uncertainty in daily lives. Relative Frequency refers to the ratio of how often an event occurs relative to the total number of trials. However, situations occured in everyday seem that probabilities observed from Relative Frequency are not matched with the common sense. For example, two coins continously flipped are not always (Head, Tail) or (Tail, Head). The law of large numbers demonstrates that if the number of trials become higher, Relative Frequency will converge to this common sense. The probability of the flipped coin is $1 \over 2$ so that the result, flipped 100 times, may become 50 Head and 50 Tail.

The language that Probability uses is Set theory. The following table is a example of the Set theory language.

Venn Diagrams are useful visualization tools, allowing to observe the relationship between sets.

Four Set Operations is enabled to Set theory; 1) Union is a operation that makes a combined set from the targeted two sets without duplication. For instance, $A = {1, 2}, B = {2, 3} \to A \cup B = {1, 2, 3}$. 2) Intersection emits a set which contains the only duplicated elements between the two sets. From the previous example, $A \cap B = { 2}$. 3) Complement, marked as $A^c$ or $\bar{A}$, derives a set which everything in $S$ but not in the set. 4) Difference produces a set everything in the original set, but not in the minus set. Likewise with the above example, $A - B = A \cap B^c = { 1}$.

Cardinality $|A|$ means the size of the set. If the set $A$ has its elements ${2,4,6,8,10}$, the value $|A| = 5$.

With these definition, three following rules are derived;

A random experiment is a process that a result occurs uncertainly. Observing with the random experiment, all possible outcomes can be observed. These outcomes are called as the sample space. In other words, the sample space becomes universal set in the context of a random experiment. Take a fair coin for example. If a coin were flipped, the possible results of the flip are head and tail so that sample space, a universal set $S$, is ${H, T }$.

A subset of sample space is called an event. With this definition about event, the probability of the event can be calculated. Likewise with the example of the fair coin, a event $A$ can be defined as the result of the flip is head. In this case, the probability of the event $P(A)$ is $1 \over 2$.

Random Variables are informally defined as a quantity or object which depends on random experiments. For instance, a random variable $A$ can be defined as “True, if a randomly drawn person from our class is female.”

Normally, random variables are categorized as four types: 1) Propositional random variables, 2) Multivalued random variables, 3) Discrete Real-valued random variables, and 4) Continuous Real-valued random variables.

Propositional random variables are the simplest type of random variables. It is defined as a random variable that can take only two values, true or false. For instance, whether it will rain tomorrow or not is a propositional random variable. This boolean type of random variable follows the axioms below.

Multivalued random variables are the random variables that can take more than two values. For example, the result of a dice roll is a multivalued random variable. The probability of the event that the result of the dice roll is 4 is $P(A) = {1 \over 6}$. The axioms of multivalued random variables are as follows.

Discrete (Real-valued) random variables are the random variables that can take countable number of values. For example, the number of students in a class is a discrete real-valued random variable. PMF is a function, used in discrete random variables, which maps the probability of each value. $RX$ is a countable set of values that the discrete random variable can take. The definition of the PMF is

$\text{Fig 1. An example of PMF [}\href{#mjx-eqn-1}{1} \text{]}$

Continuous (Real-valued) random variables are the random variables that can take infinitely many values. For example, the height of a person is a continuous real-valued random variable. The one thing that needs to be cautious is that the probability of a single value in a continuous random variable is zero. For instance, if a continuous random variable $X \in [0, 10]$, uniformly distributed, the probability of the event $P(X = 5)$ is zero.

CMF is a function, used in random variables, which maps the probability of the event that the random variable is less than or equal to a certain values. The function is defined as $CMF(x) = P(X \le x)$. PDF is used to specify the probability of the continuous random variables falling within a particular range of values. This probability is given by the integral of these variables over the range. In other words, the area under the density function but above the horizontal axis and between the lowest and the greatest variable values is the probability of the random variable. The valid PDF must satisfy the following conditions: 1) $f(x) \ge 0$ for all $x$. 2) $\int_{-\infty}^{\infty} f(x) dx = 1$.

$\text{Fig 2. An Example of Four Continuous PDF functions [} \href{#mjx-eqn-2}{2} \text{]}$

CDF is a function, used in continuous random variables, which maps the probability of the event that the random variable is less than or equal to a certain values. The function is defined as $F(x) = P(X \le x)$.

With the definition of random experiments and random variables, three axioms of probability can be defined as follows: 1) For any event $A$, $P(A) \ge 0$. 2) The probability of the sample space $P(S)$ is always $1$. 3) For disjoint events greater than or equal to 2, the probability of the union of these events is equal to the sum of the probabilities of these events. $P(A_1 \cup A_2 \cup \cdots) = P(A_1) + P(A_2) + \cdots$

Conditional Probability refers to the probability which has been considered after certain observations or facts. For instance, in poker cards, if a selected card were a black, then what is the probability of the card being a spade? The probability of the card being a spade is the Conditional Probability, and the answer is $P(\text{Spade} | \text{Black}) = \frac{13}{26} = \frac{1}{2}$. The formula for Conditional Probability is given by

Two events $A$ and $B$ are independent if $P(A \cap B) = P(A)P(B)$. In other words, the occurrence of one event does not affect the probability of the other event. From the formula of Conditional Probability, if $A$ and $B$ are independent, then $P(A|B) = P(A)$ and $P(B|A) = P(B)$.

Joint Probability represents the probability of two events occurring simultaneously. Take the poker cards again as an example. The probability of selecting a black card and a spade card is $P(\text{Black} \cap \text{Spade}) = P(\text{Black}) \times P(\text{Spade}) = \frac{13}{52} = \frac{1}{4}$.

Marginalization is the process of obtaining the probability of a event by summing over all the possible values of the other event. For instance, the probability of selecting a black card is $P(\text{Black}) = P(\text{Black} \cap \text{Spade}) + P(\text{Black} \cap \text{Club}) = \frac{13}{52} + \frac{13}{52} = \frac{26}{52} = \frac{1}{2}$.

B. Bayes Theorem

Bayes Theorem, alternatively known as Bayes’rule or Bayes’ law, is a formula that describes how to derive the probability of a hypothesis given the evidence. The formula is driven from the definition of Conditional Probability and given by

C. Probability Distributions

Probability Distributions are the mathematical functions that provide the probabilities of occurrence of different possible outcomes in an experiment. Probability Distributions are categorized as 1) Discrete Probability Distributions and 2) Continuous Probability Distributions. Discrete Probability Distributions, known as PMF, are the distributions that can take only countable values. Continuous Probability Distributions, known as PDF, are the distributions that can take infinitely many values.

Univariate Distributions in Probability Distributions, unlike Multivariate Distributions which have more than one random variable, refers to the distribution that has only one random variable. the binomial distribution, the hypergeometric distribution, the normal distribution are the examples of Univariate Distributions. The multivariate normal distribution is a commonly encountered in Multivariate Distributions. For the summary simplification, the Univariate Distributions are discussed in this review.

Mean is the first moment of a distribution, which is the average value of the distribution and is denoted by $E(X)$ or $\mu$. Variance is the second moment of a distribution, which is the measure of the spread of the distribution and is denoted by $Var(X)$. Standard Deviation is the square root of the variance and is denoted by $SD(X)$ or $\sigma_x$. In 1) Discrete Probability Distributions, the formulas of Mean, Variance, and Standard Deviation are given by

Mean is not enough to describe the distribution. Consider two random variables $X$ and $Y$ with the same mean. The distribution of $X$ can be more spread out than the distribution of $Y$. Variance is used to measure the spread of the distribution so that most moments of the distribution are utilized both Mean and Variance to describe the distribution. Variance is always non-negative, larger than or equal to zero, since it is the square of the difference between the random variable and the mean. The bigger the Variance, the more spread out the distribution is from the mean as well as the smaller the Variance, the more concentrated the distribution is around the mean.

The proof of the formula $(1)$ is as follows;

In 2) Continuous Probability Distributions, the summation and the PMFs, in the formulas of Mean, Variance, and Standard Deviation above, are replaced by the integrals and the PDFs. The formula are given by

Median is the value that separates the higher half from the lower half of the data. It gives the advantage compared to the Mean that it is not affected by the outliers so that it may be more robust value to describe the distribution. If the number of the data is odd, the median is the middle value. If the number of the data is even, the median is the average of the two middle values. Mode is the value that appears most frequently in the data. The distribution can have more than one mode, not necessarily unique since the distribution can have more than one peak. The one peak distribution is called Unimodal, the two peak distribution is called Bimodal, and the more than two peak distribution is called Multimodal.

|| |:–:|

$\text{Fig 3. Geometric visualization of Mean, Median, and Mode in distributions}$

In symmetric unimodal distributions, the Mean, Median, and Mode are the same ($\text{Mean} = \text{Median} = \text{Mode}$). In the right-skewed distribution, the Mean is greater than the Median, and the Median is greater than the Mode ($\text{Mode} \lt \text{Median} \lt \text{Mean}$). In the left-skewed distribution, the Mean is less than the Median, and the Median is less than the Mode ($\text{Mode} \gt \text{Median} \gt \text{Mean}$).

D. Discrete Probability Distributions

Five discrete probability distributions are tackled in this review: 1) Binominal Distribution, 2) Discrete Uniform Distribution, 3) Poisson Distributions, 4) Hypergeometric Distribution, and 5) Geometric Distribution.

1) Binomial Distribution is a distribution that describes the number of successes in a sequence of $n$ independent experiments. The distribution is characterized by two parameters: 1) the number of trials $n$ and 2) the probability of success $p$. The formula of the PMF of the Binomial Distribution is given by

where ${n \choose k} = \frac{n!}{k!(n-k)!}$ is the binomial coefficient. A coin tossed 5 times, A die rolled 25 times, and 50 chickens eggs examined are the examples of the Binomial Distribution.

The important thing is that the Binominal Distribution does not consider the order of the successes. The distribution that the order of the successes is considered is Geometric Distribution (5).

2) Discrete Uniform Distribution is a distribution that describes the probability of each value in the distribution is the same. The formula of the PMF of the Discrete Uniform Distribution is given by

where $a$ and $b$ are the minimum and the maximum values of the distribution. The examples of the Discrete Uniform Distribution are the roll of a fair die, the selection of a card from a deck of cards, and the selection of a number from a set of numbers.

3) Poisson Distribution is a distribution that describes the number of events occurring in a fixed interval of time or space. The distribution is characterized by one parameter: the average number of events $\lambda$. The formula of the PMF of the Poisson Distribution is given by

The Poisson Distribution is used in the number of phone calls received by a call center in an hour, the number of emails received in a day, and the number of customers arriving at a store in an hour. The Poisson Distribution is a special case of the Binomial Distribution when the number of trials $n$ is large and the probability of success $p$ is small.

If you confused about the lambda and mean of Poisson Distribution, just remember that the unit of the mean is a time.

4) Hypergeometric Distribution is a distribution that describes the number of successes in a sequence of $n$ draws without replacement from a finite population of size $N$ that contains $K$ successes. The distribution is characterized by three parameters: the population size $N$, the number of available successes in the population $K$, the number of draws $n$, and the number of observed (curious) success $k$. The formula of the PMF of the Hypergeometric Distribution is given by

The distribution is simply considered as the probability of the possible cases with the combinations without replacements.

Finally, 5) Geometric Distribution is a distribution that describes the number of trials needed to get the first success in a sequence of independent experiments. The distribution is characterized by one parameter: the probability of success $p$. The formula of the Geometric Distribution PMF is given by

E. Continuous Probability Distributions

Four continuous probability distributions are addressed in this review: 1) Normal(Gaussian) Distribution, 2) Exponential Distribution, 3) Continuous Uniform Distribution, and 4) Gamma Distribution.

1) Normal(Gaussian) Distribution is a distribution that describes the probability of the continuous random variable whose distribution is symmetric and bell-shaped. The distribution is characterized by two parameters: the mean $\mu$ and the standard deviation $\sigma$. The formula of the Normal Distribution PDF is given by

68% of the data falls within one standard deviation of the mean, 95% of the data falls within two standard deviations of the mean, and 99.7% of the data falls within three standard deviations of the mean.

The standard normal distribution, also known as Bell Curve, is a special case of the normal distribution when the mean $\mu = 0$ and the standard deviation $\sigma = 1$. The PDF of the standard normal distribution is always $f(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}$. The standard normal distribution is also used to calculate the z-score, which is the number of standard deviations a data point is from the mean. The z-score is calculated by $z = \frac{x - \mu}{\sigma}$. The standardization of the normal distribution is helpful to compare the data points among different normal distributions.

2) Exponential Distribution is a distribution that explains the waiting time between events in a process that occurs continuously and independently at a constant average rate. The distribution is characterized by one parameter: the rate parameter $\lambda$. The formula of the Exponential Distribution PDF and CDF are given by (CDF is an integral of PDF) [Practice]

The moment of Mean means that if the number of the trials is $\lambda$, the average waiting time between each trial is $\frac{1}{\lambda}$. For instance, if the bus arrives at the bus stop 2 times per hour, the average waiting time for the bus is $\frac{1}{2} = 0.5$ hour. This concept is similar to the Poisson Distribution.

3) Continuous Uniform Distribution is a distribution that tells the probability of the continuous random variable whose distribution is uniform. The distribution is characterized by two parameters: the minimum value $a$ and the maximum value $b$. The PDF and CDF of the Continuous Uniform Distribution are given by

4) Gamma Distributions

The gamma distribution is a two-parameter family of continuous probability distributions, particularly useful when dealing with waiting times. The gamma distribution is a generalization of the exponential distribution. In other words, they have same purpose, to predict the wait time until future events, but while the exponential distribution predicts the wait time until the first event, the gamma distribution predicts the wait time until the k-th event.

In order to define the gamma distribution, the gamma function is priorly needed. The gamma function $\Gamma(x)$ is defined with the following five characteristics for any positive real number $\alpha$:

$\text{1. } \Gamma(\alpha) = \int_{0}^{\infty} x^{\alpha-1}e^{-x}dx, \quad \alpha > 0$
$\text{2. } \int_{0}^{\infty} x^{\alpha-1}e^{- \lambda x}dx = \frac{\Gamma(\alpha)}{\lambda^{\alpha}}, \quad \alpha > 0, \lambda > 0$
$\text{3. } \Gamma(\alpha+1) = \alpha\Gamma(\alpha), \quad \alpha > 0$
$\text{4. } \Gamma(n) = (n-1)!, \quad n \in \mathbb{N}$
$\text{5. } \Gamma\left(\frac{1}{2}\right) = \sqrt{\pi}$

With these characteristics, the gamma distribution, $X$~Gamma($\alpha$, $\lambda$) followed by a continuous random variable $X$ with parameters $\alpha \gt 0$ and $\lambda \gt 0$, is defined as follows:

The specific case, Exponential Distribution, can be checked by setting $\alpha = 1 \rightarrow f(x) = \lambda e^{-\lambda x}$. In other words, Gamma(1, $\lambda$) = Exponential($\lambda$).

The moments of the gamma distribution are as follows:

$\text{1. } E(X) = \frac{\alpha}{\lambda}$
$\text{2. } Var(X) = \frac{\alpha}{\lambda^2}$
$\text{3. } SD(X) = \sqrt{\frac{\alpha}{\lambda^2}}$

F. Joint Distributions

Joint distributions are used to describe the probability of two or more random variables simultaneously. The PMF of joint distribution is defined as $P(x, y) = P(X = x, Y = y) = P((X = x) \text{ and } (Y = y)$.

For two discrete random variables $X$ and $Y$ which the sum of their probabilities is equal to 1, the joint PMF is written as $P(x, y) = P(X = x, Y = y) = P((X = x) \cap P(Y = y))$.

Marginal PMF is the probability of a single random variable without considering the other random variable. The marginal PMF of $X$ is defined as $P(x) = P(X = x) = \sum_{y_i \in R_Y} P(x_i, y_i)$ and the marginal PMF of $Y$ is defined as $P(y) = P(Y = y) = \sum_{x_i \in R_X} P(x_i, y_i)$. In order to $X$ and $Y$ are independent, the independent condition $P(x, y) = P(x)P(y)$ must be satisfied in all $x$ and $y$ cases. For instance, if two discrete random variables $X \in {1, 2}$ and $Y \in {1}$ are independent, the condition $P(x, y) = P(x)P(y)$ must be satisfied for $x=1, y=1$, and $x=2, y=1$.

Joint Cumulative Distribution Function (Practice)

A joint cumulative distribution function is a function of two random variables that gives the probability of them jointly. Aslike the CDF of a single random variable was defined as $F_X(x) = P(X \leq x)$, the joint CDF of two random variables is defined as $F_{XY}(x, y) = P(X \leq x, Y \leq y)$. The common for joint distributions is also applied about comma so that

Marginal CDFs, which are the CDFs of individual random variables obtained from the joint CDFs, are defined as

When it comes to joint CDFs, the following formulas are always true:

Conditioning and Independence of Joint Functions (Practice)

A formula regarding conditional probability about joint random variables is derived from the basic conditional probability $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$:

Likewise, an independence of joint random variables is derived from the basic independence of events $P(A \cap B) = P(A)P(B)$:

Conditional expectation is also similari to ordinary expectation. The only difference is that the expectation is taken with respect to the conditional PMF from the PMF. Specifically,

Law of Total Probability (Practice)

Recall that the partition is a set of events that are mutually exclusive. The law of total probability is a formula that gives the probability of an event A by summing the probabilities of the event A intersected with the partitioned events. The law is fitted to expectation as well. To be specific,

Joint Probability Density Function (Practice)

When random variables are moved from discrete to continuous, the PMF is replaced by the PDF, and the sum is replaced by the integral. The joint PDF is defined as $f(x, y)$ with the following properties:

• $f(x, y) \geq 0$
• $\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(x, y) \, dx \, dy = 1$
• $P(a \leq X \leq b, c \leq Y \leq d) = \int_{c}^{d} \int_{a}^{b} f(x, y) \, dx \, dy$

Marginal Probability Density Function (Practice)

The marginal PDF is obtained from the joint PDF by integrating the joint PDF over the other variable. For example,

Joint Cumulative Distribution Function (Practice)

The joint CDF is defined as $F_{XY}(x, y) = P(X \leq x, Y \leq y)$ with the following 6 properties:

• $F_X(x) = F_{XY}(x, \infty)$ for any $X$
• $F_Y(y) = F_{XY}(\infty, y)$ for any $Y$
• $F_{XY}(\infty, \infty) = 1$
• $F_{XY}(-\infty, y) = F_{XY}(x, -\infty) = 0$ for any $x, y$
• $P( X1 \leq x \leq X2, Y1 \leq y \leq Y2) = F_{XY}(x2, y2) - F_{XY}(x1, y2) - F_{XY}(x2, y1) + F_{XY}(x1, y1)$
• If $X$ and $Y$ are independent, then $F_{XY}(x, y) = F_X(x) \cdot F_Y(y)$

G. Conditioning for Continuous Random Variables

Conditional probability for discrete random variables was discussed in the previous section A. Similar to other transition from discrete to continuous random variables, the concept of conditional probability for continuous random variables also work with PDFs from PMFs.

Conditional PDFs and CDFs in Continuous Random Variables (Practice)

Suppose an event $A$ is defined with a continuous random variable $X$ in the interval $[a,b]$. A conditional PDF and CDF of $X$ and $X$’s conditional expectation and variance are as follows:

Conditioning by Another Random Variable (Practice)

For two jointly continuous random variables $X$ and $Y$ with their joint PDF $f_{XY}(x,y)$, the conditional PDF and CDF of $X$ given $Y = y$ are as follows:

Conditional Expectation and Variance in Continuous Random Variables (Practice)

For two jointly continuous random variables $X$ and $Y$ with their joint PDF $f_{XY}(x,y)$, the conditional expectation and variance of $X$ given $Y = y$ are:

Independent Random Variables in Continuous Random Variables (Practice)

When two jointly continuous random variables $X$ and $Y$ are independent, the following formulas are true:

Involved Functions in Continuous Random Variables (Practice)

When it comes to the expectation and variance of the function which involves two rando variables $X$ and $Y$, the following formulas are true:

H. Information Theory

$\quad$ Information Theory, established by Claude Shannon in the 1940s, addresses how much information is contained in a particular event or message, assuming that the event or message is random. The information content $I(p)$ where $p$ is the probability of the event, has the following properties:
$\quad 1.$ $I(p) \geq 0$ for all $p$: Information can not be negative
$\quad 2.$ $I(1) = 0$: An event that has probability 1 does not contain any information
$\quad 3.$ $I(p_1 \cdot p_2) = I(p_1) + I(p_2)$: The information content of two independent events is the sum of the information content of each event
$\quad 4.$ $I(p) = -\log_b(p)$: Based on the properties above, the measurement of information content can be defined
$\quad\quad$ - The unit of information is depended on the base $b$ of the logarithm:
$\quad\quad\quad \cdot$ $b=2$: bits, the most common base (default)
$\quad\quad\quad \cdot$ $b=3$: trits
$\quad\quad\quad \cdot$ $b=e$: nats
$\quad\quad\quad \cdot$ $b=10$: Hartleys

$\quad$ The entropy $H(X)$ is the expected value of the information content of a random variable $X$ and is defined as

The entropy $H(X)$ reaches its maximum value when the probability distribution is uniform.

$\quad$ Relative Entropy is a measure of the difference between two probability distributions $p$ and $q$;

Both distributions must be over the same domain. The value of relative entropy is always non-negative and zero if and only if $p$ and $q$ are the same.

$\quad$ The joint entropy $H(X,Y)$ is the expected value of the information content of two random variables $X$ and $Y$ and is defined as the following equation with three properties:

$\quad 1.$ $H(X,Y) \geq max(H(X), H(Y))$
$\quad 2.$ $H(X,Y) \leq H(X) + H(Y)$
$\quad 3.$ $H(X,Y) = H(X) + H(Y)$ if and only if $X$ and $Y$ are independent.

$\quad$ Condition Entropy $H(X \mid Y)$ gives the amount of gained information through $Y$ when $X$ is known.

$\quad$ Mutual Information $I(X,Y)$ is

I. …

I. Hypothesis Testing

$\quad$ Hypothesis testing is a statistical method to make inferences or draw conclusions about a population based on a sample of data. This leads to determine whether there is enough evicence in the sample data to support a certain claim or hypothesis about the population. A basic outline of the process is fixed as follows:

$\qquad$ 1. Define the null hypothesis $H_0$ and the alternative hypothesis $H_1$.
$\qquad\qquad$ - The null hypothesis represents the default or no-effect assumption.
$\qquad\qquad$ - The alternative represents a hypothesis aimed to support
$\qquad\qquad \bullet$ Left-tailed test when $H_1$: $M \lt M_0$ where $M$ refers to a sample mean and $M_0$ is represented a given population mean
$\qquad\qquad \bullet$ Right-tailed test when $H_1$: $M \gt M_0$
$\qquad\qquad \bullet$ Two-tailed test when $H_1$: $M \neq M_0$
$\qquad\qquad$ - If the sample data belongs to the tail region, accept $H_1$.
$\qquad$ 2. Choose a significance level $\alpha$ to determine the critical region, default as 0.05 (5%)
$\qquad$ 3. Calculate the test statistic and p-value
$\qquad$ 4. Make a decision based on the p-value and the significance level

$\quad$ Two types of tests are commonly used in hypothesis testing: z-test and t-test.

Z-test
$\quad$ Z-test is adopted when 1. the samples follow a normal distribution and 2. the number of the samples are large enough ($\mathbf{n \gt 30}$). During the test, the sample variance and the population variance are handled as equal.

cf. $n \lt 30$이더라도 normal distribution 따르면, z-test 사용할 것 ??

T-test
$\quad$ T-test is used when 1. the distribution of the sample is unknown (the standard deviation of the sample is unknown) or 2. the number of the samples are small ($\mathbf{5 \leq n \leq 30}$). The test introduces a degree of freedom, making the T value as

$\qquad \bullet$ One sample t-test
$\qquad\qquad$ One sample t-test refers to a test to determine whether the sample mean is significantly different from a known or hypothesized population mean. It can be considered as same as the z-test.

$\qquad \bullet$ Two samples t-test
$\qquad\qquad$ Two samples t-test is used to compare the means of two independent (different) groups.

$\qquad \bullet$ Paired sample t-test
$\qquad\qquad$ Paired sample t-test compares the means of two related groups to determine if the difference between them is zero.

References

\(\tag*{}\label{1} \text{[1] "Probability mass function", Wikipedia, [Online] Available: https://en.wikipedia.org/wiki/Probability_mass_function, accessed in Aug. 28th, 2024.}\) \(\tag*{}\label{2} \text{[2] "Probability density function", Wikipedia, [Online] Available: https://en.wikipedia.org/wiki/Probability_density_function, accessed in Aug. 28th, 2024.}\)

Appendix

Practices

a. Let $X$~Bernoulli($p$) and $Y$~Bernoulli($q$) are independent. Find joint PMF and joint CDF for $X$ and $Y$.

Note that $P(X=0) = 1-p, P(X=1) = p, P(Y=0) = 1-q, P(Y=1) = q$.

a. I roll a fair die. Let $X$ be the observed number. Find the conditional PMF of $X$ given that we know the number was less than 5.

b. Consider a set of points in the grid shown below. These are points in set $G = {(x,y) | x,y \in Z, |x| + |y| \leq 2}$. Suppose we pick a point randomly. (P = $1 \over 13$)
b-1. Find the marginal PMF of $X$, $Y$ and their joint PMF.

b-2. Find the conditional PMF of $X$ given $Y=1$.

b-3. Are $X$ and $Y$ independent?

$\qquad$ No, they are not independent since they are subjected to the constraint $|x| + |y| \leq 2$

c. Let X and Y be the same as in b.
c-1. Find $E[X | Y = 1]$.

c-2. Find $E[X | -1 \leq Y \leq 2]$.

c-3. Find $E[ |X| | -1 \leq Y \leq 2]$.

a. Let $X$~Geometric($p$). Find $E[X]$ by conditioning on the result of the first coin toss.

$\qquad$ From a point of view, this is a process of finding the expectation of a geometric random variable.

a. Let $X$ and $Y$ be two jointly continuous random variables with joint PDF given below. \

a-1. Find the value of $C$ that makes $f_{XY}(x, y)$ a valid joint PDF.

a-2. Find $P(0 \leq X \leq 1/2, 0 \leq Y \leq 1/2)$.

a. In the previous problem D.a, find the marginal PDFs of $X$ and $Y$.

b. Let $X$ and $Y$ be two random variables with the joint PDF given below. \

b-1. Find $R_{xy}$ and show it in $X$ and $Y$ planes.

b-2. Find constant $C$ that makes $f_{XY}(x, y)$ a valid joint PDF.

b-3. Find Marginal PDFs of $X$ and $Y$.

b-4. Find $P(Y \leq 1/2)$.

b-5. Find $P(Y \leq \frac{X}{4} \mid Y \leq \frac{X}{2})$.

c. Let $X$ and $Y$ be two jointly contious random variables with joint PDF given below. \

c-1. Show the range $R_{xy}$ in $X$ and $Y$ planes.

c-2. Find constant $C$ that makes $f_{xy}(x, y)$ a valid joint PDF.

c-3. Find Marginal PDFs of $X$ and $Y$.

a. Let $X$ and $Y$ be two independent uniform random variables on $[0,1]$. Find $F_{XY}(x, y)$.

a. Let $X$ ~ Exponential(1). ($\lambda e^{-\lambda x} = e^{-x}$)
a-1. Find Conditional PDF of $X$ given $X > 1$.

a-2. Find $E[X \mid X > 1]$.

a-3. Find $Var(X \mid X > 1)$ = $E[X^2 \mid X > 1] - \left( E[X \mid X > 1] \right)^2$

a. Let $X$ and $Y$ be two jointly continuous random variables with the joint PDF given below. \

a-1. Find conditional PDF of $X$ given $Y=y$.

a-2. Find $P(X \lt \frac{1}{2} \mid Y = y)$.

a. Let $X$ and $Y$ be the same as the previous problem H.a.
a-1. Find $E[X \mid Y = 1]$ and $Var(X \mid Y = 1)$.

a. Determine if $X$ and $Y$ are independent if their joint PDF is given below.

a-1. Find $f_X(x)$ and $f_Y(y)$.

a-2. Is it independent?

$\qquad$ No, they are not independent.

a. Let $X$ and $Y$ be two jointly continuous random variables with the joint PDF given below. \

a-1. Find $E[XY^2]$.