Continuous-Type Random Variables and Their Distributions

Motivation: from peanuts to peanut butter

Discrete random variables put their probability in lumps on countably many values, like peanuts. Many quantities of interest have uncountably many possible values, and their probability is spread out, like peanut butter. Examples are the time a train arrives, a transistor’s lifetime, or a measurement error.

For these, $P\{X = v\} = 0$ for every single $v$ , so a pmf is useless.

IntuitionThe central question

What single object describes the distribution of every random variable — discrete, continuous, or a mixture — and when probability is spread out rather than lumped, how do we describe it locally, compute expectations with it, and model the simplest continuous phenomena: a value with no preferred location in a range, and a waiting time that has no memory?

Object	Exists for	Describes
CDF $F_X$	every random variable	$P\{X\le c\}$
pmf $p_X$	discrete type	the jumps of $F_X$
pdf $f_X$	continuous type	the slope of $F_X$

graph TD
  CDF["CDF: always exists"] -->|"jump sizes"| PMF["pmf: discrete type"]
  CDF -->|"derivative"| PDF["pdf: continuous type"]
  PMF -->|"sum up to c"| CDF
  PDF -->|"integrate up to c"| CDF

Cumulative distribution functions

DefinitionCumulative distribution function

For a random variable $X$ on $(\Omega,\mathcal F,P)$ , the CDF is $F_X(c) = P\{\omega : X(\omega)\le c\} = P\{X\le c\},\qquad c\in\mathbb R.$ To be a random variable, $X$ must make every set $\{X\le c\}$ an event (a member of $\mathcal F$ ). Then $F_X$ is always defined.

Component	Detail
Signature	$F_X : \mathbb R\to[0,1]$
Domain	a threshold $c$ , which need not be a possible value of $X$
Codomain	the probability that $X$ is at most $c$
Determines	$P\{X\in A\}$ for every (measurable) set $A$

DefinitionOne-sided limits and jumps

F(x-) = \lim_{y\uparrow x}F(y),\qquad F(x+) = \lim_{y\downarrow x}F(y), \qquad \Delta F(x) = F(x) - F(x-).

ExampleA fair die (Hajek)

$F_X$ is a staircase with six jumps of size $\frac16$ , at $1,\dots,6$ . At a jump the CDF takes its right-limit value: $F_X(1) = F_X(1+) = \frac16$ and $F_X(1-) = 0$ .

PropositionProposition 3.1.2 — strict inequalities and point masses

$P\{X < c\} = F_X(c-),\qquad P\{X = c\} = \Delta F_X(c).$

ProofCountable additivity along an increasing sequence

Step 1: Approach $c$ from below. Take $c_1 < c_2 < \cdots\to c$ . Let $G_1 = \{X\le c_1\}$ and $G_j = \{c_{j-1} < X\le c_j\}$ . These are disjoint, $\{X\le c_n\} = G_1\cup\cdots\cup G_n$ , and $\{X < c\} = \bigcup_jG_j$ .

Step 2: Add. By Axiom P.2,

P\{X < c\} = \sum_jP(G_j) = \lim_n\sum_{k\le n}P(G_k) = \lim_nF_X(c_n) = F_X(c-).

Step 3: Subtract. $P\{X = c\} = P\{X\le c\} - P\{X < c\} = F_X(c) - F_X(c-)$ .

This is continuity of probability (probability-foundations) applied to the events $\{X\le c_n\}\uparrow\{X < c\}$ .

∎

FormulaInterval probabilities from the CDF

Event	Probability
$\{a < X\le b\}$	$F(b) - F(a)$
$\{a\le X\le b\}$	$F(b) - F(a-)$
$\{a < X < b\}$	$F(b-) - F(a)$
$\{X\ge c\}$	$1 - F(c-)$
$\{X > c\}$	$1 - F(c)$

ExampleReading CDFs with jumps (Hajek)

One jump. $F$ rises to 0.5 just before 0, then jumps to 1 at 0. So $P\{X = 0\} = 0.5$ and no other point has positive mass. $P\{X\le0\} = F(0) = 1$ , but $P\{X < 0\} = F(0-) = 0.5$ .

Mixed CDF. $F$ climbs linearly from 0 at $u = 0$ to 0.25 just before 5, jumps to 0.5 at 5, stays flat until 10, jumps to 0.75 at 10, then climbs to 1 by 15:

$P\{X\le1\} = F(1) = 0.05$ ;
$P\{X\le10\} = F(10) = 0.75$ ;
$P\{X\ge10\} = 1 - F(10-) = 0.5$ ;
$P\{X = 10\} = \Delta F(10) = 0.25$ ;
$P\{|X - 5|\le0.1\} = F(5.1) - F(4.9-) = 0.5 - 0.245 = 0.255$ . Most of this is the jump of 0.25 at 5.

ExampleReading a CDF with jumps and a ramp (Ross Ex. 10a)

$F(x) = 0$ for $x < 0$ ; $\frac x2$ on $[0,1)$ ; $\frac23$ on $[1,2)$ ; $\frac{11}{12}$ on $[2,3)$ ; and 1 for $x\ge3$ .

$P\{X < 3\} = F(3-) = \frac{11}{12}$ ;
$P\{X = 1\} = F(1) - F(1-) = \frac23 - \frac12 = \frac16$ ;
$P\{X > \frac12\} = 1 - F(\frac12) = \frac34$ ;
$P\{2 < X\le4\} = F(4) - F(2) = \frac1{12}$ .

TheoremProposition 3.1.5 — what makes a function a CDF

$F$ is the CDF of some random variable iff

F.1 $F$ is nondecreasing;
F.2 $\lim_{c\to+\infty}F(c) = 1$ and $\lim_{c\to-\infty}F(c) = 0$ ;
F.3 $F$ is right-continuous: $F(c) = F(c+)$ .

ProofThe “only if” direction via continuity of probability (Ross §4.10)

F.1: for $a < b$ , $\{X\le a\}\subseteq\{X\le b\}$ .

F.2: if $b_n\uparrow\infty$ , the events $\{X\le b_n\}$ increase to $\{X < \infty\} = \Omega$ , so $F(b_n)\to1$ . Symmetrically, if $b_n\downarrow-\infty$ they decrease to $\emptyset$ , so $F(b_n)\to0$ .

F.3: if $b_n\downarrow b$ , the events $\{X\le b_n\}$ decrease to $\{X\le b\}$ , so $F(b_n)\to F(b)$ .

Each step is probability-foundations. The “if” direction, constructing a random variable with a given $F$ , is not proved in either source.

∎

ExampleSpotting invalid CDFs (Hajek)

Candidate functions fail in characteristic ways:

they decrease somewhere (violates F.1);
they don’t tend to 0 at $-\infty$ (violates F.2);
at a jump they take the left value rather than the right (violates F.3).

IntuitionWhy right-continuous?

$F(c)$ is defined with " $\le$ ", so the mass at $c$ is included in $F(c)$ . Approaching from the right, the events $\{X\le c + 1/n\}$ shrink down to $\{X\le c\}$ , and continuity of probability gives $F(c+) = F(c)$ . From the left, the mass at $c$ is missing until you arrive.

Types of random variables

DefinitionDiscrete, continuous, mixed

Discrete type: countably many values carry all the probability. $p_X(u) = \Delta F_X(u)$ , $F_X(c) = \sum_{u\le c}p_X(u)$ (3.1), and $F_X$ is piecewise constant.
Continuous type: $F_X(c) = \int_{-\infty}^cf_X(u)\,du$ for some function $f_X$ .
Mixed: neither, as in the mixed CDF example above (a continuous part plus jumps). Mixed variables still have CDFs.

Important“The distribution” means any equivalent description

A CDF, a pmf, a pdf, or a transform of one of these all carry the same information about $X$ alone. Pick whichever makes the problem easiest. For discrete variables, the pmf is usually more useful than the CDF.

Continuous-type random variables

DefinitionProbability density function (Hajek Def. 3.2.1; Ross eq. 1.1)

$X$ is continuous-type (Ross: absolutely continuous) if there is a function $f_X\ge0$ , the pdf, with

F_X(c) = \int_{-\infty}^cf_X(u)\,du\quad\text{for all }c,\qquad \text{equivalently}\qquad P\{X\in B\} = \int_Bf_X(x)\,dx

for every measurable $B$ . The support is $\{u : f_X(u) > 0\}$ .

Component	Detail
Signature	$f_X : \mathbb R\to[0,\infty)$
Domain	a point $u$
Codomain	a probability per unit length, which can exceed 1
Not	$P\{X = u\}$ , which is 0

TheoremBasic properties

Density is slope: where $f_X$ is continuous, $f_X = F_X'$ (by the fundamental theorem of calculus).
No point masses: $F_X$ is continuous, so $P\{X = v\} = 0$ for every $v$ .
Endpoints don’t matter: $P\{a < X\le b\} = P\{a\le X\le b\} = P\{a < X < b\} = P\{a\le X < b\} = \int_a^bf_X.$
Normalization: $f_X\ge0$ and $\int_{-\infty}^\infty f_X = 1$ .

WarningZero-probability points still add up to 1

Every value has probability 0, yet $X$ takes some value. There is no contradiction: additivity (Axiom P.2) covers only countable unions, and the real line is uncountable (probability-foundations).

FormulaWhat density means

If $f_X$ is continuous at $u_0$ ,

f_X(u_0) = \lim_{\epsilon\to0}\frac{P\{u_0 - \frac\epsilon2 < X < u_0 + \frac\epsilon2\}}\epsilon,\qquad P\Big\{u_0 - \frac\epsilon2 < X < u_0 + \frac\epsilon2\Big\} = \epsilon f_X(u_0) + o(\epsilon). \tag{3.3–3.4}

” $f(x)\,dx\approx P\{x\le X\le x + dx\}$ .” A large $f_X(u)$ means $X$ is likely to be near $u$ .

DerivationVia the symmetric derivative

\frac{F(u_0 + h) - F(u_0 - h)}{2h} = \frac12\Big[\frac{F(u_0 + h) - F(u_0)}h + \frac{F(u_0 - h) - F(u_0)}{-h}\Big]\to f_X(u_0).

Set $h = \epsilon/2$ .

WarningA density is not a probability

$f_X$ can exceed 1. For example, uniform on $[0,\frac12]$ has $f\equiv2$ . What must be at most 1 is $\int_Bf$ . Density has units of 1/(units of $X$ ).

Expectation, LOTUS, variance

Replace sums by integrals and the pmf by the pdf:

FormulaMoments of a continuous random variable

E[X] = \int_{-\infty}^\infty uf_X(u)\,du,\qquad E[g(X)] = \int_{-\infty}^ \infty g(u)f_X(u)\,du\quad(\text{LOTUS}),

\mathrm{Var}(X) = E[(X - \mu_X)^2] = E[X^2] - \mu_X^2,\qquad \mathrm{Var}(g(X)) = \int g^2f_X - \Big(\int gf_X\Big)^2.

Linearity ( $E[aX^2 + bX + c] = aE[X^2] + bE[X] + c$ ), $\mathrm{Var}(aX + b) = a^2\mathrm{Var}(X)$ , and standardization all carry over unchanged.

IntuitionWhy the integral is the right definition

$f(x)\,dx\approx P\{x\le X\le x + dx\}$ . So $\sum x\,P\{X = x\}$ becomes $\int x\,f(x)\,dx$ , a weighted average with infinitesimal weights.

LemmaTail-integral formula (Ross Lemma 2.1)

For a nonnegative random variable $Y$ , $E[Y] = \int_0^\infty P\{Y > y\}\,dy.$

ProofSwap the order of integration

For continuous $Y$ with density $f_Y$ ,

\int_0^\infty P\{Y > y\}\,dy = \int_0^\infty\int_y^\infty f_Y(x)\,dx\,dy = \int_0^\infty\Big(\int_0^xdy\Big)f_Y(x)\,dx = \int_0^\infty xf_Y(x)\,dx.

∎

RemarkThe discrete version

For $L$ on $\{0,1,\dots\}$ , $E[L] = \sum_{k\ge0}P\{L > k\}$ . For a geometric $L$ this gives $\sum_k(1-p)^k = 1/p$ , recovering the mean in discrete-random-variables-and-distributions.

ProofLOTUS for

g\ge0

, from the lemma (Ross)

E[g(X)] = \int_0^\infty P\{g(X) > y\}\,dy = \int_0^\infty\int_{x:\,g(x) > y}f(x)\,dx\,dy = \int_{x:\,g(x) > 0}\Big(\int_0^{g(x)}dy\Big)f(x)\,dx = \int g(x)f(x)\,dx.

The general case splits $g$ into positive and negative parts.

∎

Worked densities

ExampleA symmetric quadratic density (Hajek)

$f_X(u) = A(1 - u^2)$ on $[-1,1]$ .

Normalize: $A\big[u - \frac{u^3}3\big]_{-1}^1 = \frac{4A}3 = 1$ , so $A = \frac34$ .
Probability: integrate only over the part of $[0.5,1.5]$ inside the support. $P\{0.5 < X < 1.5\} = \frac34\big[\frac23 - \frac{11}{24}\big] = \frac5{32}$ .
CDF: $F_X(c) = 0$ for $c\le-1$ , $\frac{2 + 3c - c^3}4$ on $(-1,1)$ , and $1$ for $c\ge1$ .
Moments: $uf_X(u)$ is odd, so $\mu = 0$ . $E[X^2] = \frac34\int_{-1}^1(u^2 - u^4)\,du = 0.2$ , so $\mathrm{Var}(X) = 0.2$ and $\sigma\approx0.45$ .

ExampleA quadratic density on $(0,2)$ , and a linear one (Ross)

$f(x) = C(4x - 2x^2)$ on $(0,2)$ : $C\cdot\frac83 = 1$ gives $C = \frac38$ , and $P\{X > 1\} = \frac12$ (by symmetry about 1).
$f(x) = 2x$ on $[0,1]$ : $E[X] = \int2x^2 = \frac23$ , $E[X^2] = \frac12$ , $\mathrm{Var}(X) = \frac12 - \frac49 = \frac1{18}$ .

ExampleContinuous lifetimes, binomial counts (Ross)

A radio tube’s lifetime has density $f(x) = 100/x^2$ for $x > 100$ . Each tube fails by hour 150 with probability $\int_{100}^{150}100x^{-2}\,dx = \frac13$ . With 5 tubes failing independently,

P(\text{exactly 2 replaced}) = \binom52\Big(\frac13\Big)^2\Big(\frac23 \Big)^3 = \frac{80}{243}.

(This density’s mean is infinite: $\int_{100}^\infty x\cdot100x^{-2}\,dx$ diverges. That observation is mine: it is the Pareto distribution with $\lambda = 1$ , $a = 100$ ; see Pareto distribution.)

ExampleThe density of $Y = 2X$ , two ways (Ross)

Via the CDF: $F_Y(a) = P\{X\le a/2\} = F_X(a/2)$ , so $f_Y(a) = \frac12f_X(a/2)$ .

Via the density interpretation: $\epsilon f_Y(a)\approx P\{a - \frac\epsilon2\le2X\le a + \frac\epsilon2\} = P\{\frac a2 - \frac\epsilon4 \le X\le\frac a2 + \frac\epsilon4\}\approx\frac\epsilon2f_X(a/2)$ .

Stretching the variable by 2 halves the density, so the total area stays 1.

Example $E[e^X]$ for $X$ uniform on $(0,1)$ , two ways (Ross)

Find the distribution of $Y = e^X$ : $F_Y(x) = P\{X\le\log x\} = \log x$ on $[1,e]$ , so $f_Y = 1/x$ and $E[Y] = \int_1^e\,dx = e - 1$ .

LOTUS: $\int_0^1e^x\,dx = e - 1$ . Same answer, less work.

ExampleWhich piece contains the point $p$ ? (Ross)

A unit stick is broken at a uniform point $U$ . The piece containing $p$ has length $1 - U$ if $U < p$ and $U$ if $U > p$ : $E[L_p(U)] = \int_0^p(1 - u)\,du + \int_p^1u\,du = \frac12 + p(1 - p).$ This is maximized ( $\frac34$ ) at $p = \frac12$ . The piece containing a given point tends to be the longer one, another size-bias effect.

ExampleWhen to leave for an appointment (Ross)

Arriving $s$ minutes early costs $cs$ , and $s$ minutes late costs $ks$ . The travel time $X$ has density $f$ and CDF $F$ . Leaving $t$ minutes ahead gives

\frac d{dt}E[C_t(X)] = (k + c)F(t) - k = 0\;\Rightarrow\;F(t^*) = \frac k{k+c}.

Leave at the $\frac k{k+c}$ quantile of travel time. This is the continuous version of the newsvendor’s critical ratio in discrete-random-variables-and-distributions: when lateness is costly, leave early.

Uniform distribution

DefinitionUniform $[a,b]$

f_X(u) = \begin{cases}\frac1{b-a} & a\le u\le b\\ 0 &\text{else}\end{cases} \qquad F_X(c) = \begin{cases}0 & c\le a\\ \frac{c-a}{b-a} & a < c < b\\ 1 & c\ge b.\end{cases}

“No preference within the range”: the probability of any subinterval equals its length divided by $b - a$ .

TheoremMoments

E[X] = \frac{a+b}2,\qquad E[X^2] = \frac{a^2 + ab + b^2}3,\qquad \mathrm{Var}(X) = \frac{(b-a)^2}{12}.

For $U\sim$ Uniform $[0,1]$ : $E[U^k] = \frac1{k+1}$ and $\mathrm{Var}(U) = \frac1{12}$ .

DerivationFactor differences of powers

$E[X] = \frac{b^2 - a^2}{2(b-a)} = \frac{a+b}2$ and $E[X^2] = \frac{b^3 - a^3}{3(b-a)} = \frac{a^2 + ab + b^2}3$ . Subtract $\big(\frac{a+b}2\big)^2$ to get $\frac{(a-b)^2}{12}$ .

IntuitionMean at the midpoint, spread as length squared

The balance point is the center. The variance scales with the square of the interval’s length, as it must, since variance has squared units.

ExampleUniform on $(0,10)$ (Ross)

$P\{X < 3\} = .3$ , $P\{X > 6\} = .4$ , $P\{3 < X < 8\} = .5$ : each is just the length divided by 10.

ExampleWaiting for a bus (Ross)

Buses come at 7:00, 7:15, 7:30. A passenger arrives at a time uniform on (0, 30) minutes past 7.

Waits under 5 minutes iff arriving in (10,15) or (25,30): probability $\frac{10}{30} = \frac13$ .
Waits over 10 minutes iff arriving in (0,5) or (15,20): probability $\frac13$ .

ExampleBertrand’s paradox (Ross)

What is the probability that a random chord of a circle of radius $r$ is longer than the side of the inscribed equilateral triangle?

Uniform distance from the center on $[0,r]$ : the chord is long iff the distance is less than $r/2$ . Probability $\frac12$ .
Uniform angle with the tangent on $[0^\circ,180^\circ]$ : the chord is long iff the angle is in $(60^\circ,120^\circ)$ . Probability $\frac13$ .

Both describe physical experiments: throwing a disk onto a table ruled with lines $2r$ apart gives $\frac12$ , and spinning a needle about a point on the rim gives $\frac13$ .

Important“Random” must say what is uniform

A uniform distribution is uniform in some parameterization, and different parameterizations give different answers. As with the two-child puzzle (conditional-probability-and-independence), the problem is underspecified until the sampling mechanism is stated.

Exponential distribution

DefinitionExponential $(\lambda)$

For a rate $\lambda > 0$ :

f_T(t) = \lambda e^{-\lambda t}\ (t\ge0),\qquad F_T(t) = 1 - e^{-\lambda t} \ (t\ge0),\qquad F_T^c(t) = P\{T > t\} = e^{-\lambda t}.

The pdf starts at height $\lambda$ and decays at rate $\lambda$ . A larger rate concentrates mass nearer 0.

Component	Detail
Parameter	rate $\lambda$ (events per unit time)
Complementary CDF	$e^{-\lambda t}$ , the survival probability
Typical use	time until an event that occurs “at random”: an earthquake, a failure, a call ending

TheoremMoments

E[T^n] = \frac n\lambda E[T^{n-1}] = \frac{n!}{\lambda^n},\qquad E[T] = \frac1\lambda,\qquad \mathrm{Var}(T) = \frac1{\lambda^2},\qquad \sigma_T = \frac1\lambda = E[T].

DerivationIntegration by parts

E[T^n] = \int_0^\infty t^n\lambda e^{-\lambda t}\,dt = \big[-t^ne^{-\lambda t}\big]_0^\infty + \int_0^\infty nt^{n-1}e^{-\lambda t}\,dt = \frac n\lambda E[T^{n-1}].

Then use induction from $E[T^0] = 1$ .

ImportantStandard deviation equals the mean

This mirrors the geometric distribution, whose $\sigma$ is close to its mean when $p$ is small. Waiting times without memory are as unpredictable as they are long.

ExampleExponential computations

Half-life (Hajek): $\lambda = \ln2$ gives $P\{T\ge t\} = 2^{-t}$ , and $P(T\le1\mid T\le2) = \frac{1 - \frac12}{1 - \frac14} = \frac23.$
Phone booth (Ross): a call lasts Exponential( $\frac1{10}$ ) minutes. $P\{\text{wait} > 10\} = e^{-1}\approx.368$ and $P\{10 < \text{wait} < 20\} = e^{-1} - e^{-2}\approx.233$ .
Computer lifetime (Ross): the density $\lambda e^{-x/100}$ forces $\lambda = \frac1{100}$ . Then $P\{50 < X < 150\} = e^{-1/2} - e^{-3/2} \approx.383$ and $P\{X < 100\} = 1 - e^{-1}\approx.632$ .

IntuitionMost lifetimes are shorter than the mean

$P\{T < E[T]\} = 1 - e^{-1}\approx63\%$ for every exponential. The median is $\frac{\ln2}\lambda\approx0.69/\lambda$ . (The median formula is mine.) The long right tail pulls the mean above the typical value.

Memorylessness

TheoremMemoryless property

For $s, t\ge0$ :

P(T > s + t\mid T > s) = P\{T > t\},\qquad\text{equivalently}\qquad P\{T > s + t\} = P\{T > s\}P\{T > t\}. \tag{5.1–5.2}

ProofRatio of survival functions

$\{T > s + t\}\subseteq\{T > s\}$ , so the conditional probability is $\frac{e^{-\lambda(s+t)}}{e^{-\lambda s}} = e^{-\lambda t}.$

∎

IntuitionA working component is as good as new

If a component has survived to age $s$ , its remaining life has the same distribution as a brand-new one’s. You don’t need to know its age.

TheoremOnly the exponential is memoryless (Ross)

If a nonnegative continuous $X$ is memoryless, then $X$ is exponential.

ProofCauchy’s functional equation

Step 1: Set up the equation. $g(x) = P\{X > x\}$ satisfies $g(s + t) = g(s)g(t)$ by (5.2).

Step 2: Rationals. $g(1) = g(1/n)^n$ , so $g(1/n) = g(1)^{1/n}$ and $g(m/n) = g(1)^{m/n}$ .

Step 3: Extend to all $x$ . $g$ is right-continuous (a survival function), so $g(x) = g(1)^x$ for all $x\ge0$ .

Step 4: Write as an exponential. $g(1) = g(\tfrac12)^2\ge0$ , so $g(x) = e^{-\lambda x}$ with $\lambda = -\log g(1)$ . This is the exponential survival function.

∎

ExampleThe post office (Ross)

Smith waits while Jones and Brown are each being served, with exponential service times of the same rate. What is the probability that Smith leaves last? $\frac12$ . When one of the others leaves, Smith starts service. By memorylessness, the remaining customer’s remaining time is a fresh exponential, identical in distribution to Smith’s. By symmetry, each finishes first with probability $\frac12$ .

ExampleThe battery and the 5000-mile trip (Ross)

Battery life is exponential with mean 10,000 miles. The chance of completing a 5000-mile trip is $e^{-5000/10000} = e^{-1/2}\approx.607$ , whatever the battery’s age. For a non-exponential life distribution you would need the current mileage $t$ : $\frac{1 - F(t + 5)}{1 - F(t)}.$

The exponential as a limit of geometrics

TheoremScaled geometric waits converge to the exponential (Hajek)

A clock ticks every $h$ seconds. A bulb fails at each tick with probability $p = \lambda h$ . The number of ticks $L_h$ is geometric $(\lambda h)$ , and the failure time is $T_h = hL_h$ . Then for every $c\ge0$ ,

P\{T_h > c\} = (1 - \lambda h)^{\lfloor c/h\rfloor}\;\xrightarrow[h\to0] {}\;e^{-\lambda c}.

So $T_h$ converges in distribution to Exponential $(\lambda)$ .

ProofCompound-interest limit

Step 1: $P\{T_h > c\} = P\{L_h > \lfloor c/h\rfloor\} = (1 - \lambda h)^{\lfloor c/h\rfloor}$ by the geometric tail (2.8).

Step 2: $(1 - \lambda h)^{1/h}\to e^{-\lambda}$ (from $(1 - \lambda/n)^n\to e^{-\lambda}$ ), so $(1 - \lambda h)^{c/h}\to e^{-\lambda c}$ .

Step 3: the floor doesn’t matter, since $1\ge(1 - \lambda h)^{c/h}/(1 - \lambda h)^{\lfloor c/h\rfloor}\ge1 - \lambda h\to1$ .

∎

IntuitionContinuous time is fine-grained discrete time

Memoryless in discrete time (geometric) becomes memoryless in continuous time (exponential). The per-tick probability $\lambda h$ becomes a rate $\lambda$ . This is also why simulations can approximate exponential times by a small time step.

Hazard (failure) rate functions

DefinitionHazard rate (Ross §5.5.1)

For a positive continuous lifetime $X$ with CDF $F$ and density $f$ : $\lambda(t) = \frac{f(t)}{\bar F(t)},\qquad \bar F = 1 - F.$ Then $P\{X\in(t, t + dt)\mid X > t\}\approx\lambda(t)\,dt$ : the conditional failure intensity of a $t$ -unit-old item.

Component	Detail
Signature	$\lambda : [0,\infty)\to[0,\infty)$
Domain	an age $t$
Codomain	failure probability per unit time, given survival to $t$
Discrete analogue	$h(k) = P(L = k\mid L\ge k)$ (discrete-random-variables-and-distributions)

TheoremThe hazard determines the distribution

$F(t) = 1 - \exp\Big\{-\int_0^t\lambda(s)\,ds\Big\}. \tag{5.4}$

ProofIntegrate a logarithmic derivative

$\lambda(s) = \frac{f(s)}{1 - F(s)} = -\frac d{ds}\log(1 - F(s))$ . So $\int_0^t\lambda = -\log(1 - F(t))$ , using $F(0) = 0$ . Exponentiate.

∎

TheoremConstant hazard ⟺ exponential ⟺ memoryless

The exponential has $\lambda(t) = \frac{\lambda e^{-\lambda t}} {e^{-\lambda t}} = \lambda$ , which is why $\lambda$ is called the rate.
Conversely, a constant hazard $c$ gives $F(t) = 1 - e^{-ct}$ by (5.4).
A memoryless lifetime must have the same remaining-life distribution at every age, hence a constant hazard, hence it is exponential. This is a second proof of uniqueness.

ExampleLinear hazard gives the Rayleigh law (Ross)

$\lambda(t) = a + bt$ gives $F(t) = 1 - e^{-at - bt^2/2}$ and $f(t) = (a + bt)e^{-(at + bt^2/2)}$ . With $a = 0$ this is the Rayleigh density. An increasing hazard models wear-out.

Example“Smokers die at twice the rate” (Ross)

Suppose $\lambda_s(t) = 2\lambda_n(t)$ at every age. The survival probability from age $A$ to $B$ is $\exp\{-\int_A^B\lambda\}$ , so

P(\text{smoker survives }A\to B) = \big[P(\text{nonsmoker survives }A\to B)\big]^2.

Survival is squared, not halved. With $\lambda_n = \frac1{30}$ on ages 50–60: nonsmoker $e^{-1/3}\approx.7165$ , smoker $e^{-2/3}\approx.5134$ .

IntuitionHazards add in the exponent

Doubling a rate doubles the integrated hazard, which squares the survival probability. Rates compose multiplicatively on survival, just as independent series components multiply their survival probabilities.

Linear scaling of pdfs and the Gaussian distribution

Linear scaling of pdfs

TheoremDensity of $aX + b$

If $X$ has density $f_X$ and $a\ne0$ , then $Y = aX + b$ has density $f_Y(y) = \frac1{|a|}f_X\Big(\frac{y - b}a\Big).$

ProofThrough the CDF

$a > 0$ : $F_Y(y) = P\{X\le\frac{y-b}a\} = F_X\big(\frac{y-b}a\big)$ . Differentiate.

$a < 0$ : the inequality flips, so $F_Y(y) = P\{X\ge\frac{y-b}a\} = 1 - F_X\big(\frac{y-b}a\big)$ , and differentiating gives $-\frac1af_X(\cdot) = \frac1{|a|}f_X(\cdot)$ .

This is also the monotone case of the change-of-variables theorem (Functions of a random variable).

∎

IntuitionShift slides the graph; scale stretches and flattens it

Adding $b$ slides the density sideways. Multiplying by $a$ stretches it horizontally by $|a|$ and divides its height by $|a|$ , so the area stays 1. The $Y = 2X$ example above is the case $a = 2$ , $b = 0$ .

NoteTODO — Hajek’s presentation

Hajek’s own section on scaling pdfs is not yet in sources/. The result above is from Ross §5.4 and §5.7.

The normal (Gaussian) distribution

DefinitionNormal $(\mu,\sigma^2)$

f(x) = \frac1{\sqrt{2\pi}\,\sigma}e^{-(x-\mu)^2/2\sigma^2},\qquad -\infty < x < \infty.

A bell curve symmetric about $\mu$ . Standard normal $Z$ : $\mu = 0$ , $\sigma = 1$ .

ProofIt integrates to 1: the polar-coordinates trick

Step 1: Standardize. Substituting $y = (x-\mu)/\sigma$ , it suffices to show $I = \int e^{-y^2/2}dy = \sqrt{2\pi}$ .

Step 2: Square it. $I^2 = \iint e^{-(x^2+y^2)/2}\,dx\,dy$ .

Step 3: Go polar. With $dx\,dy = r\,dr\,d\theta$ ,

I^2 = \int_0^\infty\int_0^{2\pi}e^{-r^2/2}r\,d\theta\,dr = 2\pi\big[-e^{-r^2/2} \big]_0^\infty = 2\pi.

The key step: a one-dimensional integral with no elementary antiderivative becomes an easy two-dimensional one.

∎

TheoremLinear transformations stay normal

If $X\sim N(\mu,\sigma^2)$ , then $aX + b\sim N(a\mu + b,\,a^2\sigma^2)$ . In particular $Z = (X - \mu)/\sigma\sim N(0,1)$ .

ProofApply the scaling rule

For $a > 0$ , $f_Y(x) = \frac1af_X\big(\frac{x-b}a\big) = \frac1{\sqrt{2\pi}a\sigma}\exp\{-(x - b - a\mu)^2/2(a\sigma)^2\}$ , which is the $N(a\mu + b, a^2\sigma^2)$ density. $a < 0$ is similar.

∎

TheoremThe parameters are the mean and variance

E[Z] = 0,\quad\mathrm{Var}(Z) = 1;\qquad E[X] = \mu,\quad\mathrm{Var}(X) = \sigma^2.

ProofOdd integrand, then integration by parts

$E[Z] = \frac1{\sqrt{2\pi}}\int xe^{-x^2/2}dx = \big[-\frac1{\sqrt{2\pi}} e^{-x^2/2}\big]_{-\infty}^\infty = 0$ .

$\mathrm{Var}(Z) = \frac1{\sqrt{2\pi}}\int x\cdot xe^{-x^2/2}dx$ . By parts with $u = x$ , $dv = xe^{-x^2/2}dx$ , this is $\frac1{\sqrt{2\pi}}\int e^{-x^2/2}dx = 1$ .

Then $X = \mu + \sigma Z$ gives $E[X] = \mu$ and $\mathrm{Var}(X) = \sigma^2$ .

∎

DefinitionThe standard normal CDF $\Phi$

\Phi(x) = \frac1{\sqrt{2\pi}}\int_{-\infty}^xe^{-y^2/2}dy,\qquad \Phi(-x) = 1 - \Phi(x)\ \text{(by symmetry)},\qquad F_X(a) = \Phi\Big(\frac{a - \mu} \sigma\Big).

$\Phi$ has no closed form. Use a table, a calculator or software.

$x$	0.5	1.0	1.5	1.645	2.0	2.33	2.5	3.0
$\Phi(x)$	.6915	.8413	.9332	.95	.9772	.99	.9938	.9987

Component	Detail
Signature	$\Phi : \mathbb R\to(0,1)$
Domain	a number of standard deviations
Codomain	the probability that a standard normal falls below it
Use	every normal probability, after standardizing

ExampleStandardize, then look up (Ross)

$X\sim N(3, 9)$ , so $\sigma = 3$ :

$P\{2 < X < 5\} = \Phi(\frac23) - \Phi(-\frac13) = \Phi(\frac23) - 1 + \Phi(\frac13)\approx.3779$ ;
$P\{X > 0\} = P\{Z > -1\} = \Phi(1)\approx.8413$ ;
$P\{|X - 3| > 6\} = P\{|Z| > 2\} = 2[1 - \Phi(2)]\approx.0456$ .

ExampleGrading on the curve (Ross)

With A above $\mu + \sigma$ , B in $(\mu,\mu+\sigma)$ , C in $(\mu - \sigma,\mu)$ , D in $(\mu - 2\sigma,\mu - \sigma)$ and F below:

Grade	A	B	C	D	F
share	.1587	.3413	.3413	.1359	.0228

About 16%, 34%, 34%, 14% and 2%. Equivalently, about 68% of a normal lies within $1\sigma$ of the mean and about 95% within $2\sigma$ .

ExampleNormal models in practice (Ross)

Paternity case: gestation is $N(270, 100)$ days. The defendant was away from 290 to 240 days before the birth. If he is the father, the gestation was over 290 or under 240 days: $1 - \Phi(2) + 1 - \Phi(3)\approx.0241$ .
Noisy binary channel: send $+2$ for 1 and $-2$ for 0, receive $R = x + N$ with $N\sim N(0,1)$ , and decode 1 iff $R\ge.5$ . $P(\text{error}\mid1) = P\{N < -1.5\}\approx.0668$ and $P(\text{error}\mid0) = P\{N\ge2.5\}\approx.0062$ .
Value at risk: the VAR is the loss $\nu$ exceeded with only 1% probability. For a normal gain $X\sim N(\mu,\sigma^2)$ , $.01 = 1 - \Phi\big(\frac{\nu+\mu}\sigma\big)$ gives $\nu = 2.33\sigma - \mu$ . Among normal investments, the smallest VAR belongs to the one with the largest $\mu - 2.33\sigma$ .

The DeMoivre–Laplace limit theorem

TheoremNormal approximation to the binomial

If $S_n$ is binomial $(n,p)$ , then for any $a < b$ ,

P\Big\{a\le\frac{S_n - np}{\sqrt{np(1-p)}}\le b\Big\}\;\to\;\Phi(b) - \Phi(a)\qquad(n\to\infty).

NoteProof deferred

This is a special case of the central limit theorem (concentration-inequalities-and-limit-theorems), and Ross does not prove it separately. DeMoivre proved the $p = \frac12$ case in 1733, and Laplace extended it in 1812.

ImportantTwo approximations to the binomial

Regime	Approximation
$n$ large, $p$ small, $np$ moderate	Poisson $(np)$ (discrete-random-variables-and-distributions)
$np(1-p)$ large (Ross: $\ge10$ )	Normal $(np,\,np(1-p))$

DefinitionContinuity correction

A binomial is integer-valued, so before approximating write $P\{X = i\} = P\{i - \frac12 < X < i + \frac12\}$ and $P\{X\ge i\} = P\{X\ge i - \frac12\}$ .

ExampleThe approximation at work (Ross)

20 heads in 40 fair flips: the normal approximation gives $\Phi(.16) - \Phi(-.16)\approx.1272$ . Exact: $\binom{40}{20}2^{-40}\approx.1254$ .
Admissions: 450 are accepted and each attends with probability .3. $P(\text{more than 150 attend}) = P\{X\ge150.5\}\approx1 - \Phi(1.59) \approx.0559$ .
A useless diet: each of 100 people’s cholesterol drops with probability $\frac12$ by chance. $P(\ge65\text{ drop}) = P\{X\ge64.5\}\approx1 - \Phi(2.9)\approx.0019$ .

ExampleHow big must a poll be? (Ross)

52% of New Yorkers favor a proposal. In a random sample of $n$ , the count is hypergeometric but nearly binomial $(n,.52)$ (discrete-random-variables-and-distributions), so $P\{\text{majority of the sample in favor}\}\approx\Phi(.04\sqrt n).$ This is .5528 at $n = 11$ , .6562 at $n = 101$ and .8973 at $n = 1001$ . Exceeding .95 needs $.04\sqrt n > 1.645$ , i.e. $n\ge1692$ .

RemarkWhy “normal”

Gauss used the curve in 1809 to predict astronomical positions, hence “Gaussian.” By the late 19th century many believed well-behaved data were “normally” bell-shaped, and Karl Pearson’s usage made “normal curve” standard. The central limit theorem is the partial explanation.

Other continuous distributions

Laplace (double exponential) distribution

DefinitionLaplace $(\lambda)$

Equally likely positive or negative, with exponential $(\lambda)$ absolute value:

f(x) = \frac12\lambda e^{-\lambda|x|},\qquad F(x) = \begin{cases}\frac12e^{ \lambda x} & x < 0\\ 1 - \frac12e^{-\lambda x} & x > 0.\end{cases}

ExampleThe binary channel with Laplacian noise (Ross)

Same channel as above, now with Laplace(1) noise: $P(\text{error}\mid1) = P\{N < -1.5\} = \frac12e^{-1.5}\approx.1116$ and $P(\text{error}\mid0) = P\{N\ge2.5\} = \frac12e^{-2.5}\approx.041$ . Both are worse than with standard normal noise (.0668 and .0062), because the Laplace tails decay like $e^{-|x|}$ rather than $e^{-x^2/2}$ .

Gamma distribution

DefinitionGamma $(\alpha,\lambda)$

For $\alpha, \lambda > 0$ :

f(x) = \frac{\lambda e^{-\lambda x}(\lambda x)^{\alpha-1}}{\Gamma(\alpha)}, \quad x\ge0,\qquad \Gamma(\alpha) = \int_0^\infty e^{-y}y^{\alpha-1}dy.

TheoremThe gamma function extends the factorial

$\Gamma(\alpha) = (\alpha - 1)\Gamma(\alpha - 1)$ and $\Gamma(1) = 1$ , so $\Gamma(n) = (n-1)!$ for integers $n$ .

ProofIntegration by parts

$\Gamma(\alpha) = \big[-e^{-y}y^{\alpha-1}\big]_0^\infty + (\alpha - 1) \int_0^\infty e^{-y}y^{\alpha-2}dy = (\alpha - 1)\Gamma(\alpha - 1)$ . Iterate down to $\Gamma(1) = \int_0^\infty e^{-y}dy = 1$ .

∎

FormulaMean and variance

$E[X] = \frac\alpha\lambda,\qquad \mathrm{Var}(X) = \frac\alpha{\lambda^2}.$ The mean: $E[X] = \frac1{\lambda\Gamma(\alpha)}\int_0^\infty \lambda e^{-\lambda x}(\lambda x)^\alpha dx = \frac{\Gamma(\alpha+1)} {\lambda\Gamma(\alpha)} = \frac\alpha\lambda$ . Ross leaves the variance as an exercise (compute $E[X^2]$ the same way).

RemarkSpecial cases

Integer $\alpha = n$ : the $n$ -Erlang distribution, the time of the $n$ -th event of a Poisson process (discrete-random-variables-and-distributions).
$\alpha = 1$ : the exponential distribution.
$\lambda = \frac12$ , $\alpha = \frac n2$ : the chi-squared distribution with $n$ degrees of freedom. It arises as the squared error in hitting a target in $n$ -dimensional space with normal coordinate errors, and Ross develops it in Chapter 6.

Weibull distribution

DefinitionWeibull $(\nu,\alpha,\beta)$

F(x) = 1 - \exp\Big\{-\Big(\frac{x - \nu}\alpha\Big)^\beta\Big\},\ \ x > \nu, \qquad f(x) = \frac\beta\alpha\Big(\frac{x - \nu}\alpha\Big)^{\beta-1}\exp \Big\{-\Big(\frac{x - \nu}\alpha\Big)^\beta\Big\}.

It is widely used for lifetimes under a weakest-link model, in which an object fails when any of its many parts fails. It was originally proposed for fatigue data.

Cauchy distribution

DefinitionCauchy with parameter $\theta$

f(x) = \frac1\pi\cdot\frac1{1 + (x - \theta)^2},\qquad -\infty < x \infty.

ExampleA spinning flashlight (Ross)

A narrow-beam flashlight one unit from the $x$ -axis is spun, and $X$ is where the beam hits the axis. The angle $\theta$ is uniform on $(-\frac\pi2,\frac\pi2)$ and $X = \tan\theta$ , so

F(x) = P\{\theta\le\tan^{-1}x\} = \frac12 + \frac1\pi\tan^{-1}x,\qquad f(x) = \frac1{\pi(1 + x^2)}.

This is the Cauchy distribution with $\theta = 0$ .

WarningNo mean

The Cauchy tails decay like $1/x^2$ , so $\int|x|f(x)\,dx$ diverges and the Cauchy has no mean. (This remark is not in Ross.)

Beta distribution

DefinitionBeta $(a,b)$

f(x) = \frac{x^{a-1}(1 - x)^{b-1}}{B(a,b)},\quad 0 < x < 1,\qquad B(a,b) = \int_0^1x^{a-1}(1 - x)^{b-1}dx = \frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}. \tag{6.3}

It models a random quantity confined to an interval, which can be rescaled to $[0,1]$ .

RemarkShapes

$a = b$ is symmetric about $\frac12$ , and more concentrated as $a$ grows. $a = b = 1$ is the uniform. $b > a$ skews toward small values, and $a > b$ toward large ones.

TheoremMean and variance

$E[X] = \frac a{a+b},\qquad \mathrm{Var}(X) = \frac{ab}{(a+b)^2(a + b + 1)}.$

ProofRatios of beta functions

$E[X] = \frac{B(a+1,b)}{B(a,b)} = \frac{\Gamma(a+1)\Gamma(b)}{\Gamma(a+b+1)} \cdot\frac{\Gamma(a+b)}{\Gamma(a)\Gamma(b)} = \frac a{a+b}$ , using $\Gamma(x+1) = x\Gamma(x)$ . Similarly $E[X^2] = \frac{B(a+2,b)}{B(a,b)} = \frac{(a+1)a}{(a+b+1)(a+b)}$ . Subtract $(E[X])^2$ .

Identity (6.3) is proved later in Ross (Ch. 6).

∎

Pareto distribution

DefinitionPareto $(\lambda, a)$

If $X\sim$ exponential $(\lambda)$ and $a > 0$ , then $Y = ae^X$ is Pareto with index $\lambda$ and minimum $a$ :

P\{Y > y\} = \Big(\frac ay\Big)^\lambda,\qquad f_Y(y) = \lambda a^\lambda y^{-(\lambda+1)},\qquad y\ge a.

DerivationFrom the exponential

$P\{Y > y\} = P\{X > \log(y/a)\} = e^{-\lambda\log(y/a)} = (a/y)^\lambda$ .

TheoremMoments exist only below the index

E[Y] = \frac{\lambda a}{\lambda - 1}\ (\lambda > 1),\qquad \mathrm{Var}(Y) = \frac{\lambda a^2}{(\lambda - 2)(\lambda - 1)^2}\ (\lambda > 2),\qquad E[Y^n] = \frac{a^n\lambda}{\lambda - n}\ (\lambda > n).

$E[Y] = \infty$ if $\lambda\le1$ .

ProofVia

Y = ae^X

$E[Y^n] = a^nE[e^{nX}] = a^n\int_0^\infty e^{nx}\lambda e^{-\lambda x}dx = \frac{a^n\lambda}{\lambda - n}$ for $\lambda > n$ . The integral diverges otherwise.

∎

RemarkPower laws

The density is a constant times a power of $y$ , a power law. Pareto models fit incomes and wealth, internet file sizes, supercomputer job times, meteorite sizes, and yearly maximum rainfall. The radio-tube density $100/x^2$ above is Pareto with $\lambda = 1$ , $a = 100$ , which is why its mean is infinite. The discrete relative is the zeta distribution (discrete-random-variables-and-distributions).

Functions of a random variable

ImportantThe CDF method

To find the distribution of $Y = g(X)$ , write $\{g(X)\le y\}$ as an event about $X$ , compute its probability with $F_X$ , then differentiate.

ExampleThree CDF-method computations (Ross)

Powers of a uniform: $X\sim U(0,1)$ , $Y = X^n$ . For $0\le y\le1$ , $F_Y(y) = P\{X\le y^{1/n}\} = y^{1/n}$ , so $f_Y(y) = \frac1ny^{1/n-1}$ .
Squares: $Y = X^2$ . For $y\ge0$ , $F_Y(y) = F_X(\sqrt y) - F_X(-\sqrt y)$ , so $f_Y(y) = \frac1{2\sqrt y}\big[f_X(\sqrt y) + f_X(-\sqrt y)\big]$ .
Absolute values: $Y = |X|$ . $F_Y(y) = F_X(y) - F_X(-y)$ , so $f_Y(y) = f_X(y) + f_X(-y)$ for $y\ge0$ .

The last two are two-to-one maps: both $\pm\sqrt y$ (or $\pm y$ ) land on $y$ , so their densities add.

TheoremChange of variables for monotone $g$ (Ross Thm. 7.1)

Let $X$ have density $f_X$ , and let $g$ be strictly monotone and differentiable. Then $Y = g(X)$ has density

f_Y(y) = \begin{cases}f_X\big(g^{-1}(y)\big)\Big|\dfrac d{dy}g^{-1}(y)\Big| & y = g(x)\text{ for some }x,\\ 0 & \text{otherwise.}\end{cases}

Component	Detail
Input	a density $f_X$ and a strictly monotone, differentiable $g$
Output	the density of $g(X)$
Correction factor	$\lvert(g^{-1})'\rvert$ , how much $g^{-1}$ stretches a small interval
Not covered	many-to-one maps (use the CDF method, as above)

ProofIncreasing case

$F_Y(y) = P\{g(X)\le y\} = P\{X\le g^{-1}(y)\} = F_X(g^{-1}(y))$ . Differentiate by the chain rule. $g^{-1}$ is increasing, so its derivative is nonnegative and no absolute value is needed. For $y$ outside the range of $g$ , $F_Y$ is 0 or 1 there, so $f_Y = 0$ . The decreasing case flips the inequality and produces the absolute value.

∎

ExamplePowers of a nonnegative variable (Ross)

$X\ge0$ with density $f$ , and $Y = X^n$ : $g^{-1}(y) = y^{1/n}$ , so $f_Y(y) = \frac1ny^{1/n-1}f\big(y^{1/n}\big).$ For $n = 2$ this is $\frac1{2\sqrt y}f(\sqrt y)$ : the squares formula above with $f(-\sqrt y) = 0$ .

ExampleThe lognormal distribution (Ross)

If $X\sim N(\mu,\sigma^2)$ , then $Y = e^X$ is lognormal. With $g^{-1}(y) = \log y$ and $(g^{-1})'(y) = 1/y$ ,

f_Y(y) = \frac1{\sqrt{2\pi}\,\sigma y}\exp\Big\{-\frac{(\log y - \mu)^2} {2\sigma^2}\Big\},\qquad y > 0.

Stock prices: assuming each day’s price ratio $S_n/S_{n-1}$ is lognormal is the same as assuming $S_n = S_{n-1}e^X$ with $X$ normal, that is, normal log-returns.

NoteTODO — Generating a random variable with a specified distribution

Needs the inverse-CDF (and related) simulation method. Source text not yet extracted.

NoteTODO — The area rule for expectation based on the CDF

This is the tail-integral formula (Expectation, LOTUS, variance above, $E[Y] = \int_0^\infty P\{Y > y\}\,dy$ ) applied as a computational shortcut; needs confirmation against Ross’s exact presentation before folding the two together. Source text not yet extracted.

NoteThe multivariate case lives in a companion note

Transforming a joint pdf under linear, one-to-one, and many-to-one mappings is covered in jointly-distributed-random-variables (still a stub as of this writing — see its own TODO).

Quick reference

Object	Statement	Note
CDF	$F_X(c) = P\{X\le c\}$	exists for every random variable
Strict inequality and point mass	$P\{X < c\} = F(c-)$ , $P\{X = c\} = \Delta F(c)$	continuity of probability
Valid CDF	nondecreasing; limits 0 and 1; right-continuous	F.1–F.3
Discrete	$p_X = \Delta F$ , $F = \sum_{u\le c}p_X$	piecewise constant
Continuous	$F(c) = \int_{-\infty}^cf$ , $f = F'$	$P\{X = v\} = 0$
Density meaning	$P\{\lvert X - u\rvert < \epsilon/2\}\approx\epsilon f(u)$	$f$ can exceed 1
Mean, LOTUS	$\int uf$ , $\int g(u)f(u)\,du$	linearity carries over
Variance	$E[X^2] - \mu^2$ ; $\mathrm{Var}(aX + b) = a^2\mathrm{Var}(X)$
Tail formula	$E[Y] = \int_0^\infty P\{Y > y\}\,dy$ for $Y\ge0$	discrete: $\sum P\{L > k\}$
Scaling	$Y = 2X\Rightarrow f_Y(a) = \frac12f_X(a/2)$
Departure time	$F(t^*) = k/(k+c)$	critical ratio
Stick breaking	$E[L_p] = \frac12 + p(1-p)$	size bias
Uniform $[a,b]$	$f = \frac1{b-a}$ ; mean $\frac{a+b}2$ ; variance $\frac{(b-a)^2}{12}$	$E[U^k] = \frac1{k+1}$
Bertrand	$\frac12$ or $\frac13$	“random” is underspecified
Exponential $(\lambda)$	$f = \lambda e^{-\lambda t}$ ; $P\{T > t\} = e^{-\lambda t}$
Exponential moments	$E[T^n] = n!/\lambda^n$ ; $\sigma = \mu = 1/\lambda$	median $\ln2/\lambda$
Memoryless	$P(T > s + t\mid T > s) = P(T > t)$	only the exponential
Post office	Smith last with probability $\frac12$	memorylessness + symmetry
Geometric limit	$h\cdot$ Geom $(\lambda h)\to$ Exp $(\lambda)$	$(1 - \lambda h)^{c/h}\to e^{-\lambda c}$
Hazard	$\lambda(t) = f/\bar F$ ; $F = 1 - e^{-\int\lambda}$	constant ⟺ exponential
Linear hazard	$F = 1 - e^{-at - bt^2/2}$	Rayleigh when $a = 0$
Double the rate	square the survival probability	$.7165\to.5134$
Scaling	$f_{aX+b}(y) = \frac1{\lvert a\rvert}f_X(\frac{y-b}a)$	area stays 1
Normal	$N(\mu,\sigma^2)$ ; $F(a) = \Phi(\frac{a-\mu}\sigma)$	$\Phi(-x) = 1 - \Phi(x)$
Normal ranges	≈68% within $1\sigma$ , ≈95% within $2\sigma$	$aX+b$ stays normal
DeMoivre–Laplace	$(S_n - np)/\sqrt{np(1-p)}\approx Z$	continuity correction
Laplace	$\frac12\lambda e^{-\lambda\lvert x\rvert}$	heavier tails than normal
Gamma	mean $\alpha/\lambda$ , variance $\alpha/\lambda^2$ ; $\Gamma(n) = (n-1)!$	Erlang, exponential, $\chi^2$
Weibull	$1 - e^{-((x-\nu)/\alpha)^\beta}$	weakest link
Cauchy	$\frac1{\pi(1+x^2)}$	$\tan$ of a uniform angle; no mean
Beta	mean $\frac a{a+b}$ ; variance $\frac{ab}{(a+b)^2(a+b+1)}$	$B = \Gamma(a)\Gamma(b)/\Gamma(a+b)$
Pareto	$P\{Y > y\} = (a/y)^\lambda$ ; $E[Y^n] = \frac{a^n\lambda}{\lambda-n}$	power law
Change of variables	$f_Y = f_X(g^{-1})\lvert(g^{-1})'\rvert$	monotone $g$
Lognormal	$e^{N(\mu,\sigma^2)}$	normal log-returns

Facts to keep to hand

The CDF is universal; the pmf and pdf are special cases. Jumps of $F$ are point masses, its slope is density. Mixed variables have both and still have a CDF. Every interval probability follows from $F$ and its left limits.
Right-continuity comes from " $\le$ ". Mass at $c$ is included in $F(c)$ , so approaching from the right loses nothing. Approaching from the left gives $P\{X < c\}$ .
Continuous means no atoms. Points have probability 0, endpoints are irrelevant, and only countable additivity is required. The density is a rate (probability per unit length), not a probability, and can exceed 1.
Every discrete formula ports by replacing sums with integrals. This covers the mean, LOTUS, linearity, variance, standardization, and the critical-ratio optimization (newsvendor becomes departure time). The tail formula $E[Y] = \int P\{Y > y\}\,dy$ is a new tool that also holds discretely.
Uniform is only uniform in a stated coordinate. Its mean is the midpoint and its variance is length²/12. Bertrand’s paradox shows that “random” without a mechanism is not a model.
The exponential is the continuous geometric. It is the $h\to0$ limit of geometric waits, it is memoryless (uniquely so, by Cauchy’s functional equation), it has constant hazard, and its $\sigma$ equals its mean. Memorylessness powers symmetry arguments like the post office.
The hazard rate is the natural lifetime parameter. Survival is $e^{-\int\lambda}$ . Constant hazard means exponential, increasing hazard means wear-out (Rayleigh), and scaling the hazard by $k$ raises survival probabilities to the $k$ -th power.
Standardize, then look up. Every normal probability reduces to $\Phi$ at $(a-\mu)/\sigma$ . Linear maps keep normals normal, and binomials with large $np(1-p)$ are approximately normal (DeMoivre–Laplace), with a continuity correction.
Tails decide which model is safe. Normal tails fall like $e^{-x^2/2}$ , Laplace like $e^{-|x|}$ , and Pareto and Cauchy only polynomially. Polynomial tails can make moments infinite (Pareto with $\lambda\le1$ , Cauchy), and heavier noise tails mean more channel errors.
To transform a density, go through the CDF. For monotone maps this gives $f_X(g^{-1})|(g^{-1})'|$ . For many-to-one maps, add the contributions of every preimage.

References

Hajek, Probability with Engineering Applications (ECE 313 notes) — chapter 3, §§3.1–3.4 (scaling-rule and Gaussian sections pending)
Ross, A First Course in Probability — chapter 4, §4.10; chapter 5, §§5.1–5.7

Related