Information Theory and Portfolio Theory

$\frac1n\log S_n^* = \frac1n\sum_i\log\mathbf b^{*t}\mathbf X_i$, a sample mean of i.i.d. terms, which converges to $\mathbb E\log\mathbf b^{*t}\mathbf X = W^*$ almost surely. The horse-race version used only the weak law.

∎

Linear in $F$: $W$ is an integral against $dF$.

Concave in $\mathbf b$: $\log(\lambda\mathbf b_1 + (1-\lambda)\mathbf b_2)^t\mathbf X \ge \lambda\log\mathbf b_1^t\mathbf X + (1-\lambda)\log\mathbf b_2^t\mathbf X$ pointwise. Take expectations.

Convex in $F$: let $\mathbf b_\lambda^*$ be log-optimal for $\lambda F_1 + (1-\lambda)F_2$. By linearity in $F$,

$$W^*(\lambda F_1 + (1-\lambda)F_2) = \lambda W(\mathbf b_\lambda^*, F_1) + (1-\lambda)W(\mathbf b_\lambda^*, F_2) \le \lambda W^*(F_1) + (1-\lambda)W^*(F_2)$$

since no single portfolio beats each market’s own optimum.

∎

Step 1: Concavity reduces optimality to a first-order test. $W$ is concave on the simplex, so $\mathbf b^*$ is optimal iff, for every $\mathbf b\in\mathcal B$, the one-sided derivative along $\mathbf b_\lambda = (1-\lambda)\mathbf b^* + \lambda\mathbf b$ at $\lambda = 0^+$ is $\le 0$.

Step 2: Compute the derivative.

$$\frac{d}{d\lambda}\mathbb E\ln\mathbf b_\lambda^t\mathbf X\Big|_{0^+} = \lim_{\lambda\downarrow0}\frac1\lambda\mathbb E\ln\left(1 + \lambda\Big(\frac{\mathbf b^t\mathbf X}{\mathbf b^{*t}\mathbf X} - 1\Big)\right) = \mathbb E\frac{\mathbf b^t\mathbf X}{\mathbf b^{*t}\mathbf X} - 1$$

Dominated convergence justifies swapping the limit and the expectation. So the test is $\mathbb E\frac{\mathbf b^t\mathbf X}{\mathbf b^{*t}\mathbf X}\le1$ for all $\mathbf b$.

Step 3: Vertices suffice. The expression is linear in $\mathbf b$, so checking the vertices $\mathbf e_j$ is enough: $\mathbb E\frac{X_j}{\mathbf b^{*t}\mathbf X}\le1$ for every $j$.

Step 4: Equality for held stocks. If $b_j^* > 0$, the segment from $\mathbf e_j$ through $\mathbf b^*$ extends beyond $\mathbf b^*$ inside the simplex. Then the two-sided derivative exists and must vanish, which forces equality. If $b_j^* = 0$, the segment can’t extend, and only the inequality is required.

∎

(⇒) Multiply $\mathbb E\frac{X_i}{\mathbf b^{*t}\mathbf X}\le1$ by $b_i$ and sum over $i$: $\mathbb E\frac{\mathbf b^t\mathbf X}{\mathbf b^{*t}\mathbf X}\le\sum b_i = 1$.

(⇐) By Jensen, $\mathbb E\log\frac S{S^*}\le\log\mathbb E\frac S{S^*}\le\log1 = 0$.

∎

$\mathbb E\log S_n = \sum_i\mathbb E\log\mathbf b_i^t(\mathbf X^{i-1})\mathbf X_i$. Each term is maximized separately. Since $\mathbf X_i$ is independent of the past, the best day-$i$ choice ignores the past and equals $\mathbf b^*$. So the maximum is $nW^*$.

∎

Step 1: The ratio has mean at most 1. By one-day optimality, applied day by day with independence, $\mathbb E\frac{S_n}{S_n^*}\le1$.

Step 2: Markov. By Markov’s inequality, $\Pr\big(\frac{S_n}{S_n^*} > t_n\big) < \frac1{t_n}$, i.e. $\Pr\big(\frac1n\log\frac{S_n}{S_n^*} > \frac1n\log t_n\big) < \frac1{t_n}$.

Step 3: Choose $t_n = n^2$ and sum. $\sum_n\Pr\big(\frac1n\log\frac{S_n}{S_n^*} > \frac{2\log n}n\big)\le\sum_n\frac1{n^2} = \frac{\pi^2}6 < \infty$.

Step 4: Borel–Cantelli. Only finitely many of these events occur, almost surely. So eventually $\frac1n\log\frac{S_n}{S_n^*} < \frac{2\log n}{n}\rightarrow0$.

∎

$$\Delta W = \int f\log\frac{\mathbf b_f^t\mathbf x}{\mathbf b_g^t\mathbf x} = \int f\log\left(\frac{\mathbf b_f^t\mathbf x}{\mathbf b_g^t\mathbf x}\cdot\frac{g}{f}\right) + D(f\Vert g) \overset{(a)}{\le} \log\int g\,\frac{\mathbf b_f^t\mathbf x}{\mathbf b_g^t\mathbf x} + D(f\Vert g) \overset{(b)}{\le}\log1 + D(f\Vert g)$$

• (a) Jensen (log is concave), applied with weights $f$.
• (b) $\mathbf b_g$ is log-optimal under $g$, so by one-day optimality $\mathbb E_g\frac{\mathbf b_f^t\mathbf X}{\mathbf b_g^t\mathbf X}\le1$.

∎

Given $Y = y$, the informed investor uses the portfolio that is log-optimal for $f(\mathbf x\mid y)$, while the uninformed one uses the portfolio for $f(\mathbf x)$. By the previous theorem, $\Delta W_{Y=y}\le D\big(f(\mathbf x\mid y)\Vert f(\mathbf x)\big)$. Average over $y$:

$$\Delta W\le\iint f(y)f(\mathbf x\mid y)\log\frac{f(\mathbf x\mid y)}{f(\mathbf x)} = \iint f(\mathbf x,y)\log\frac{f(\mathbf x,y)}{f(\mathbf x)f(y)} = I(\mathbf X;Y)$$

∎

By stationarity, $W^*(\mathbf X_n\mid\mathbf X^{n-1})$ is nondecreasing in $n$: more past can’t hurt, and shifting time changes nothing. So it has a limit, possibly $+\infty$. The block average is the running mean of these terms, so by the Cesàro mean theorem it has the same limit. This mirrors the entropy-rate proof with every inequality reversed.

∎

$$\mathbb E\left[\frac{S_{n+1}}{S_{n+1}^*}\Big|\mathbf X^n\right] = \frac{S_n}{S_n^*}\,\mathbb E\left[\frac{\mathbf b_{n+1}^t\mathbf X_{n+1}}{\mathbf b_{n+1}^{*t}\mathbf X_{n+1}}\Big|\mathbf X^n\right]\le\frac{S_n}{S_n^*}$$

using the Kuhn–Tucker condition for the conditional distribution. The martingale convergence theorem gives the limit $V$ with $\mathbb EV\le S_0/S_0^* = 1$. Kolmogorov’s inequality for positive supermartingales gives the bound on the supremum.

∎

Let $W = \frac{VS}{S^*}\ge0$. By independence and one-day optimality, $\mathbb EW = \mathbb EV\cdot\mathbb E\frac S{S^*}\le1$. With $F$ the CDF of $W$ and $U^*$ uniform on $[0,2]$:

$$\Pr(W\ge U^*) = \int_0^2\Pr(W > w)\,\frac12\,dw\le\frac12\int_0^\infty\big(1 - F(w)\big)\,dw = \frac12\mathbb EW\le\frac12$$

using the tail-integral formula $\mathbb EW = \int_0^\infty(1-F)$.

∎

Let $I$ minimize $p_i/q_i$; the edge cases $p_I = 0$ or $q_I = 0$ are trivial. Since $\frac{p_i}{q_i}\ge\frac{p_I}{q_I}$ implies $\frac{p_i}{p_I}\ge\frac{q_i}{q_I}$,

$$\frac{\sum p_i}{\sum q_i} = \frac{p_I\big(1 + \sum_{i\ne I}p_i/p_I\big)}{q_I\big(1 + \sum_{i\ne I}q_i/q_I\big)}\ge\frac{p_I}{q_I}$$

∎

Step 1: A product of sums is a sum of products. For any strategy,

$$S_n(\mathbf x^n) = \prod_{i=1}^n(b_{i1}x_{i1} + b_{i2}x_{i2}) = \sum_{j^n\in\{1,2\}^n}w(j^n)\,x(j^n), \qquad w(j^n) = \prod_ib_{ij_i},\quad x(j^n) = \prod_ix_{ij_i}$$

So an $n$-day market is a one-day market with $2^n$ “stocks” (sequences $j^n$). The strategy puts weight $w(j^n)$ on sequence $j^n$, and sequence $j^n$ returns $x(j^n)$. The $w(j^n)$ sum to 1.

Step 2: Reduce to weights. By the ratio-of-sums lemma,

$$\frac{\hat S_n}{S_n^*} = \frac{\sum_{j^n}\hat w(j^n)x(j^n)}{\sum_{j^n}w^*(j^n)x(j^n)}\ge\min_{j^n}\frac{\hat w(j^n)}{w^*(j^n)}$$

Step 3: The hindsight portfolio’s weights. A constant rebalanced portfolio $\mathbf b = (b, 1-b)$ puts $b^k(1-b)^{n-k}$ on any $j^n$ with $k$ ones. Maximized over $b$ (at $b = k/n$),

$$w^*(j^n) = \Big(\frac kn\Big)^k\Big(\frac{n-k}n\Big)^{n-k} = 2^{-nH(k/n)}$$

These sum to more than 1, because the hindsight investor picks a different $b$ for each sequence. A causal investor can’t.

Step 4: Achievability. Choose $\hat w(j^n) = V_n\,w^*(j^n)$. This is a valid distribution over sequences: $V_n$ is exactly the normalizer, since $\sum_{j^n}w^*(j^n) = \sum_k\binom nk2^{-nH(k/n)} = 1/V_n$. Then $\hat w/w^*\equiv V_n$, so $\hat S_n/S_n^*\ge V_n$ for every market sequence.

Step 5: Nobody does better: extremal markets. For each $j^n$, let $\mathbf x^n(j^n)$ be the market where on day $i$ only stock $j_i$ pays ($\mathbf x_i = \mathbf e_{j_i}$). On such a market, any strategy’s wealth is exactly $w(j^n)$, so $\sum_{j^n}S_n(\mathbf x^n(j^n)) = 1$. The hindsight optimum is $S_n^*(\mathbf x^n(j^n)) = 2^{-nH(n_1/n)} = \hat w(j^n)/V_n$, and these sum to $1/V_n$. A minimum is at most a weighted average, so

$$\min_{\mathbf x^n\in K}\frac{S_n(\mathbf x^n)}{S_n^*(\mathbf x^n)}\le\frac{\sum_KS_n}{\sum_KS_n^*} = \frac1{1/V_n} = V_n$$

The proof extends directly to $m > 2$.

∎

$$\hat{\mathbf b}_{i+1}^t\mathbf x_{i+1} = \frac{\int\mathbf b^t\mathbf x_{i+1}S_i(\mathbf b,\mathbf x^i)\,d\mu}{\int S_i(\mathbf b,\mathbf x^i)\,d\mu} = \frac{\int S_{i+1}(\mathbf b,\mathbf x^{i+1})\,d\mu}{\int S_i(\mathbf b,\mathbf x^i)\,d\mu}$$

so the product over days collapses to $\int S_n\,d\mu$ (with $S_0 = 1$).

∎

Step 1: Hindsight weight. For $j^n$ with $l$ ones, $\prod_ib_{j_i}^* = 2^{-nH(l/n)}$.

Step 2: Mixture weight.

$$\int_0^1b^l(1-b)^{n-l}\frac{db}{\pi\sqrt{b(1-b)}} = \frac1\pi B\Big(l + \frac12,\ n - l + \frac12\Big), \qquad B\Big(l + \frac12, n - l + \frac12\Big) = \frac{\pi}{2^{2n}}\cdot\frac{\binom{2n}{n}\binom nl}{\binom{2n}{2l}}$$

using $B(\lambda_1,\lambda_2) = \frac{\Gamma(\lambda_1)\Gamma(\lambda_2)}{\Gamma(\lambda_1+\lambda_2)}$ and $\Gamma(n+\frac12) = \frac{1\cdot3\cdots(2n-1)}{2^n}\sqrt\pi$. Check: $n = 1$, $l = 0$ gives $B(\frac12,\frac32) = \frac\pi2$ both ways.

Step 3: Combine. By the lemma, $\hat S_n/S_n^*\ge\min_l\frac{B(l+\frac12, n-l+\frac12)/\pi}{2^{-nH(l/n)}}\ge\frac1{2\sqrt{n+1}}$, the last step being a result of Cover and Ordentlich.

∎

$\log p(X_n\mid X_{n-k}^{n-1})$ and $\log p(X_n\mid X_{-\infty}^{n-1})$ are fixed functions of the shifted process, so they are ergodic. Birkhoff gives the limits. The initial term $-\frac1n\log p(X_0^{k-1})\rightarrow0$.

∎

Step 1: Means. Summing over the support $A$ of $p$, $\mathbb E\frac{p^k(X_0^{n-1})}{p(X_0^{n-1})} = \sum_Ap^k = p^k(A)\le1$. Conditioning on the infinite past and summing over the conditional support likewise gives $\mathbb E\frac{p(X_0^{n-1})}{p(X_0^{n-1}\mid X_{-\infty}^{-1})}\le1$.

Step 2: Markov with $t_n = n^2$. $\Pr\big(\frac1n\log\text{ratio}\ge\frac1n\log n^2\big)\le\frac1{n^2}$, which is summable. By Borel–Cantelli the event occurs only finitely often, and $\frac{2\log n}n\rightarrow0$.

This is the same Markov-plus-Borel–Cantelli argument as log-optimal asymptotic optimality: a ratio with mean at most 1 can’t grow exponentially.

∎

From the first sandwich inequality and the Markov-approximation lemma, $\limsup_n-\frac1n\log p(X_0^{n-1})\le\lim_n-\frac1n\log p^k(X_0^{n-1}) = H^k$ for every $k$. From the second, $\liminf_n-\frac1n\log p(X_0^{n-1})\ge H^\infty$. So