Computational Aspects Of Prediction Markets
The information aggregation problem; partition model of knowledge, common prior, omniscient forecast; market mechanisms (call market, continuous double auction, market maker, parimutuel) and empirical record (IEM, marginal traders, HP); rational expectations equilibrium, fully revealing equilibria, efficient market hypothesis and no-trade theorems; combinatorial markets over 2^|E| states, complete markets, combined vs compound orders, conditional securities S_(phi|psi), order payoff vectors and the matching problem; divisible matching polynomial (LP) for O(log n) events and co-NP-complete in general (Theorem 26.6), indivisible NP- and Sigma_2^p-complete (Theorem 26.7); Bayesian-network-structured markets and operational completeness; automated market makers, the thin-market problem, market scoring rules and the logarithmic cost function with loss b ln|Omega|, quadratic cost function, sequential shared scoring rules; dynamic parimutuel markets with the share-ratio cost function; the Boolean market model, price formation, myopic truthful Bayesian traders, common knowledge of aggregates (Theorem 26.9), XOR non-convergence, weighted threshold functions as exactly the computable aggregates (Theorems 26.12–26.13), and convergence within n rounds (Theorems 26.14–26.15)
Motivation: aggregating scattered knowledge
DefinitionThe information aggregation problem
An aggregator wants a forecast of an uncertain variable, say the global average temperature in 2020. Many informants hold different, nonindependent pieces of relevant information. The goal is a mechanism that elicits and combines this information into a collective forecast, ideally equal to the omniscient forecast: what someone with direct access to everyone’s information would predict.
DefinitionPrediction market
The aggregator creates a security whose payoff depends on the outcome, e.g. $ if the 2020 temperature equals , and lets informants trade it freely. Believers in global warming buy at or above today’s temperature; skeptics sell. The trading price is read as the collective forecast of the expected value.
Also called information markets, Arrow–Debreu securities, contingent claims, event futures, idea futures. (Time value of money ignored.)
IntuitionThe central question
A market can in principle turn private beliefs into a price that knows what everyone knows. But operating it may be computationally intractable, subsidizing it must be done with bounded loss, and even a well-run market may fail to compute the answer. Where exactly are these computational limits?
The chapter locates computation in three places:
| Where | Problem | Section |
|---|---|---|
| the auctioneer | matching orders over exponentially many combined events | Combinatorial prediction markets |
| the market maker | updating prices after every trade with bounded loss | Automated market makers |
| the market as a whole | does the price converge to the right answer, and how fast? | The market as a distributed computer |
Background
The partition model of knowledge
DefinitionDefinition 26.1 — partition model
- is the set of states of the world. Exactly one state is true.
- Agent ‘s information is a partition of . She knows which cell contains the true state but can’t distinguish states within a cell.
- The combined information is the coarsest common refinement of .
Usually a common prior is added. Agents form posteriors by conditioning on their cell.
ExampleFigure 26.1 — three base events
Events generate 8 states. For example, means true, false, true. Agent knows only whether holds, so . Together the three agents pin down the state exactly: is the finest partition, into singletons.
Forecasts.
- A forecast is an estimate of for a commonly known function .
- An event is an .
- A joint forecast is a joint distribution over several functions.
- The omniscient forecast conditions on the true cell of .
ImportantAggregation is mechanism design
Each agent’s information is private. The task is a mechanism whose equilibrium output is as close as possible to the omniscient forecast (see mechanism-design). A market pays agents for moving the price toward the truth.
Mechanisms and evidence
| Mechanism | How trades happen |
|---|---|
| call market | batch bids; clear at the -th or -th lowest price, or between ( = number of sellers) |
| continuous double auction | trade immediately whenever two orders cross, usually at the earlier bidder’s price |
| market maker / bookmaker | always willing to buy and sell at posted, changing prices |
| parimutuel | winners split the total pool in proportion to their stakes |
Variables can be binary, discrete, continuous or joint, and combinatorial versions of each mechanism exist.
ExampleEmpirical record
- Iowa Electronic Market (since 1988, real money, mostly elections): on average more accurate and less volatile than polls. It reacts within minutes, often before news spreads, even though individual traders are biased.
- Marginal-trader hypothesis: accuracy comes from a few active, less biased, price-making traders, not from the average trader.
- Options, futures and sports betting odds are accurate forecasts of their underlying variables.
- Play-money markets (Hollywood Stock Exchange, NewsFutures, Foresight Exchange) match experts and sometimes real-money markets.
- Lab experiments: with enough securities and practice, prices approach omniscient forecasts.
- Field: HP’s internal sales market beat official forecasts. Microsoft and Google run internal markets.
NoteEvaluation is comparative
A market can’t surface information nobody has, so accuracy is judged only against other aggregation methods.
Rational expectations
WarningSecurities have no consumption value
People buy a security only because they expect to resell or cash it out higher. Prices carry information about others’ beliefs, which changes each trader’s demand. So the classical fixed-demand-curve model fails.
DefinitionDefinition 26.2 — rational expectations equilibrium (REE)
Traders are risk-neutral Bayesians with a common prior and private information . Take to be the finest partition. An REE is a mapping such that, in every state, if each trader conditions demand on and on the price , the market clears at exactly . It is a self-fulfilling state-to-price correspondence.
IntuitionUnpacking the fixed point
Any nonconstant price map induces a partition : seeing the price rules out states that would have produced a different price. Each trader’s effective valuation becomes , which yields a clearing price . is an REE iff : believing in the map and acting on it reproduces the map.
TheoremFully revealing equilibria
REE exist under fairly general conditions. Generically, there are fully revealing REE, with whenever , so that : the price reveals everyone’s combined information.
NoteCriticism
REE says nothing about how traders would learn or agree on . The price-formation process must be modelled separately. The market as a distributed computer shows it can fail.
DefinitionEfficient market hypothesis (strong form)
The price fully reflects all information relevant to the security’s value. This is the theoretical case for prediction markets: many small pieces of private information end up in the price.
WarningNo-trade theorems
In a fully revealing REE, everyone conditions on the same information, so they share posteriors and valuations, and nobody trades. More strongly, rational agents won’t trade even out of equilibrium, because a counterparty’s willingness to trade reveals information. Paradox: the price contains everyone’s information, but nobody trades to put it there.
ImportantWhy trade happens anyway
No-trade results are fragile. They need risk neutrality and common knowledge of full rationality. Real sources of expected profit: irrational traders, hedgers, liquidity traders, and a subsidizing market maker.
Combinatorial prediction markets
Setup. A finite set of Boolean base events (“gas costs more than $3”), linearly independent in the sense that no event is determined by the others. Then .
DefinitionComplete market
Let pay $1 iff the state is . A market is complete if traders can build a portfolio replicating any . Classic results: complete markets can be guaranteed efficient.
CautionCompleteness is exponential
A complete market needs at least securities. With many base events, even listing them is impossible. There may be far fewer combinations traders care about, which raises two questions: is there a succinct representation for orders, and can matches be found in it?
Two forms of expressive bidding:
| Order type | Trader says | Auctioneer problem |
|---|---|---|
| combined | ”this bundle of securities, each within its limit price, or nothing” | like combinatorial auctions (Chapter 11), but bids are often divisible; called combined value trading |
| compound | ”a security on this Boolean formula over ” | expressible events; a strict superset of combined orders |
NoteExecution risk
Combined orders remove execution risk: prices moving midway through a planned series of trades and ruining the earlier ones.
Compound securities and the matching problem
DefinitionConditional security
For Boolean formulas over :
- pays $1 if ;
- pays $0 if ;
- cancelled and refunded if .
The unconditional is .
DefinitionOrders and payoff vectors
An order = ” units of at price .” Here is a buy (accepting any price ) and a sell (any price ). Divisible orders accept any fraction with ; indivisible orders accept exactly or nothing. The payoff vector over states is
If the auctioneer accepts fractions , its surplus in each state is
DefinitionDefinitions 26.3 / 26.5 — the matching problem
Does there exist with some such that for every state ? That is, can the auctioneer accept a nonempty set of orders without risk, breaking even at worst?
- Indivisible: .
- Divisible: .
ExampleExample 26.4 — two orders that match
, with states ordered .
- Buy 1 of at 0.4: .
- Sell 1 of at 0.3: .
The orders match. The auctioneer keeps $0.10 cash plus one unit of .
Complexity of matching
Let be the input size, which also bounds the number of base events.
| base events (poly many states) | unlimited base events (exponentially many states) | |
|---|---|---|
| divisible | P (LP) | co-NP-complete |
| indivisible | NP-complete | -complete |
TheoremTheorem 26.6 — divisible matching
Divisible matching is polynomial-time for base events and co-NP-complete in general.
ProofLP, a small certificate for infeasibility, and a SAT reduction
Step 1: The LP (few events). Variables . One constraint per state: Maximize . A match exists iff the optimum is . With events there are polynomially many states, so the LP is polynomial. (The chapter notes this approach may miss matchings with exactly zero surplus.)
Step 2: In co-NP (many events). Now there are exponentially many constraints, but each is short and easy to evaluate. With orders, LP theory says that if the optimum is 0, some constraints already force it. Guess them and solve the small LP. That is a polynomial certificate that no match exists.
Step 3: co-NP-hard, via SAT. Given a formula , take its variables as base events and a single buy order for at 0.5.
- satisfiable: in a satisfying state the auctioneer pays out, giving surplus , so no fraction matches.
- unsatisfiable: the security never pays, so surplus is in every state, and it matches.
The key step: a match exists iff is unsatisfiable. So matching is as hard as UNSAT.
∎
TheoremTheorem 26.7 — indivisible matching
Indivisible matching is NP-complete for base events and -complete in general.
NoteProof omitted in the chapter
The chapter omits it for space.
IntuitionWhere the two sources of hardness come from
Indivisibility adds a combinatorial choice (which subset of orders), which gives the NP-type “exists” layer. Exponentially many states add a universal check (surplus in every state), which gives the co-NP-type “for all” layer. With both, you get , i.e. .
Compact markets: exploit independence
DefinitionBN-structured market
Suppose base events factor as a Bayesian network: where the parents have smaller indices. Trade only the conditional securities , one per parent configuration. That is securities, far fewer than when parent sets are small.
TheoremOperational completeness
A BN-structured market is operationally complete (its equilibrium is Pareto optimal for the traders present) if all traders’ risk-neutral independencies agree with the network’s. For traders with constant absolute risk aversion (exponential utility), agreement on Markov independencies suffices.
NoteStated without proof
Completeness in all circumstances still needs securities.
Automated market makers
Why not just a continuous double auction?
WarningTwo problems with pure order matching
- Thin markets. An informed trader may wait forever for a counterparty. Expecting none, traders don’t come: a chicken-and-egg breakdown. It is worst in combinatorial markets, where attention is spread over exponentially many securities.
- Revealing intent. Posting a limit order tips off others. Traders play a zero-sum game with each other, so no-trade logic bites.
DefinitionMarket maker
An agent always ready to trade at posted bid and ask prices, which may be equal. It bridges buyers and sellers who arrive at different times.
ImportantSubsidy defeats no-trade
The market maker can lose money, which is a deliberate subsidy. With a subsidy, the no-trade theorems no longer apply, so informed traders can rationally expect to profit. The loss must be bounded, which requires updating prices systematically after every trade.
Desiderata for an automated market maker:
- bounded (predictable) loss;
- informed traders are motivated to trade whenever their information would move the price;
- computing new prices after each trade is tractable.
Market scoring rules (Hanson)
A proper scoring rule pays for a probability report so that truthful reporting is optimal. Hanson turns any such rule into a market maker.
DefinitionCost-function market maker
There are mutually exclusive, exhaustive securities, and is the total quantity of security held by traders. A cost function records total trader spending. Moving outstanding shares from to (any bundle, combined or compound order; negative entries are sells) costs The instantaneous price of is . The winning security pays $1 per share.
| Component | Detail |
|---|---|
| Signature | |
| Domain | outstanding shares of every outcome |
| Codomain | total money collected from traders so far |
| Prices | the gradient , read as the market’s probability forecast |
FormulaLogarithmic market scoring rule (LMSR)
The maximum loss is . Larger means more liquidity (more shares near the current price) but more risk. It is deployed at InklingMarkets, Net Exchange and Microsoft.
DerivationWhy LMSR’s worst-case loss is
Start at , so . The market maker has collected and owes at most (if the most-held outcome wins). Since , (This derivation is mine; the chapter states the bound.)
FormulaQuadratic market scoring rule
WarningQuadratic is impractical
Prices are linear in quantities, so they would leave . The maker can sell only a small fixed number of shares, and once one security hits its cap it can’t accept buys on the others. LMSR accepts arbitrarily large orders.
IntuitionSequential shared scoring rule
Equivalently, the maker starts with an initial probability estimate. Each trader pays the scoring-rule payment for the previous report and receives the payment for her own. Myopically, reporting truthfully is still optimal. The final trader is paid by the maker, so the maker’s loss is at most the maximum payment to the last trader minus the payment from the first.
graph LR M["Market maker: initial estimate"] -->|"trader 1 pays score of maker's report"| T1["Trader 1 reports"] T1 -->|"trader 2 pays score of trader 1's report"| T2["Trader 2 reports"] T2 -->|"..."| TN["Final trader reports"] TN -->|"maker pays final trader's score"| M
Dynamic parimutuel markets
DefinitionDynamic parimutuel market
In a parimutuel game, winners split the total pool in proportion to their shares. The dynamic variant makes share prices vary according to a cost function, so traders can sell before resolution. To the trader it acts like a market maker.
FormulaShare-ratio cost function
It is the unique dynamic parimutuel cost function with (price ratios equal share ratios).
DerivationWith : prices below 1, payoffs above 1
Price: . The pool is , so the payoff per winning share is . Hence a winning bet never loses money.
The mechanism is also arbitrage-free. The maker seeds initial shares with dollars, which is its maximum risk.
| Market scoring rule | Dynamic parimutuel | |
|---|---|---|
| payout per winning share | fixed $1 | a share of the total pool (uncertain) |
| price as forecast | direct probability | less direct |
| trading strategy | simpler | more complex |
| appeal | forecasting | risk-seeking gamblers |
The market as a distributed computer
Turn the viewpoint inside out: treat market plus traders as a computational device. What can it compute, and how fast?
IntuitionWhere the computation happens
In the traders’ belief updates. Each trader observes the price, a signal combining others’ information, and merges it logically with her own private information. This also addresses REE’s gap: how are equilibrium prices reached? Plausible dynamics can get stuck at an uninformative equilibrium even when a fully revealing one exists.
The Boolean market model
DefinitionBoolean market
- traders, where trader privately knows one bit . So and splits by the value of .
- A security pays $1 iff , for a commonly known .
- The traders’ combined information determines , so an efficient market should converge to price .
DefinitionPrice formation
Synchronous rounds. Each trader submits money and quantity (no credit or short-sale limits). The market clears at Trader ends with units and money: a net buy if and a net sell if . The price is announced, beliefs are updated, and the process repeats until nothing changes.
Simplification: for all , so is the average bid.
NoteWhy force
- It forces trade, sidestepping no-trade theorems without modelling hedgers or subsidies explicitly.
- Known volumes help traders learn from prices. This arguably gives them too much power, yet convergence can still fail.
DefinitionAgent behaviour
Agents are risk-neutral, myopic (each round treated as the last), and bid truthfully their current expected value of . They share a common prior , update by Bayes’ rule, and this behaviour is common knowledge.
ExampleExample 26.8 — OR, first round
Two traders, uniform prior, .
- : is equally likely 0 or 1, so trader 1 bids .
- : for sure, so trader 2 bids .
The first-round price is .
Knowledge dynamics and equilibrium
After round :
- the common-knowledge set is (states an outside observer, seeing only prices, considers possible);
- trader ‘s knowledge is .
Start with . The observer knows the map from states to first-round prices, so , and so on. Knowledge only shrinks, so the process reaches an equilibrium in finitely many rounds.
TheoremTheorem 26.9 — agreement at equilibrium
Every trader’s expectation equals the uninformed observer’s, and equals the price.
NoteStated without proof
It follows from known results on common knowledge of aggregates.
WarningAgreement ≠ aggregation
Traders can agree on an expectation strictly between 0 and 1 while their combined information determines exactly. Their knowledge sets may differ as long as the conditional expectations coincide.
ExampleExample 26.10 — XOR gets stuck
Two traders, uniform prior, .
- Trader 1: whatever is, iff , which has probability . So she bids regardless.
- Trader 2 likewise always bids .
The price is , which reveals nothing about either bit. Nobody updates, so equilibrium is reached at . Both traders agree (as Theorem 26.9 requires), yet is never revealed.
IntuitionWhy XOR fails
Each bid is a function of one bit, and the price is their sum. For XOR, each trader’s expected value is the same whatever her bit is, so the sum carries no signal. A function computable from sums of per-bit contributions must be linearly separable.
Computable aggregates are exactly threshold functions
DefinitionDefinition 26.11 — weighted threshold function
is a weighted threshold function iff there are reals with OR is threshold (, ). XOR is not.
TheoremTheorem 26.12 — threshold ⇒ always computed
If is a weighted threshold function, then for every prior the equilibrium price of equals .
TheoremTheorem 26.13 — non-threshold ⇒ sometimes not
If is not a weighted threshold function, some prior makes the price of fail to converge to .
TheoremTheorems 26.14–26.15 — convergence time
- For a threshold with inputs and any prior, the price reaches within at most rounds.
- Some function on inputs, with some prior , needs rounds in the worst case.
NoteProofs omitted
The chapter omits Theorems 26.12–26.15 for space. It notes that simulations suggest near-constant expected convergence time for almost every threshold function.
ImportantWhat a single security can compute
One market price computes precisely the functions a single linear threshold unit (a perceptron) computes. Anything else, such as XOR, needs several securities combined. How many is open.
Open questions
NoteCombinatorial markets
- The bidder’s side: how to choose formulas, quantities and prices, possibly with penalties on bid complexity.
- Less expressive bidding languages with polynomial matching that are still useful.
- Heuristic or approximate matching; logical-reduction algorithms.
- Permutation combinatorics (horse-race orderings: ” wins,” “top 3,” ” beats ”): languages with tractable matching.
- Sharing auctioneer surplus with traders, and the incentives this creates.
- Matching a single new order against a no-match book (the standard continuous double auction rule).
- Computational equilibrium: can a market settle where no bounded agent finds an improvement, yet it isn’t a true equilibrium?
NoteMarket makers
- For every polynomially matchable bidding language, is there a polynomial-time market-scoring-rule maker?
- Market makers are online algorithms (see online-mechanisms): accept or reject orders one at a time under a worst-case loss budget. Can other online algorithms accept more orders for the same bound?
NoteMarkets as computers
- Noisy prices, and misinformed or irrational traders: what is still computable?
- Incremental recomputation when a few input bits flip.
- A distributed-computing model of decentralized markets.
- The complexity of traders’ belief updates.
- Replacing the truthful-bidding assumption with game-theoretic rationality.
- Average-case convergence over the prior.
- The minimum number of threshold securities needed to implement a given function, and security designs that speed up convergence.
Quick reference
| Object | Statement | Note |
|---|---|---|
| Aggregation problem | elicit and combine private info into the omniscient forecast | mechanism design |
| Partition model (Def 26.1) | partitions ; combined info = coarsest common refinement | common prior |
| REE (Def 26.2) | self-fulfilling map | generically fully revealing |
| Efficient market hypothesis | price reflects all relevant information | foundation for prediction markets |
| No-trade theorems | rational agents with common knowledge never trade | broken by hedgers, noise, subsidies |
| Complete market | needs securities | combinatorial blow-up |
| $1 if , $0 if , refunded if | compound / conditional | |
| Order payoff | ||
| Matching | : in every state | risk-free acceptance |
| Ex. 26.4 | buy @0.4 + sell @0.3 ⇒ | match |
| Thm 26.6 | divisible: P (few events), co-NP-complete (many) | LP; SAT reduction |
| Thm 26.7 | indivisible: NP-complete (few), -complete (many) | |
| BN market | securities | operationally complete if independencies agree |
| AMM desiderata | bounded loss; incentive to trade; tractable updates | subsidy defeats no-trade |
| Cost-function maker | pay ; price | $1 per winning share |
| LMSR | , softmax prices | loss |
| Quadratic MSR | prices linear in | only small fixed purchases |
| Sequential scoring | pay predecessor’s score, receive your own | equivalent to the cost-function form |
| Share-ratio DPM | , | pays a share of the pool |
| Boolean market | bits ; security pays ; price = mean bid () | myopic truthful Bayesians |
| Ex. 26.8 (OR) | bids 0.5 and 1 ⇒ price 0.75 | |
| Thm 26.9 | at equilibrium, all expectations = observer’s = | common knowledge of aggregates |
| Ex. 26.10 (XOR) | stuck at 0.5 forever | agreement without aggregation |
| Threshold function | linearly separable | |
| Thms 26.12–26.13 | computable for all priors ⟺ threshold | |
| Thms 26.14–26.15 | rounds; some instances need |
Facts to keep to hand
- Rational expectations promises aggregation; computation decides whether you get it. Fully revealing REE exist generically, but the auctioneer may face co-NP- or -hard matching, and natural price dynamics compute only threshold functions.
- No-trade theorems explain why subsidies matter. In a zero-sum game among rational traders, willingness to trade reveals information and kills the trade. A bounded-loss market maker injects money so that informed trading becomes profitable. LMSR’s is literally the price of information.
- Complexity has two independent layers. Indivisibility adds an “exists a subset” choice, and exponentially many states add a “for every state” check. Divisible with few states is P, and each added layer climbs one step, up to with both.
- Structure is the escape from . Bayesian-network conditional securities give polynomially many instruments, and they are enough whenever traders’ independencies match the network.
- A price is a sum, so it computes linear threshold functions. Each trader contributes a bid that depends on her information, and the price averages them. Functions that are not linearly separable (XOR) can leave every bid uninformative. One security acts like one perceptron.
- Agreement is not aggregation. At equilibrium everyone agrees on the price (common knowledge of aggregates), but agreeing on can coexist with the truth being fully determined by what they jointly know.
- Market makers are cost functions whose gradient is the forecast. LMSR prices are a softmax, and the quadratic rule has linear prices, which breaks the range. The share-ratio parimutuel pays from the pool. The sequential-scoring-rule reading explains why truthful moves are myopically optimal.
References
- Nisan, Roughgarden, Tardos, Vazirani, Algorithmic Game Theory — chapter 26 (Pennock, Sami)