Message Authentication Codes and Hash Functions
Integrity as a separate goal from secrecy. Why encryption doesn't authenticate (bit-flipping in stream ciphers and CTR/OFB, IV flips in CBC, block reordering in ECB). MAC syntax and existential unforgeability under adaptive chosen-message attack; why tags must be super-logarithmic; replay attacks as the application's job. PRF ⇒ fixed-length MAC (2⁻ⁿ proof); three broken attempts at variable length; the identifier/index/length construction and its case analysis; basic CBC-MAC (fixed length only), why random IVs and outputting all blocks break it, and three provably secure variable-length fixes. Collision-resistant hash functions: keyed families, the collision experiment, second-preimage and preimage resistance and their hierarchy, the birthday attack (2^{ℓ/2}), constant-memory and meaningful-message variants, the Merkle–Damgård transform and its proof, unkeyed hashes in practice (MD5, SHA-1 attacks, SHA-2). NMAC and HMAC as NMAC with derived keys, and why H(k‖m) fails (length extension). Encrypt-then-MAC with unique tags gives CCA security; the three ways to combine encryption and authentication (encrypt-and-MAC and MAC-then-encrypt fail generically, encrypt-then-MAC is secure); independent keys. Exercises worked.
Corresponds to Katz & Lindell, Introduction to Modern Cryptography, Chapter 4.
Motivation: was this message really sent, unchanged?
IntuitionIntegrity is not secrecy
A supermarket emails a supplier an order for 10,000 crates of soda. Nothing is secret, but the supplier must know (1) the order is authentic (email addresses are trivial to spoof) and (2) it arrived unmodified. Unprotected orders, banking requests, email and SMS can’t be trusted at all, yet caller IDs and return addresses are routinely treated as proof of identity. Modification can’t be prevented on an open channel; the goal is to detect it so that tampered messages can be discarded. Privacy and authentication are distinct goals that are often confused; the chapter ends by combining them correctly.
Encryption does not authenticate
WarningCiphertexts are malleable
- Stream ciphers, OFB, CTR: , so flipping a ciphertext bit flips the same plaintext bit. Fixed-format headers make it easy to flip a buy/sell flag or alter an amount, even without knowing its value.
- CBC: flipping a bit of the flips the same bit of the first plaintext block, often the header.
- ECB: blocks can be reordered, with nothing scrambled.
- Any block cipher: relying on the receiver to notice a “scrambled” block is application-dependent and unreliable.
An extra mechanism is required: a message authentication code.
Message authentication codes
DefinitionMAC syntax (Def. 4.1)
A tuple of ppt algorithms :
- , uniform in ;
- , the tag, for ;
- ;
- correctness: (relaxable to ).
Fixed-length with parameter : Mac is defined only on , and Vrfy rejects other lengths. MACs are private-key primitives: the parties share .
DefinitionForgery experiment (Def. 4.2)
- .
- queries a MAC oracle adaptively (query set ) and outputs .
- Success iff and .
is existentially unforgeable under an adaptive chosen-message attack (“secure”) if every ppt succeeds with negligible probability.
| Component | Choice | Why |
|---|---|---|
| Power | adaptive tag queries on chosen messages | adversaries may influence what gets authenticated (an assistant drafting letters the boss signs) |
| Break | a valid tag on any new message | no assumption about meaningful formats: secure for every application |
NoteTags must be super-logarithmic
Guessing a random tag of length succeeds with probability at least . If , that is , which is not negligible. Secure MACs need tags of bits.
WarningReplay attacks are outside the MAC’s job
Alice’s authenticated “pay Bob $1,000” can be resent ten times by Bob without forging anything. Whether a replay is legitimate depends on the application and its history. Defences: sequence numbers (MAC’d with the message; reuse is rejected) or timestamps (with duplicate rejection). A replay with a changed counter requires forging the MAC.
Fixed-length MAC from a PRF
AlgorithmConstruction 4.3
; for ; iff .
TheoremTheorem 4.4 — A PRF is a secure fixed-length MAC
If is a PRF, Construction 4.3 is existentially unforgeable for -bit messages.
ProofRandom function first, then a distinguisher
Ideal world: with a truly random , for any the value is uniform from the adversary’s view, so success is at most .
Distinguisher : answer ‘s queries with . When outputs , check and , and output 1 iff both hold. Then and , so is negligible, hence so is .
∎
Extending to variable-length messages
WarningThree tempting constructions that fail
Split into blocks :
- MAC only the first block: change the other blocks freely.
- MAC the XOR of all blocks: modify two blocks while keeping their XOR.
- MAC each block separately (ECB-like): blocks can be dropped, repeated, reordered, and spliced from different messages.
The fix must bind each block to its position, its message, and the total length.
AlgorithmConstruction 4.5 — Variable-length MAC
Split (length below , padded with ) into blocks of bits. Choose a random identifier . For each :
(each field bits). The tag is . Vrfy recomputes using the tag’s .
What each field blocks: stops reordering, stops truncation and extension, stops splicing across messages.
TheoremTheorem 4.6 — Construction 4.5 is secure
If is a PRF, Construction 4.5 is existentially unforgeable.
ProofCase analysis in the random-function world
With a truly random , take a forgery with :
- is new: was never evaluated at any , so each is a fresh guess, succeeding with probability at most .
- appeared in exactly one oracle tag, for with blocks:
- : some block has , so is fresh: .
- : was never applied to any : .
- appeared in two or more oracle tags: honest identifiers collide with probability at most over queries, which is negligible (birthday).
Then the distinguisher step is as in Theorem 4.4.
∎
NoteCost and variations (Exercise 4.3)
A message of bits needs PRF calls and a -bit tag, which is impractical. Any fixed-length MAC can replace the PRF. Replacing with a last-block bit removes the need to know the length in advance (useful for streaming). Outputting shrinks the tag to bits (the harder proof).
ExampleA broken two-block MAC (Exercise 4.1)
for -bit halves is insecure: there is no link between the halves. Query to get and to get , then output the forgery . (Worked here.)
CBC-MAC
AlgorithmBasic CBC-MAC (Construction 4.7)
For with -bit blocks: , , tag . That is block-cipher calls and an -bit tag.
TheoremTheorem 4.8 — Secure for one fixed length
For any fixed , if is a PRF then basic CBC-MAC is a secure MAC for -bit messages (the proof is involved and omitted). The length can be any fixed value, not tied to the PRF’s input length.
WarningCBC-MAC is not CBC encryption
| Feature | CBC encryption | CBC-MAC |
|---|---|---|
| IV | random, essential | fixed (), essential |
| Output | all blocks | only the last block, essential |
Harmless-looking changes break it, and libraries often expose one “CBC function” for both uses. Implement exactly the proven construction.
ExampleThe attacks (Exercises 4.4–4.5)
- Random IV (tag ): for one block, , so is a valid tag on . Any change to the first block can be absorbed by the IV.
- All blocks output (two blocks): query to get and to get . Then has tag .
- Variable lengths with basic CBC-MAC: if for one block , then the two-block message has tag . This gives little control over the forged content, but one known attack signals that others may exist.
(Worked here.)
AlgorithmProvably secure variable-length CBC-MAC
- Length-derived key: , then basic CBC-MAC under (send too).
- Prepend the length: the first block is . Appending the length is insecure.
- Encrypt the last block (ECBC): . No need to know the length in advance. Derive and to store one key.
(The standardised descendant is CMAC, which uses derived subkeys to handle the final block.)
Collision-resistant hash functions
IntuitionFrom data structures to cryptography
Hash tables want few collisions among non-adversarial data. Cryptography demands that no efficient adversary, actively searching, can find any collision. Compression guarantees collisions exist (pigeonhole), so they must merely be hard to find. Without compression it is trivial: the identity function is collision-resistant.
DefinitionHash function (Def. 4.9)
: is a public key (it just selects a function from a family; it is not secret), and for . Fixed-length with parameter : defined only on , so it compresses.
DefinitionCollision resistance (Def. 4.10)
Experiment : , , success iff and . Collision-resistant if every ppt succeeds with negligible probability.
TheoremHierarchy of hash security (Exercise 4.6)
| Notion | Given | Find |
|---|---|---|
| Collision resistance | any with | |
| Second-preimage resistance | and random | with |
| Preimage resistance (one-wayness) | and | any with |
Collision resistance ⇒ second-preimage resistance (a second preimage is a collision) ⇒ preimage resistance (invert for random ). The returned differs from with good probability because compresses: most outputs have many preimages.
The birthday attack
AlgorithmGeneric collision finding
Hash distinct random inputs and look for a repeated output. Modelling as random, with -bit outputs,
(23 people make a shared birthday more likely than not). Being far from random only helps the attacker.
ImportantOutput length must be twice the security level
Resisting time- collision attacks needs output length . Asymptotically and are the same (both polynomial iff ), but practically, 128-bit outputs give only about collision security, which is feasible. No generic attack finds second preimages or preimages faster than about .
AlgorithmBirthday attacks with constant memory, and with meaningful messages
- Constant memory (cycle finding): iterate and until . Then and collide (usually distinct), after about steps with no table. Time is far more feasible than storage.
- Meaningful collisions: a sentence with independent synonym choices has many variants ( for the book’s example). With 64 binary word choices a letter has versions. Generate variants of a dismissal letter and of a recommendation letter, then find a cross-collision. This needs memory, so the cycle trick doesn’t apply.
The Merkle–Damgård transform
AlgorithmConstruction 4.11 — Merkle–Damgård
Given a compression function , for with :
- zero-pad to blocks ;
- (any fixed constant, e.g. ) and ;
- output , the length block.
It reduces “hash arbitrary lengths” to “compress by even one bit” (Exercise 4.7 generalises the block sizes). It underlies MD5, SHA-1 and SHA-2.
TheoremTheorem 4.12 — Merkle–Damgård preserves collision resistance
If is collision-resistant, so is .
ProofAny collision in exposes a collision in
Let with .
- : the final calls and have equal outputs on different inputs.
- : let be the largest with . It exists because some . Its outputs are equal: either (then from the final call with equal ), or by maximality . Either way it is a collision in .
The formal reduction runs the -collision finder and applies this extraction (Exercise 4.10).
∎
ExampleVariations on Merkle–Damgård (Exercise 4.11)
- (a) Drop the length block: insecure with zero padding. and (when isn’t a multiple of ) pad to the same blocks.
- (b) Output instead of hashing it: secure. Equal outputs force and , and the case-2 argument applies.
- (c) Random included in the output: secure. Equal outputs force equal s, reducing to the standard proof.
- (d) No IV, : secure. In the maximal-index argument, if no index differs then forces , so . Different lengths are caught by the length block.
(Worked here.)
ExampleComposing hashes (Exercises 4.8–4.9)
- is collision-resistant: a collision is either a collision of the inner , or of the outer on two distinct inner outputs.
- Concatenation is collision-resistant if either component is (a collision must collide in both). That is useful for hedging between candidate designs. It does not hold for (second-)preimage resistance: a contrived component that leaks its input breaks the combination.
Hash functions in practice
WarningUnkeyed hashes don’t fit the asymptotic definition
Real hashes have no Gen and fixed output length.
- For a fixed compressing , a collision exists, so the constant algorithm “print ” succeeds with probability 1. No analogue exists for a randomly keyed family.
- Fixed output length means no security parameter: a collision is always findable in constant time .
The meaningful claim is concrete: no known feasible algorithm finds collisions.
NoteReconciling theory and practice
- The fixed and constants can be viewed as a key chosen once and used by everyone for years, which is acceptable even in theory.
- Reductions are constructive: “breaking the scheme yields an actual collision” is a meaningful guarantee if nobody can exhibit one.
RemarkMD5, SHA-1 and successors
MD5 (128-bit output, birthday about ) and SHA-1 (160-bit, about ) iterate compression functions in Merkle–Damgård style. Wang et al. (2004–5) produced practical MD5 collisions (now found in minutes, including colliding PostScript files with arbitrary chosen content) and a SHA-1 attack, below the birthday bound. Never use MD5 where collision or even second-preimage resistance matters. This drove adoption of SHA-2 (256- and 512-bit outputs) and a new standard. (Since then, the first explicit SHA-1 collision was published in 2017, “SHAttered”, and SHA-3, a non–Merkle–Damgård sponge, was standardised in 2015.)
NMAC and HMAC
DefinitionSecretly keyed hashing via the IV
For a Merkle–Damgård hash with compression function (output bits):
- , keying the compression function’s chaining input;
- computed with secret, so the first step is .
AlgorithmNMAC (Construction 4.13)
Keys and :
The inner keyed hash compresses to bits; the outer keyed compression function acts as a fixed-length MAC on that digest.
TheoremTheorem 4.14 — Security of NMAC
If the secretly keyed (IV = ) is collision-resistant and the secretly keyed (prefix ) is a secure fixed-length MAC, then NMAC is a secure arbitrary-length MAC. This is hash-and-MAC; the same proof appears for signatures in Chapter 12.
ProofSketch: a forgery either collides or forges the outer MAC
Let be the forged message and the queries.
- Some has : a collision in the keyed inner hash, contradicting collision resistance.
- Otherwise is a new input to . The adversary has forged the fixed-length MAC on messages of the form .
∎
NoteWhy key the inner hash?
If is truly collision-resistant, is unnecessary. Keying it is a hedge: collisions in needn’t give collisions in ; the attacker can’t compute the keyed hash offline and must get MAC values from honest parties; and it sees only the outer output. The hedge is weaker than it looks: modern attacks work for any IV, and any known prefix’s chaining value acts as an IV. The proof doesn’t exploit the outer hiding.
AlgorithmHMAC (Construction 4.15)
NMAC needs a custom IV, which libraries don’t expose. HMAC uses the standard IV and a single key, padded to the block length:
with fixed constants: ipad = 0x36 repeated, opad = 0x5C repeated (RFC 2104).
TheoremTheorem 4.16 — HMAC is NMAC with derived keys
The first compression call of each hash derives the keys: and . The outer hash of a single digest block is one keyed compression call, so
If additionally is a PRG (length-doubling), HMAC is a secure MAC.
WarningWhy not just ? (Exercise 4.13)
With Merkle–Damgård, the tag is the chaining value after an extra “block” . So from the tag on alone, anyone can continue the iteration over appended blocks and output the tag on without the key: length extension. This is why HMAC’s outer keyed hash exists. HMAC arrived at the right time (practitioners called CBC-MAC “too slow” and were using such broken heuristics) and became an industry standard: efficient, simple, and provably secure under assumptions believed true of practical hashes.
CCA-secure encryption from encrypt-then-MAC
AlgorithmConstruction 4.17
Given CPA-secure and a secure MAC, with independent keys :
- : , , output .
- : if output , else .
Idea: the adversary can’t create new valid ciphertexts, so the decryption oracle becomes useless.
DefinitionUnique tags
Each has exactly one valid tag. Construction 4.3, CBC-MAC and HMAC have unique tags; Construction 4.5 (random ) does not.
TheoremTheorem 4.18 — CPA + MAC with unique tags ⇒ CCA
If is CPA-secure and is a secure MAC with unique tags, Construction 4.17 is CCA-secure.
ProofTwo reductions: decryption queries are useless, then CPA
Step 1: New valid decryption queries are negligible. Let be the event that submits a decryption query that wasn’t output by the encryption oracle yet doesn’t decrypt to . Build a forger : pick itself, guess , and simulate encryption using its MAC oracle. Answer decryption queries from its table of past encryptions (else ). Output the -th decryption query as the forgery. Unique tags guarantee was never MAC’d (otherwise would have come from the encryption oracle), so a valid new pair is a real forgery. The simulation is perfect until the first valid query, giving success at least . Hence is negligible.
Step 2: Reduce to CPA. picks , forwards encryption queries to its CPA oracle, attaches tags, answers decryption queries from its table (else ), and relays the challenge. On the views are identical. If had advantage , then , a contradiction.
∎
NoteUnique tags and “useless” oracles
Without unique tags, change the challenge’s tag to another valid one, query its decryption, and get . Exercise 4.14 shows Construction 4.5 still works under a strong unforgeability requirement: even a new tag on an already-queried message is a forgery. Making the decryption oracle useless is the point: real adversaries can at best decrypt ciphertexts the honest parties already sent.
Combining privacy and authentication
DefinitionSecure combination (Def. 4.19)
With independent keys , a combination with and is secure if it is both CCA-secure as encryption and existentially unforgeable as a MAC. The standard is all-or-nothing: it must be secure for every CPA-secure scheme and every secure MAC, because libraries and standards swap components (3DES → AES).
WarningEncrypt and authenticate:
Insecure. A secure MAC may leak : is still unforgeable. Even with real MACs (Exercise 4.15): CBC-MAC and HMAC are deterministic, so equal messages get equal tags. That breaks multi-message eavesdropping security outright (did the client send the same command twice?).
WarningAuthenticate then encrypt:
Not generically secure. Counterexample: encode each bit, and or (arbitrarily), with decoding to , then encrypt with a fresh-pad stream cipher (CTR with random IV, still CPA-secure). To learn the first bit, flip the first two ciphertext bits and ask only whether the result is valid:
- bit 1: , still decodes to 1, valid;
- bit 0: , decodes to , invalid.
Repeat for every bit to decrypt fully, using only a validity oracle, which is very realistic. Specific instantiations can be secure (SSL’s, per Krawczyk), but relying on that is bad practice. (Real-world descendants are padding-oracle attacks: Vaudenay 2002, Lucky 13 in 2013.)
TheoremTheorem 4.20 — Encrypt then authenticate is secure
with and , for any CPA-secure scheme and any secure MAC, is a secure combination. CCA security is Theorem 4.18; unforgeability follows by reduction to the MAC as in its Step 1 (Exercise 4.16). It upgrades CPA to CCA.
NoteSecure combination versus CCA security
A secure combination gives CCA security plus authenticity. CCA security alone needn’t give authenticity: they are different goals that happen to share a construction.
WarningUse independent keys for different goals
With (CPA-, even CCA-secure) and (a secure MAC since is also a PRP), sharing one key gives : the plaintext in the clear. Reusing keys across goals needs its own explicit proof.
RemarkEncrypt or also authenticate?
Default to both. Even disk encryption benefits: undetected edits to financial files can cause real harm, and missing integrity can even leak secrecy (the authenticate-then-encrypt attack above).
Remaining exercises
ExampleMACs with a verification oracle (Exercise 4.2)
Add a oracle to the forgery experiment: real attackers learn whether forged packets are accepted.
- Deterministic MACs (unique tags): equivalent to Definition 4.2. Before any successful forgery, every Vrfy query on a new message answers 0 (except negligibly), and queries on old messages are answerable from the transcript. A reduction guesses the index of the first accepting query, losing a factor .
- Probabilistic MACs: not equivalent. A scheme can have many valid tags whose acceptance pattern leaks key bits, for example verification that accepts slightly malformed tags depending on a key bit, so Vrfy queries reconstruct the key.
(Sketch worked here.)
Quick reference
| Object | Statement | Note |
|---|---|---|
| Malleability | stream/CTR/OFB bit flips; CBC IV flips; ECB reordering | encryption ≠ integrity |
| MAC security | no valid tag on a new message after adaptive queries | tags bits |
| Replay | the application’s job | sequence numbers, timestamps |
| PRF MAC | forgery | |
| Variable-length | binds position, length, message | |
| CBC-MAC | , output | fixed length only; IV 0; last block only |
| Fixes | length-keyed; prepend length; encrypt last block | appending length is insecure |
| Collision resistance | ppt can’t find with | ⇒ second preimage ⇒ preimage |
| Birthday | about evaluations | output ≥ 2 × security bits |
| Merkle–Damgård | , output | collision in ⇒ collision in |
| Practice | MD5 broken; SHA-1 broken (2017 collision); SHA-2, SHA-3 | unkeyed: concrete security |
| NMAC | keyed CR inner + MAC outer | |
| HMAC | NMAC with PRG-derived keys | |
| insecure | length extension | |
| Encrypt-then-MAC | CPA + MAC (unique tags) ⇒ CCA + authenticity | the safe default |
| Encrypt-and-MAC, MAC-then-encrypt | not generically secure | tag leaks; validity oracle |
| Keys | independent per goal | shared key can leak |
Facts to keep to hand
- Authentication is its own goal. Encryption leaves ciphertexts malleable. A MAC must resist forgery on any new message under adaptive chosen-message attack. Replays are the application’s problem.
- PRFs make MACs; details make them variable-length. is a fixed-length MAC. Variable length needs binding of order, length and message (Construction 4.5) or CBC-MAC with a proven length fix. Small deviations (random IV, all blocks output, appended length) break it.
- Collision resistance is limited by the birthday bound and built by Merkle–Damgård. Generic attacks cost about ; MD reduces arbitrary lengths to a compression function; practical hashes are unkeyed and judged concretely, and MD5 and SHA-1 have fallen.
- Hash-based MACs need nesting; combinations need encrypt-then-MAC. HMAC (NMAC with derived keys) defeats length extension. Encrypt-then-MAC with independent keys turns CPA security into CCA security plus authenticity. The other two orders fail generically.
References
- Katz & Lindell, Introduction to Modern Cryptography — Chapter 4
- Bellare, Kilian & Rogaway, “The Security of the Cipher Block Chaining MAC” (2000)
- Bellare, Canetti & Krawczyk, “Keying Hash Functions for Message Authentication” (1996) — NMAC/HMAC
- Bellare & Namprempre, “Authenticated Encryption: Relations among Notions” (2000); Krawczyk, “The Order of Encryption and Authentication” (2001)