Machine Learning Security and Data Privacy
Stub. Security and privacy problems specific to machine learning: adversarial examples and model evasion, membership-inference and model-inversion attacks that leak training data, and the formal privacy models (differential privacy, secure multiparty computation) used to train or query models without exposing individual records.
Machine learning creates privacy and security problems that don’t reduce to classical ones: a trained model is itself a (lossy, but often exploitable) summary of its training data, so membership-inference attacks (did a specific record appear in training?) and model-inversion attacks (reconstructing something close to an actual training example) are real threats even without any traditional data breach. Adversarial examples are the security-side analogue — small, often imperceptible input perturbations that reliably fool a model — raising a robustness question distinct from privacy. Differential privacy and secure multiparty computation are the two formal frameworks for training or querying a model while provably bounding what any output reveals about individual inputs, which is the rigorous version of the ad hoc anonymization that fails so often in the social-network setting.
Outline (TODO — flesh out each)
- Adversarial examples and model evasion
- Membership-inference attacks
- Model-inversion attacks
- Differential privacy (deep dive: differential-privacy-mechanisms-and-lower-bounds, differentially-private-estimation-and-machine-learning)
- Secure multiparty computation for privacy-preserving ML