mlsec

MLsec, short for machine learning security, is an interdisciplinary research area that studies the security and privacy of machine learning systems. It investigates threats to models, data, and deployments across the machine learning lifecycle, and develops defenses and evaluation methodologies. Researchers in MLsec examine how learning systems can be manipulated, leaked, or misused, and how to build more robust and privacy-preserving models.

Threat models consider attacker goals and capabilities, including white-box or black-box access to models, access to training data, and control of inputs. Major attack classes include adversarial examples that cause misclassification with small perturbations, data poisoning during training to degrade performance, backdoor triggers that activate on specific inputs, model extraction to steal model parameters, and privacy attacks such as membership inference and model inversion that reveal information about training data.

Defenses span robust training and optimization, detection of adversarial inputs, certified robustness guarantees, data sanitization, and privacy-preserving techniques. Privacy-preserving ML relies on differential privacy, secure computation, homomorphic encryption, and privacy-aware federated learning with secure aggregation. Evaluation methods emphasize realistic threat models, attack success rates, robustness metrics, and checks for dataset bias and overfitting.

MLsec has applications in critical systems such as medical imaging, autonomous vehicles, cybersecurity, finance, and cloud-based ML services, where breaches or errors can have serious consequences. The field also addresses governance, risk assessment, and safe AI development practices.

While sometimes described as a distinct field, MLsec is closely tied to machine learning, cybersecurity, and privacy research. It comprises theoretical work on security properties, as well as practical tools and benchmarks for evaluating attackers and defenses. The literature emphasizes multidisciplinary approaches and reproducibility.