babylonpolice.com

Machine learning (ML) models trained on data from potentially untrusted
sources are vulnerable to poisoning. A small, maliciously crafted subset of the
training inputs can cause the model to learn a "backdoor" task (e.g.,
misclassify inputs with a certain feature) in addition to its main task. Recent
research proposed many hypothetical backdoor attacks whose efficacy heavily
depends on the configuration and training hyperparameters of the target model.

Given the variety of potential backdoor attacks, ML engineers who are not
security experts have no way to measure how vulnerable their current training
pipelines are, nor do they have a practical way to compare training
configurations so as to pick the more resistant ones. Deploying a defense
requires evaluating and choosing from among dozens of research papers and
re-engineering the training pipeline.

In this paper, we aim to provide ML engineers with pragmatic tools to audit
the backdoor resistance of their training pipelines and to compare different
training configurations, to help choose one that best balances accuracy and
security.

First, we propose a universal, attack-agnostic resistance metric based on the
minimum number of training inputs that must be compromised before the model
learns any backdoor.

Second, we design, implement, and evaluate Mithridates a multi-stage approach
that integrates backdoor resistance into the training-configuration search. ML
developers already rely on hyperparameter search to find configurations that
maximize the model's accuracy. Mithridates extends this standard tool to
balance accuracy and resistance without disruptive changes to the training
pipeline. We show that hyperparameters found by Mithridates increase resistance
to multiple types of backdoor attacks by 3-5x with only a slight impact on
accuracy. We also discuss extensions to AutoML and federated learning.