Manuscripts
Universal Jailbreak Backdoors from Poisoned Human Feedback
Javier Rando, Florian Tramèr

Code Twitter

Privacy Side Channels in Machine Learning Systems
Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher A. Choquette-Choo, Matthew Jagielski, Milad Nasr, Eric Wallace, Florian Tramèr

Blogpost

Evaluating Superhuman Models with Consistency Checks
Lukas Fluri, Daniel Paleka, Florian Tramèr

Code Twitter Blogpost

Poisoning Web-Scale Training Datasets is Practical
Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, Florian Tramèr

Press: [1, 2, 3, 4]

Considerations for Differentially Private Learning with Large-Scale Public Pretraining
Florian Tramèr, Gautam Kamath, Nicholas Carlini

Twitter

2023
Are aligned neural networks adversarially aligned?
Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramèr, Ludwig Schmidt
NeurIPS 2023

Blogpost

Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini
INLG 2023

Tight Auditing of Differentially Private Machine Learning
Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramèr, Matthew Jagielski, Nicholas Carlini, Andreas Terzis
USENIX Security 2023 Distinguished paper award

Extracting Training Data from Diffusion Models
Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace
USENIX Security 2023

Twitter Press: [1, 2, 3, 4, 5, 6]

Evading Black-box Classifiers Without Breaking Eggs
Edoardo Debenedetti, Nicholas Carlini, Florian Tramèr
ICML AdvML Frontiers Workshop 2023

Code Twitter

A law of adversarial risk, interpolation, and label noise
Daniel Paleka, Amartya Sanyal
ICLR 2023

A Light Recipe To Train Robust Vision Transformers
Edoardo Debenedetti, Vikash Sehwag, Prateek Mittal
IEEE SaTML 2023

Code Video Twitter

2022
Red-Teaming the Stable Diffusion Safety Filter
Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, Florian Tramèr
NeurIPS ML Safety Workshop 2022 Best paper award

Code Twitter Press

Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets
Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, Nicholas Carlini
ACM CCS 2022

Code Twitter Press