Nicolas Papernot

I am chairing a workshop at ICLR 2020 on Trustworthy ML, consider submitting your work. This a workshop co-organized with Carmela Troncoso, Florian Tramer (co-chair), Nicholas Carlini, and Shibani Santurkar.

Positions available

• If you are interested in joining my research group as a graduate student, please fill the following form and apply to the CS or ECE (select "software systems" field) program. Unfortunately, I cannot respond to all prospective graduate students but filling the form ensures that I keep a record of your application.
• If you are interested in joining my research group as a postdoc, please send me an email directly with your CV and research statement.

Upcoming Talks

Here is a list of talks I will be giving. Feel free to reach out if you will be attending one of these events and would like to meet.

• 6/2020 - RAISA3 at the European Conference on AI keynote
• 3/2020 - Carnegie Mellon University lecture

A complete list of talks I previously gave is available in my CV.

Students

• Mingyue Yang (PhD student, started Winter 2020, co-supervised with David Lie)
• Saina Asani (Research Assistant, started Winter 2020)
• Yunxiang Zhang (Research Intern, starting Winter 2020)
• Laura Zhukas (Undergraduate Student Researcher, Fall 2019)
• Christopher Choquette Choo (Engineering Science Thesis, started Fall 2019)
• Nick Jia (Engineering Science Thesis, started Fall 2019)
• Baiwu Zhang (MEng, started Fall 2019)
• Varun Chandrasekaran (visiting PhD student, Fall 2019)
• Lucas Bourtoule (MASc, started Fall 2019)
• Adelin Travers (PhD, started Fall 2019, co-supervised with David Lie)
• Matthew Jagielski (Google Brain intern, Summer 2019)

Selected publications

A complete list of publications is available in my CV.

• Machine Unlearning. Lucas Bourtoule, Varun Chandrasekaran, Christopher Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, Nicolas Papernot. preprint
• Rearchitecting Classification Frameworks For Increased Robustness . Varun Chandrasekaran, Brian Tang, Nicolas Papernot, Kassem Fawaz, Somesh Jha, Xi Wu. preprint
• Making the Shoe Fit: Architectures, Initializations, and Tuning for Learning with Privacy. Nicolas Papernot, Steve Chien, Shuang Song, Abhradeep Thakurta, Ulfar Erlingsson. preprint
• Thieves of Sesame Street: Model Extraction on BERT-based APIs. Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer. Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia. conference
• Improving Differentially Private Models via Active Learning. Zhengli Zhao, Nicolas Papernot, Sameer Singh, Neoklis Polyzotis, and Augustus Odena. Presented at the NeurIPS 2019 Workshop on Privacy in Machine Learning. workshop
• High-Fidelity Extraction of Neural Network Models. Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, Nicolas Papernot. preprint
• How Relevant Is the Turing Test in the Age of Sophisbots?. Dan Boneh, Andrew J. Grotto, Patrick McDaniel, Nicolas Papernot. IEEE Security and Privacy Magazine. invited
• MixMatch: A Holistic Approach to Semi-Supervised Learning. David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel. Proceedings of the 33rd Conference on Neural Information Processing Systems, Vancouver, Canada. conference
• Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness. Jorn-Henrik Jacobsen, Jens Behrmannn, Nicholas Carlini, Florian Tramer, Nicolas Papernot. Presented at the ICLR 2019 workshop on Safe ML, New Orleans, Louisiana. workshop
• Analyzing and Improving Representations with the Soft Nearest Neighbor Loss. Nicholas Frosst, Nicolas Papernot, Geoffrey Hinton. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA. conference
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. Nicolas Papernot and Patrick McDaniel. preprint
• Scalable Private Learning with PATE. Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Ulfar Erlingsson. Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada. conference
• Ensemble Adversarial Training: Attacks and Defenses. Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel. Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada. conference
• Towards the Science of Security and Privacy in Machine Learning. Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Proceedings of the 3rd IEEE European Symposium on Security and Privacy, London, UK. conference
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. Nicolas Papernot, Martin Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Proceedings of the 5th International Conference on Learning Representations, Toulon, France. best paper
• Practical Black-Box Attacks against Machine Learning. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z.Berkay Celik, and Ananthram Swami. Proceedings of the 2017 ACM Asia Conference on Computer and Communications Security, Abu Dhabi, UAE. conference
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples. Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. technical report
• Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks. Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Proceedings of the 37th IEEE Symposium on Security and Privacy, San Jose, CA. conference
• The Limitations of Deep Learning in Adversarial Settings. Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. Proceedings of the 1st IEEE European Symposium on Security and Privacy, Saarbrucken, Germany. conference

Recorded Talks and Blog Posts

These resources are a good overview of my research interests. The following three videos are (left) a lecture I gave in the Spring 2019 on security and privacy in machine learning, (middle) an oral I gave on PATE at ICLR 2017, and (right) a talk that highlights our early work on security in machine learning.

Here is a list of blog posts discussing some of the research questions I'm interested in:

• The academic job search for computer scientists in 10 questions
• How to know when machine learning does not know
• Machine Learning with Differential Privacy in TensorFlow
• Privacy and machine learning: two unexpected allies?
• The challenge of verification and testing of machine learning
• Is attacking machine learning easier than defending it?
• Breaking things is easy

Teaching

• [Winter 2020] ECE1513: Introduction to Machine Learning
• [Fall 2019] ECE1784H: Trustworthy Machine Learning