Nicolas Papernot

Student and postdoc opportunities I will join the University of Toronto and Vector Institute as an assistant professor in the Fall 2019:

• If you are interested in joining my research group as a graduate student, please fill the following form and apply to the ECE or CS program.
• If you are interested in joining my research group as a postdoc, please send me an email directly with your CV and research statement.

Research scientist opportunities The Vector Institute is looking for research scientists interested in working on machine learning. Research scientists at Vector can supervise fully-funded graduate students from affiliated universities, and collaborate freely with the other members of the Institute. Please see this post and send me an email if you are interested.

2019

• Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness. Jorn-Henrik Jacobsen, Jens Behrmannn, Nicholas Carlini, Florian Tramer, Nicolas Papernot. Accepted at ICLR 2019 workshop on Safe ML, New Orleans, Louisiana. workshop
• A General Approach to Adding Differential Privacy to Iterative Training Procedures. H. Brendan McMahan, Galen Andrew, Ulfar Erlingsson, Steve Chien, Ilya Mironov, Nicolas Papernot, Peter Kairouz. Presented at NeurIPS 2018 workshop on Privacy Preserving Machine Learning, Montreal, Canada. workshop
• On Evaluating Adversarial Robustness. Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry. technical report
• Analyzing and Improving Representations with the Soft Nearest Neighbor Loss. Nicholas Frosst, Nicolas Papernot, Geoffrey Hinton. preprint
• Prototypical Examples in Deep Learning: Metrics, Characteristics, and Utility. Nicholas Carlini, Ulfar Erlingsson, Nicolas Papernot. preprint
• Adversarial Examples Influence Human Visual Perception. Gamaleldin F. Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian Goodfellow, Jascha Sohl-Dickstein. Computational and Systems Neuroscience (Cosyne) 2019, Lisbon, Portugal conference

2018

• A Marauder's Map of Security and Privacy in Machine Learning. Nicolas Papernot. Keynote at the 11th ACM Workshop on Artificial Intelligence and Security colocated with the 25th ACM Conference on Computer and Communications Security, Toronto, Canada keynote
• Adversarial Examples that Fool both Human and Computer Vision. Gamaleldin F. Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian Goodfellow, Jascha Sohl-Dickstein. 32nd Conference on Neural Information Processing Systems (NIPS), Montreal, Canada conference
• The challenges of making machine learning robust against adversarial inputs. Ian Goodfellow, Patrick McDaniel, Nicolas Papernot. Communications of the ACM (July 2018) column
• Characterizing the Limits and Defenses of Machine Learning in Adversarial Settings. Nicolas Papernot. dissertation
• CleverHans v2.1.0: an adversarial machine learning library. Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Kurakin et al. technical report
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning. Nicolas Papernot and Patrick McDaniel. preprint
• Scalable Private Learning with PATE. Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, Ulfar Erlingsson. 6th International Conference on Learning Representations, Vancouver, Canada conference
• Ensemble Adversarial Training: Attacks and Defenses. Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel. 6th International Conference on Learning Representations, Vancouver, Canada conference
• Towards the Science of Security and Privacy in Machine Learning. Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. 3rd IEEE European Symposium on Security and Privacy, London, UK conference

2017

• Adversarial Examples for Malware Detection. Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. European Symposium on Research in Computer Security, Oslo, Norway. conference
• On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches. Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Nicolas Papernot, Kunal Talwar, Li Zhang. 30th IEEE Computer Security Foundations Symposium, Santa Barbara, CA, USA invited paper
• Extending Defensive Distillation. Nicolas Papernot, Patrick McDaniel. Workshop track at 38th IEEE Symposium on Security and Privacy, San Jose, CA workshop
• Adversarial Attacks on Neural Network Policies. Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, Pieter Abbeel. Workshop Track at the 5th International Conference on Learning Representations, Toulon, France. workshop
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, and Kunal Talwar. 5th International Conference on Learning Representations, Toulon, France conference best paper
• The Space of Transferable Adversarial Examples. Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel preprint
• Practical Black-Box Attacks against Machine Learning. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z.Berkay Celik, and Ananthram Swami. ACM Asia Conference on Computer and Communications Security, Abu Dhabi, UAE conference
• On the (Statistical) Detection of Adversarial Examples. Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel preprint

2016

• Machine Learning in Adversarial Settings. Patrick McDaniel, Nicolas Papernot, Z. Berkay Celik. IEEE Security & Privacy Magazine column
• On The Integrity Of Deep Learning Systems In Adversarial Settings. Nicolas Papernot. masters thesis
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples. Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. technical report
• Crafting Adversarial Input Sequences for Recurrent Neural Networks. Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. Military Communications Conference (MILCOM), Baltimore, MD conference
• Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks. Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami 37th IEEE Symposium on Security and Privacy, San Jose, CA conference
• The Limitations of Deep Learning in Adversarial Settings. Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 1st IEEE European Symposium on Security and Privacy, Saarbrucken, Germany conference

2015

• Enforcing Agile Access Control Policies in Relational Databases using Views. Nicolas Papernot, Patrick McDaniel, and Robert Walls. Military Communications Conference (MILCOM), Tampa, FL conference

2014

• Security and Science of Agility. P. McDaniel, T. Jaeger, T. F. La Porta, Nicolas Papernot, R. J. Walls, A. Kott, L. Marvel, A. Swami, P. Mohapatra, S. V. Krishnamurthy, I. Neamtiu. ACM Workshop on Moving Target Defense workshop

I co-author a blog on the security and privacy of machine learning with Ian Goodfellow at www.cleverhans.io. I also write blog posts unrelated to machine learning on Medium and keep track of them here.

• cleverhans.io Machine Learning with Differential Privacy in TensorFlow
• cleverhans.io Privacy and machine learning: two unexpected allies?
• cleverhans.io The challenge of verification and testing of machine learning
• cleverhans.io Is attacking machine learning easier than defending it?
• cleverhans.io Breaking things is easy
• A review of “Return-Oriented Programming: Systems, Languages, and Applications.”
• Detecting phishing websites using a decision tree
• Kerberos: An Authentication Service for Computer Networks
• About Usable Security
• Keys Under Doormats: Mandating Insecurity by Requiring Government Access to All Data and Communications
• Internet of Things Security at Enigma 2016
• Healthcare Security at Enigma 2016
• Natural Language Processing

When a recording of the talk is available, the title links to the corresponding video. The following three embedded videos are a lecture I gave on security and privacy in machine learning (left) and two talks that highlight works representative of my research on privacy (middle) and security (right) in machine learning.

2019

• Title TBD (Princeton University)
• Title TBD (CVPR workshop on Challenges and Opportunities for Privacy and Security)
• Title TBD (CVPR workshop on Autonomous Driving)
• Title TBD (Microsoft)
• Title TBD (National Academies Workshop on the Implications of AI and ML for Cybersecurity)
• A Marauder's Map of Security and Privacy in Machine Learning (Palo Alto Networks)
• A Marauder's Map of Security and Privacy in Machine Learning (UC Berkeley) lecture
• A Marauder's Map of Security and Privacy in Machine Learning (Google Brain Zurich)
• A Marauder's Map of Security and Privacy in Machine Learning (EPFL Applied Machine Learning Days)

2018

• A Marauder's Map of Security and Privacy in Machine Learning (AISec '18) keynote (technical report, slides)
• Security and Privacy in Machine Learning (Google Launchpad Studio)
• Security and Privacy in Machine Learning (MSR Cambridge AI Summer School)
• Characterizing the Space of Adversarial Examples in Machine Learning (NVIDIA)
• Characterizing the Space of Adversarial Examples in Machine Learning (2nd ARO/IARPA Workshop on AML)
• Characterizing the Space of Adversarial Examples in Machine Learning (MIT-IBM Watson AI Lab)
• Characterizing the Space of Adversarial Examples in Machine Learning (Microsoft Research Cambridge)
• Characterizing the Space of Adversarial Examples in Machine Learning (University of Toronto)
• Characterizing the Space of Adversarial Examples in Machine Learning (EPFL)
• Characterizing the Space of Adversarial Examples in Machine Learning (University of Southern California)
• Characterizing the Space of Adversarial Examples in Machine Learning (University of Michigan)
• Characterizing the Space of Adversarial Examples in Machine Learning (Max Planck Institute for Software Systems)
• Characterizing the Space of Adversarial Examples in Machine Learning (Columbia University)
• Characterizing the Space of Adversarial Examples in Machine Learning (University of Virginia)
• Characterizing the Space of Adversarial Examples in Machine Learning (Intel Labs)
• Characterizing the Space of Adversarial Examples in Machine Learning (McGill University)
• Characterizing the Space of Adversarial Examples in Machine Learning (University of Florida)
• Security and Privacy in Machine Learning (Age of AI Conference)
• Security and Privacy in Machine Learning (Bar Ilan University)
• Security and Privacy in Machine Learning (IVADO)
• Security and Privacy in Machine Learning (Ecole Polytechnique Montreal)
• Security and Privacy in Machine Learning (Element AI)
• Security and Privacy in Machine Learning (INRIA Data Institute) tutorial

2017

• Security and Privacy in Machine Learning (IEEE WIFS 2017) tutorial
• Lecture on Security and Privacy in Machine Learning (Prof. Trent Jaeger's computer security class, Penn State) lecture
• Adversarial Machine Learning with CleverHans (ODSC West, joint tutorial with Nicholas Carlini) tutorial
• Security and Privacy in Machine Learning (Georgian Partners annual summit)
• Private Machine Learning with PATE (With the Best online conference)
• Gradient Masking in Machine Learning (Adversarial Machine Learning Workshop, Stanford University)
• Security and Privacy in Machine Learning (Ecole Centrale de Lyon)
• Security and Privacy in Machine Learning (Oxford University)
• Adversarial Machine Learning with CleverHans (ICML workshop on Reproducibility in ML) tutorial
• Adversarial Examples in Machine Learning (AI with the Best, jointly with Patrick McDaniel)
• Security and Privacy in Machine Learning (Deep Learning Summit Singapore)
• Security and Privacy in Machine Learning (Microsoft Research Cambridge)
• Security and Privacy in Machine Learning (University of Cambridge)
• Adversarial Examples in Machine Learning (Stanford AI Salon, joint invitation with Ian Goodfellow) panel
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (Stanford)
• Adversarial Machine Learning (Data Mining for Cyber Security meetup)
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (Symantec)
• Adversarial Examples in Machine Learning (Usenix Enigma 2017)
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (LeapYear)
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (Immuta)
• Machine Learning and Security (NSF 2017 Secure and Trustworth Cyberspace PIs Meeting) panel

2016

• Security and Privacy in Machine Learning (Ecole Centrale de Lyon)
• Adversarial Examples in Machine Learning (LinkedIn)
• Adversarial Examples in Machine Learning (Stanford)
• Adversarial Examples in Machine Learning (Berkeley)
• Adversarial Examples in Machine Learning (joint talk with Ian Goodfellow at AutoSens, Brussels)
• What role will AI play in the future of autonomous vehicles and ADAS? (AutoSens 2016) panel
• Adversarial Examples in Machine Learning (Google)

• Varun Chandrasekaran (PhD student visiting from University of Wisconsin in the Fall 2019)
• Lucas Bourtoule (MASc student, starting Fall 2019)
• Adelin Travers (PhD student, starting Fall 2019, co-supervised with David Lie)
• Matthew Jagielski (Google Brain intern, Summer 2019)

2019

• Canada CIFAR AI Chair

2018

• Recognized among top 30% reviewers for NIPS (Neural Information Processing Systems)
• Wormley Family Graduate Fellowship
• Outstanding Research Assistant Award (The Pennsylvania State University)
• Student Travel Award (6th International Conference on Learning Representations)

2017

• Student Travel Award (34th International Conference on Machine Learning)
• Student Travel Award (5th International Conference on Learning Representations)
• Best Paper Award (5th International Conference on Learning Representations)

2016

• Outstanding Graduate Research Award (The Pennsylvania State University)
• Google PhD Fellowship in Security (I am mentored by Samy Bengio)

2015

• Microsoft CyberSpace 2025 Essay Contest - 2nd place

2010

• Scholarship for Exceptional Academic Achievements (McGill) [declined]

Program committee member

Conferences: AAAI (2019), ACSAC (2018), AsiaCCS (2018), CCS (2018, 2019), GameSec (2018), NDSS (2018), IEEE S&P "Oakland" (2020), PETS (2019), USENIX Security (2019, 2020)

Workshops: Deep Learning and Security at IEEE Security & Privacy (2018), Privacy and Security at CVPR (2018), Dependable and Secure Machine Learning colocated with DSN (2018)

Organizing committee

• ICML Workshop on the Security and Privacy of Machine Learning (2019)
• 2nd DSN Workshop on Dependable and Secure ML
• NeurIPS workshop on Security in ML (2018) chair [Website]
• NIPS competition on adversarial ML (2018)
• NIPS workshop on Secure ML (2017)
• Self-Organizing Conference on Machine Learning "SOCML" (2017)
• With the Best online conference on Cybersecurity (2017)

Reviewer

Conferences: ACM WiSec (2016), DIMVA (2016), ICML (2019, 2018, 2017), ICLR (2019), IEEE Security & Privacy "Oakland" (2017, 2018), NIPS (2018, 2017), USENIX Security (2018)

Journals: IEEE Transactions on Dependable and Secure Computing (2017), IEEE Transactions on Information Forensics and Security (2017), IEEE Pervasive special issue on "Securing the IoT" (2017), Journal of Computer Security (2018)

Funding: Agence Nationale de la Recherche (2017), AI Xprize (2017-), Google Faculty Research Awards (2018, 2017)

Other: IEEE Security & Privacy Magazine (2017)

Invited participant

• "When Humans Attack" workshop at the Data & Society Research Institute (2018)
• ARO/IARPA Workshop on Adversarial Machine Learning, University of Maryland (2018)
• ARO Workshop on Adversarial Machine Learning, Stanford (2017)
• DARPA Workshop on Safe Machine Learning, Simons Institute (2017)

Defense committee member

• Ryan Sheatsley (M.Sc., Pennsylvania State University)

2019

• TF Privacy: The Verge, VentureBeat
• ML Security and Privacy: Le Monde

2018

• Differential privacy with PATE: TWiML
• Adversarial examples for humans: MIT Technology Review, The Next Web, IEEE Spectrum

2017

• Adversarial machine learning: Co.Design, The Verge, GCN
• CleverHans library: Die Zeit

2016

• Black-box attacks against ML: Communications of the ACM, Wired, Popular Science
• Differential privacy with PATE: Quartz
• CleverHans library: Quartz