Developing Tools for Testing Adversarial Attacks Against Natural Language Classifiers

Proofpoint, Inc. Computer Science, 2022–23

Liaison(s): Cameron Malloy, Adam Starr POM ’18, Dana Harris CMC ’22
Advisor(s): Blake Jackson
Students(s): David Pitt (PM-S), Katie Johnson (PM-F), James Lucassen, Nanako Noda, Ingrid Wu

Proofpoint uses natural language processing systems to classify and filter out malicious or fraudulent emails. The Proofpoint Clinic team is improving existing tools to test the ways in which malicious content can evade Proofpoint’s models while remaining human-readable. The team is also researching attacks that switch between multiple languages to confuse language models and defenses against these attacks.