Vulnerability of Natural Language Classifiers to Evolutionary Generated Adversarial Text
Manjinder Singh, Alexander E. I. Brownlee, Mohamed Elawady
Read on arXiv →
Key claim
GAversary reduces model accuracy significantly with genetic algorithms.
In plain English
Imagine you're building a natural language processing system that needs to be robust against attacks. Adversarial inputs can trick your model by making tiny changes to the text that are hard for humans to notice but can completely confuse the model. Current methods to generate these adversarial examples often rely on specific knowledge of the model's structure, which can limit their effectiveness and adaptability. This is what's called model-specific vulnerability. When attackers know how a model works, they can exploit its weaknesses more easily, leading to significant drops in performance when faced with these adversarial inputs.
What this paper introduces is a new way to create these adversarial examples without needing to understand the model's inner workings. The approach uses a genetic algorithm, which is a method inspired by natural selection, to explore potential word replacements that maintain the original meaning while still fooling the model. By using GloVe embeddings, the algorithm can find semantically similar words to replace, which helps in crafting more effective adversarial examples.
In practice, GAversary has shown to drastically reduce the accuracy of target models on benchmark datasets, outperforming previous methods like BAE and A2T. However, it does come with a trade-off: it tends to change more words in the input and takes a bit longer to run. For anyone building NLP systems, this means you have a new tool that can help you understand and defend against adversarial attacks more effectively, even if it requires a bit more computational effort.
The paper introduces a novel approach using genetic algorithms for generating adversarial attacks in NLP.
The claims are supported by comparisons against established methods and benchmark datasets.
Deep reliability assessment
The paper’s methodology supports that a GA guided by counterfitted GloVe with a context mask can craft untargeted, logit-guided black-box adversarial examples that substantially drop accuracy on binary sentiment classifiers, albeit with higher perturbation rates. It likely overclaims generality to broader NLP tasks and truly opaque black-box settings (label-only), and underplays the practical trade-offs of increased word changes and potential query costs despite the reported ~5% runtime increase.
Reproducibility
No code or data link is mentioned; the method relies on TextAttack (fitness-proportionate selection, counterfitted GloVe embeddings, NLTK stopwords) and specifies some hyperparameters (e.g., S=60), but datasets, target models, and full training/eval protocol are not enumerated in the excerpt, making reproduction only partially specified.
Key figure
The key diagram illustrates the mutation step: mask the target word, derive a context-based embedding from surrounding tokens, and choose replacement candidates whose GloVe vectors are closest to the masked context to propose semantically plausible edits.