Adversarially Trained Linear Transformers Are Optimal Robust In-Context Learners for Gaussian Mixtures

Summary

Researchers ask whether robustness from adversarial pretraining transfers to unseen tasks without further adversarial training. For Gaussian-mixture classification, a sufficiently deep linear transformer adversarially trained across tasks asymptotically attains the robust Bayes error on unseen tasks via in-context learning from clean demonstrations, while a standardly trained model cannot. The paper also analyzes convergence under gradient flow, an accuracy-robustness trade-off, and demonstration complexity.

Related items

• InfoDoes Target Alignment Mean Target Recovery? An Evidence-Ladder Study of Adversarial Claims on Contrastive EncodersSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoBRANCH: Bypassing Multi-Scanner AI GuardrailsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoDetecting Adversarial Images through Response Profiles of Vision-Language ModelsSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoGraphRectify: Graph-Based Transfer of Adversarial Example Detectors Across Neural NetworksSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoVCR-Bench: A Modular Open-Source Benchmark for Video Classification RobustnessSimilar attack · Arxiv (cs.RO + cs.CV security)