InfoResearchPreprintLLM-specific
Antidistillation is Cheap: Lightweight Distillation-Resistance via Stylistic Corruption
- Published
- Record updated
Summary
Researchers propose a lightweight defense against knowledge distillation, where a competitor clones a black-box LLM's capabilities through its API. The method dynamically injects style-steering vectors into the teacher model's activations when a query is judged malicious, so adversaries receive outputs that look pristine while ordinary users get unmodified responses. The authors report that the approach disrupts student learning with negligible overhead and preserves text naturalness.
Related items
- InfoCutpurse: Decomposing Explanation-Guided Model Extraction on Tabular DataSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- InfoFingerprinting the Dispatchers behind Distributed Model ExtractionSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- InfoModel Extraction of Convolutional Neural Networks with Max-PoolingSimilar attack · OpenAlex (peer-reviewed AI security)
- InfoTrustworthy Distributed Machine Learning in Cloud-Edge-IoT Systems: A Heterogeneity-Aware Survey of Security, Privacy, and RobustnessSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- InfoPreventing Model Extraction in Contextual Linear OptimizationSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)