Skip to content
InfoResearchPreprintLLM-specific

Antidistillation is Cheap: Lightweight Distillation-Resistance via Stylistic Corruption

Published
Record updated
View JSON

Summary

Researchers propose a lightweight defense against knowledge distillation, where a competitor clones a black-box LLM's capabilities through its API. The method dynamically injects style-steering vectors into the teacher model's activations when a query is judged malicious, so adversaries receive outputs that look pristine while ordinary users get unmodified responses. The authors report that the approach disrupts student learning with negligible overhead and preserves text naturalness.