Watermark Basins: Black-Box Detection of Distilled LLMs

Summary

Watermark Basin Alignment (WM-Basin) is a black-box detector that identifies models distilled from a watermarked teacher without access to weights or logits. It projects generated text into gradient space and tests for alignment with a key-specific watermark basin orthogonal to ordinary fine-tuning directions. At a 0.1% false-positive rate it detects 79% of distilled models at n-gram=1 and 45% at n-gram=3, versus 29% and 1.3% for the token-level baseline, and the signal survives quantization, pruning and reinforcement-learning post-training.