InfoResearchPeer-reviewedLLM-specific
Inner-Probe: Discovering Copyright-Related Data Generation in LLM Architecture
- Published
- Record updated
Summary
Researchers propose Inner-Probe, a lightweight framework that estimates how much copyrighted subdatasets influence LLM-generated text. Instead of relying only on text, it trains an LSTM-based network on multihead attention (MHA) results during output generation to analyze subdataset contributions. It also detects noncopyrighted text via a concatenated global projector trained with unsupervised contrastive learning. Reported results include 3× efficiency over semantic model training on Books3, 15.04%–58.7% higher accuracy than baselines on the Pile, and a 0.104 AUC increase for noncopyrighted data filtering.