K-TCDP: A Temporal Correlated DP Mechanism for LoRA Supervised Fine-Tuning

Summary

The paper proposes K-TCDP, a finite-memory noise mechanism for differentially private LoRA supervised fine-tuning of large language models. It adds controlled negative temporal correlation to full-dimensional Gaussian noise in the LoRA adapter, so earlier perturbation can be partly offset later, while remaining compatible with standard RDP accounting. On four GLUE tasks and DART (BLEU-4, ROUGE-L), it consistently outperforms DP-SGD at the same privacy levels with small computational overhead.