Skip to content
InfoResearchPreprint

Cracks in Embeddings: Robustness and Stability of JEPA-Based Chest X-Ray Representations

Published
Record updated
View JSON

Summary

Researchers tested four JEPA-based chest X-ray encoders, trained on 78,468 NIH ChestX-ray14 images, under two black-box adversarial attacks (SPSA, Boundary Attack), three image corruptions, and a label-scarcity test. The perturbations caused severe, model-dependent damage, with encoder-level Cosine Similarity dropping as low as 0.33 and AUROC falling by as much as 0.12 for I-JEPA under the Boundary Test. Adversarial fine-tuning, which trained the predictor on clean and perturbed images together, raised embedding consistency above 0.97 in nearly every case and improved AUROC for most models, with the largest gain of +0.10 on I-JEPA.

Mitigation

Adversarial fine-tuning, which trains the small predictor on clean and perturbed images together, raised embedding consistency above 0.97 in nearly every case and improved AUROC for most models.