Cracks in Embeddings: Robustness and Stability of JEPA-Based Chest X-Ray Representations
- Published
- Record updated
Summary
Researchers tested four JEPA-based chest X-ray encoders, trained on 78,468 NIH ChestX-ray14 images, under two black-box adversarial attacks (SPSA, Boundary Attack), three image corruptions, and a label-scarcity test. The perturbations caused severe, model-dependent damage, with encoder-level Cosine Similarity dropping as low as 0.33 and AUROC falling by as much as 0.12 for I-JEPA under the Boundary Test. Adversarial fine-tuning, which trained the predictor on clean and perturbed images together, raised embedding consistency above 0.97 in nearly every case and improved AUROC for most models, with the largest gain of +0.10 on I-JEPA.
Mitigation
Adversarial fine-tuning, which trains the small predictor on clean and perturbed images together, raised embedding consistency above 0.97 in nearly every case and improved AUROC for most models.
Related items
- LowAdversarial Tracking: Attacks on Image Sensors for Object Tracking in Autonomous VehiclesSimilar attack · Arxiv (cs.RO + cs.CV security)
- InfoImage watermarking robustness under MLLM-guided semantic editing: Benchmark, defense, and diagnosisSimilar attack · Elsevier Security Journals
- LowYour Robot's Safety Functions Already Work. What If the Input Lies?Similar attack · VicOne Blog
- InfoSpeech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust TokenizerSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- InfoAdversarial Retraining for Machine Learning-Based Man-in-the-Middle Attack Detection in Smart Grid Industrial Control SystemsSimilar attack · OpenAlex (peer-reviewed AI security)