Adversarial machine learning

Research cited in the article indicates that roughly 250 documents or images can distort the behavior of a large language model, far below earlier assumptions of thousands or millions of corrupted data points. Online communities have already begun seeding fabricated facts to influence LLM training data, and a Purdue, Texas A&M and UT Austin team found that junk data causes capability decay that clean data added later did not fully reverse.

The author evaluates Microsoft Counterfit, a command-line tool for testing machine learning models and endpoints against adversarial attacks. Counterfit hosts attack modules from the Adversarial Robustness Toolbox and TextAttacks, and it can be extended with new attacks and targets. The author builds a custom Husky AI target that loads a Keras model and sample images, with the goal of turning Shadowbunny into a husky through adversarial examples.

Johann Rehberger (wunderwuzzi23) demonstrates the Adversarial Robustness Toolbox (ART), originally created by IBM and moved to the Linux AI Foundation in July 2020, to generate adversarial examples against his Husky AI binary image classifier. Using ART's FastGradientMethod with targeted=True and eps=0.04, he perturbs an image of a plush bunny so the model's prediction shifts from about 0.002 to 66% husky. The post presents this as an easier alternative to his earlier manual perturbation attacks.