Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs

Summary

Researchers show that poisoning a retrieval-augmented generation (RAG) system can amplify bias in an LLM's outputs, even for gender-neutral queries. Their Bias Retrieval and Reward Attack (BRRA) framework generates adversarial documents using multi-objective reward functions, manipulates retrieval with subspace projection, and uses a cyclic feedback mechanism, with experiments on several mainstream models showing significant bias increases. The paper also explores a dual-stage defense mechanism to mitigate the attack.

Mitigation

The source mentions a dual-stage defense mechanism that it says can effectively mitigate the impacts of the attack, but it does not describe its specifics, configuration or implementation.

Topics

• Retrieval-augmented generation

Related items

• InfoIAMRAGBENCH: Benchmarking Authorization Expansions in LLM-Generated AWS IAM Policies under RAG PoisoningSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoAccept Changes, Reject Attacks: Robust Memory Management for LLM AgentsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoA Scoping Review of Agentic AI: Applications, Emerging Trends, Risks & Future DirectionsSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoGuardRAG: A Post-Retrieval Defense against Poisoning Attacks in Retrieval-Augmented GenerationSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoRanking Is Not Deciding: Benchmark Shortcuts and Operating-Point Collapse in Query-Time RAG Poisoning DefensesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)