Exploring Inference Patterns of LLMs as Symbolic Interactions

Summary

The paper examines whether a sparse set of symbolic interactions can faithfully explain the inference patterns of large language models, and uses interactions to diagnose LLM representations. The authors find that 50 to 100 interactions are usually enough to mimic changes in an LLM's output scores across an exponential number of masked inputs. They also report that simple interactions are more consistently extracted across architectures and semantically similar inputs than complex ones, and that low-order interactions can guide model compression.