Perseus: A Full-Stack System for Homomorphic Model Inference

Summary

Perseus is a free, open-source Python library for encrypted model inference using Fully Homomorphic Encryption, covering the cryptographic backend, data packing, encrypted nonlinearities and end-to-end model execution. It adds optimizations at each level, including in-kernel regeneration of key-switching keys, a variant of Cachemir packing, and a bootstrap placer. It stays within 3% of plaintext accuracy and perplexity and is 3.96x faster at single-token decoding on Llama-3-8B than any other implementation.