VRMaaS: Verifiable Reward-Model-as-a-Service for Trustworthy LLM Alignment

Summary

Researchers propose VRMaaS, a framework that makes each reward model score returned by a service a proof-carrying artifact, binding request identity, committed weights, verifier and configuration identifiers, and the package digest so auditors can check it without rerunning the private model. A GPU prototype on 1B, 3B, and 8B reward models keeps 93.5-99% pairwise agreement with the baseline and rejects every tested service-field substitution, generating complete artifacts in 49.8-206.1 seconds and verifying them in 1.94-5.11 seconds.