Jev: Decision Models on Trial

Summary

The article compares Laya, an open-weight model from ConvAI Innovations, and Jev, a hosted model from TypeSafe, against a custom-trained 1B-parameter encoder on content safety tasks. The decision models answer one yes/no question per harm category, with the higher of two probabilities used as the category score. The comparison spans 24 harm categories across six datasets, with reference labels generated by an LLM rather than human-adjudicated ground truth.