Practical Insights Into AI System Product Quality Evaluation

Summary

This study applies the ISO/IEC 25059 standard to evaluate the functional suitability of an AI system, using a case study of a system that analyzes images of oil platform decks. The authors discuss the practical nuances of assessing functional correctness and share specific properties defined to operationalize the evaluation. They stress defining acceptable specification limits in safety-critical contexts and systematically considering the representativeness and sample size of test cases.