Practical Insights Into AI System Product Quality Evaluation

This research examines how ISO/IEC 25059 (an international standard for evaluating AI system quality) can be applied in practice, using an AI system that analyzes images of oil platform decks as a test case. The study highlights that when checking if AI systems work correctly, teams need to carefully define what counts as acceptable performance, especially for safety-critical applications (systems where failures could cause serious harm), and they should choose test cases (examples used to verify the system works) that realistically represent how the system will be used in the real world.