Testing methodology for AI

A product or service must meet user expectations, fulfill contractual requirements, and comply with legal requirements and standards. In an AI project, too, all of this must be checked and tested accordingly.

However, the requirements for AI-based systems are different. The behavior of an artificial intelligence is deterministically difficult to predict, which means that classic test design methods such as limit analysis or the white-box method can no longer be used, or at least not as easily. The established approach to testing conventional software (in the non-AI sense), namely to use a test oracle to determine what a system's response should be and then to check whether the actual response matches the predicted one, is therefore also no longer so easy to apply.

New test methods such as metamorphic testing are therefore required, and the question arises as to where the previous test procedure can continue to be used and where the test process and the methods used must be adapted or set up anew.

However, the previous test procedure still has its raison d'être in this context. This is because AI only fulfills certain subtasks in some systems. Other systems are AI-based, but contain conventional components or work together with them. In these cases, the conventional parts of these systems and also the interfaces between AI and non-AI are tested using conventional methods. So previous test methods and quality metrics will still be applicable in this case and a mix.

On the question "Which test methods are applicable and economical in my project?" imbus can advise you and offer support in the planning, specification and execution of tests.

Get adivce now!