![[Translate to Englisch:]](/fileadmin/Repositories/Imbus/Header_Banner/Header_AI.png "[Translate to Englisch:]")
Scenario-based testing as a new test method
An autonomous system is characterized by the fact that it is able to independently control and reach a given goal. The system is free to choose the path to the goal within a wide range. The tasks that the system must solve (on this path) can be divided into subtasks. For example, driving towards an intersection; stopping if the traffic light is red; starting again as soon as the traffic light is green. Testing autonomous systems therefore means systematically checking as many of the possible or conceivable situations as possible within the framework of various such task chains.
The systematic testing of such a chain of tasks therefore requires the most comprehensive possible catalogue of such conceivable or possible situations or events. Analogous to the equivalence classes used in testing classic IT systems, these are broken down into individual parameters and can then be systematically linked to form task or scenario chains. Within the framework of this so-called scenario-based testing, certain situations are then specifically brought about in order to examine and test how the autonomous system acts in each case. The biggest difference to testing classic IT systems is that the same test sequence (e.g. driving over an intersection) must be repeated in a very large number of variations of the test environment (intersection with/without traffic lights, with/without pedestrians, during the day, at night, in the rain, ... etc.). Normally the test environment is constant or limited to a few predefined variations.
What is needed in this context is a central scenario catalogue, published by a neutral instance and in a manufacturer-independent, generally usable format. Intensive work is therefore being done on the standardization of suitable formats for the exchange of such scenarios. "ASAM Open-SCENARIO" of the automotive industry, for example, defines such a format for the description of the dynamic contents of driving and traffic simulators.
Safety & Security as central quality features:
The purpose of all quality assurance measures for a product is to create confidence that the product concerned meets the requirements of its stakeholders (users, customers, manufacturers, legislators, etc.).
The ISO 25010 standard is the international standard that defines a universal grid for quality characteristics of software and IT systems. It distinguishes between the following quality characteristics: Functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability and portability. These quality characteristics can and should therefore also be used as a starting point when creating a test plan or test case catalog for testing an autonomous system. However, the criteria "functional safety" and "security" are of outstanding importance and weighting.
Test-driven development („shift left“ & „shift right“)
The interaction of the autonomous system with a complex environment leads to an astronomical number of potentially testable scenarios. This combinatorics can only be mastered if the testing and validation of the resulting software takes place continuously at the level of each individual component and from the first day of development ("shift left").
The European Commission's ethical guidelines for a trustworthy AI clearly state the following requirements: "The testing and validation of the system should take place as early as possible to ensure that the system behaves as intended throughout its life cycle and especially during use". In addition, there must also be continuous monitoring of the systems in operation ("shift-right"). And, if necessary, there must be a rapid response to problems in operation.
