Test Case Prioritization in Highly-Configurable Software Systems

Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of faults. The importance of this technique has been recognized in the context of Highly-Configurable Systems (HCSs), where the potentially huge number of configurations makes testing extremely challenging. In this book, we address the problem of test case prioritization in HCSs to help to improve the effectiveness of testing allowing faster feedback to software testers and ensuring that configurations with the highest fault detection ability will have been assessed if testing is stopped by any circumstance. We present thirteen prioritization objectives, six of them based on functional properties of the system and seven objectives based on non-functional properties of the HCS under test. Furthermore, we propose single-objective and multi-objective prioritization approaches based on evolutionary algorithms. Additionally, we provide an industry-strength HCS case study with more than 2 billions of configurations to be used as a realistic subject for further and reproducible validation of HCS testing techniques.