Toolborne Integratio... Research Projects Detection of Rare So...

Software reliability estimation on the basis of
the operational experience of pre-developed components

Project CORE - Component based Reliability Estimation


In safety-critical application areas with high reliability demands, a quantitative assessment of software reliability is crucial for licensing the use of software systems. For the purpose of estimating software reliability, it is not the number of software faults which is significant, but rather the frequency of their manifestation during runtime. Therefore, a sound reliability assessment has to be carried out by anticipating as far as possible the expected operational profile during the testing phase.

As described in former annual research reports a preliminary sub-project investigated techniques for logging usage data and visualise corresponding operational profiles in order to document past operational experience. To do so, a tool was implemented allowing the logging of all external method calls of a predefined Java component during runtime. This technique for logging runtime calls is intended to support systematic decision-making concerning the need for additional tests of functionalities to be re-used in a new context.

In another former sub-project a method was developed permitting to estimate the future operational profile of a software program. This method was applied to a real-world medical system for the calibration of hearing aids. Based on this information the test process can be adjusted to future software usage by selecting single test cases with operationally representative frequencies. In such a way, test results can be converted into reliability estimates by means of reliability growth models.

In the past, the application of statistical hypothesis testing for the purpose of quantitative reliability estimation was investigated. Based on a preliminary testing or usage period, an upper bound of the failure probability can be determined at any given confidence level. In the case of pre-developed software components, component-specific reliability estimates can be composed into a single system reliability assessment. For this purpose, approaches were developed which allow for the architecture-based combination of component-specific reliability estimates. Moreover, techniques for the sensitivity analysis and optimisation of additional testing were derived.

Apart from these theoretical considerations, more practical questions concerning the recording, the analysis and the statistical evaluation of past operational usage were considered. For this purpose, a general guideline was developed, supporting the statistical analysis of operational data. Within a student thesis, a tool was designed and implemented, supporting the extraction of independent subsets of operational data by application of genetic algorithms.

In order to investigate the practical applicability of this guideline, an industrial research cooperation with the automotive supplier ZF Friedrichshafen AG was accomplished: a software-based gearbox controller was analysed in the light of the applicability of statistical testing.

This project was finally finished within the last annual report period by a PhD-Thesis considering these topics, which was published in a series of the working reports of the department of computer science.


Contact Person: Dr. Sven Söhnlein