Weibull models for software reliability: case studies Pavel Praksa Tom´ aˇ s Musilb Petr Zajacc Pierre-Etienne Labeaud mailto:
[email protected] December 6, 2007 a VSB ˇ
- Technical University of Ostrava, Dept. of Applied Mathematics, Center of Quality and Reliability of Production (CQR.CZ), Universit´ e Libre de Bruxelles (ULB), Service de M´ etrologie Nucl´ eaire (SMN) b Czech Technical University in Prague, Sun Microsystems Czech c VSB ˇ - Technical University of Ostrava d ULB, SMN, Av. F.D. Roosevelt 50, 1050 Bruxelles, Belgium
1
Motivation 1 Software Reliability is the probability of failure-free software operation for a specified period of time in a specified environment. Software Reliability is also an important factor affecting system reliability. It differs from hardware reliability in that it reflects the design perfection, rather than manufacturing perfection. The high complexity of software is the major contributing factor of Software Reliability problems. Jiantao Pan, Carnegie Mellon University, 1999
2
Motivation 2: Notions from IEEE Standard Glossary of Software Engineering Terminology(IEEE Std 610.12-1990) • Availability. The degree to which a system or component is operational and accessible when required for use. Often expressed as a probability. • Reliability. The ability of a system or component to perform its required functions under stated conditions for a specified period of time. • Software diversity. A software development technique in which two or more functionally identical variants of a program are developed from the same specification by different programmers or programming teams with the intent of providing error detection, increased reliability, additional documentation, or reduced probability that programming or compiler errors will influence the end results. 3
Motivation 3 Software-Reliability-Engineered Testing The concept of reliability here is defined as the probability of execution without failure for some specified interval, generally called mission time. John D. Musa, AT&T Bell Laboratories and James Widmaier, National Security Agency http://www.stsc.hill.af.mil/crosstalk/1996/06/reliabil.asp
4
Motivation 4 Software Reliability Techniques http://sw-assurance.gsfc.nasa.gov/disciplines/reliability/index.php Reliability techniques can be divided into two categories: 1. Trending reliability tracks the failure data produced by the software system to develop a reliability operational profile of the system over a specified time. 2. Predictive reliability assigns probabilities to the operational profile of a software system; for example, the system has a 5 percent chance of failure over the next 60 operational hours. In practice, reliability trending is more appropriate for software, whereas predictive reliability is more suitable for hardware. Trending reliability can be further classified into four categories: Error Seeding, Failure Rate, Curve Fitting, and Reliability Growth. 5
Contents 1 Basic Mathematical Concepts
8
2 Bathtub curve model
9
3 Assumed Weibull models
13
4 Maximum likelihood estimation of Weibull models
17
4.1
Case A (One parameter Exponential model) . . . . . . . 18
4.2
Case B (Three parameter Weibull model, with nonnegative location parameter) . . . . . . . . . . . . . . . 19
4.3
Case C (Three parameter Weibull model, with negative location parameter) . . . . . . . . . . . . . . . . . . . . 21 6
4.4
Case D (Two parameter Weibull model) . . . . . . . . . 22
5 Results of parameter estimations
23
5.1
Estimation of 3-parameter Weibull model . . . . . . . . 24
5.2
Estimation of parameters for a software reliability task . 36
6 Conclusions
40
7 Acknowledgment
42
7
1
Basic Mathematical Concepts
Reliability R(t) is the probability that a system will be successful in the interval from time 0 to time t: R(t) = P (T > t), t ≥ 0
(1)
where T is a random variable denoting the time-to-failure or failure time. Unreliability F (t) is defined as the probability that the system will fail by time t: F (t) = P (T ≤ t) = 1 − R(t), t ≥ 0. F (t) is the failure distribution function.
8
(2)
2
Bathtub curve model
Maintenance modeling and consequent maintenance optimization of the system availability and or maintenance costs deeply depend especially on the ability of the model to predict the future behavior of the system. In our studies, we analyze a family of generalized Weibull models with 3 parameters which are based on the cumulative distribution function (CDF) β
−( t−ν η )
F (t) = 1 − e
9
,t ≥ ν ≥ 0
(3)
with the corresponding hazard function:
λ(t) =
β η
t−ν η
β−1
,t ≥ ν ≥ 0
(4)
The symbol η > 0 is called scale parameter, the symbol ν ≥ 0 is called location parameter. When ν = 0, the model is reduced to the 2 parameter Weibull. Knowledge about future behavior of the system is reflected by the shape parameter β. We speak about the bathtub curve reliability model, widely used in the reliability domain, see Figure 1.
10
Figure 1: The bathtub curve reliability model. 11
Weibull models appear to be very flexible not only for reliability models, but also for various technical applications. Weibull models were originally used for the scattering static strength of brittle materials [8], recently also for the modeling of wind speed[6], for analysis of unwanted radar echo signals [7], for image retrieval [3], and in the maintainability field for ageing models [5] .
12
3
Assumed Weibull models
We assume the following probability density functions for our parameter estimation study: • Weibull model with 2 parameters: βa −1 β βa t −( ηta ) a e fa (t) = ηa ηa
(5)
The cumulative distribution function (CDF) of the 2-parameter Weibull is: β −( ηta ) a Fa (t) = 1 − e (6)
13
The scale parameter symbol ηa has the following meaning: Solving equation (6) for t = ηa we have: βa
a −( η ηa )
Fa (ηa ) = 1 − e
= 1 − e−1 = 1 − 0.3679 = 0.6321
(7)
So, the scale parameter ηa is the 63rd percentile of the two parameter Weibull distribution. The scale parameter is sometimes called the Characteristic Life. The shape parameter β describes the speed of the ageing process in Weibull models and has no dimension.
14
• Weibull model with 3 parameters including a positive shift ν > 0. The positive shift means that failures are not possible directly from the start of operation: βb −1 t−νb βb βb t − νb − η b e fb (t) = (8) ηb ηb This probability density function is valid for t > νb and it is expected that fb (t) = 0 otherwise.
15
• Weibull with 3 parameters, including a negative shift ν < 0. The Weibull model with a negative shift can be used for model the pre-ageing components. The probability density function has the following conditional form: βc −1 c βc −( t−ν βc t−νc ) η c e ηc ηc (9) fc (t) = c βc −( −ν e ηc ) This probability density function is valid for νc ≤ 0 and it is expected that fc (t) = 0 for νc > 0.
16
4
Maximum likelihood estimation of Weibull models
In this section we describe parameter estimation studies using the Maximum Likelihood Estimation method. We assume the following probability density functions for the the distribution of failure times. Complete (i.e. no censoring) set of independent failure data ti > 0 for i = 1, 2, . . . , n are considered.
17
4.1
Case A (One parameter Exponential model)
Let us start with a model with constant failure rate λ(t) = λa > 0, which represents random failures. The corresponding likelihood function has then the following form: La =
n Y
i=1
−λa ti
λa e
=
λna e−λa
Pn
i=1
ti
(10)
By the logarithming of La we finally obtain: ln(La ) = n ln(λa ) − λa
n X i=1
18
ti
(11)
4.2
Case B (Three parameter Weibull model, with non-negative location parameter)
The three parameter Weibull model, with non-negative location parameter νb has the following failure rate: βb −1 βb t − νb λb (t) = H(t − νb ) (12) ηb ηb By assuming ti ≥ νb for i = 1, 2, . . . , n, the likelihood function can be written as follows: " # R n n β −1 b β ti Y βb ti − νb Y t −ν b λb (u) du − − iη b b = e λb (ti )e 0 Lb = ηb ηb i=1 i=1
(13)
19
By logarithming of Lb we finally obtain: " X βb # n βb ti − νb ti − νb ln(Lb ) = n ln + − (βb − 1) ln ηb ηb ηb i=1
(14)
20
4.3
Case C (Three parameter Weibull model, with negative location parameter)
The three parameter Weibull model, with a location parameter νc < 0 has the likelihood function with the following conditional form: βc −1 ti −νc βc −( ηc ) βc ti −νc n e Y ηc ηc Lc = (15) −νc βc e−( ηc ) i=1 By logarithming of Lc we finally obtain: β −νc c ln(Lc ) = ln(Lb ) + n ηc
21
(16)
4.4
Case D (Two parameter Weibull model)
The two parameter Weibull model is a special case of the three parameter Weibull model with the location parameter νb = 0. Following (12), we can write the failure rate as follows: βd −1 βd t λd (t) = ηd ηd
(17)
The expression of the likelihood function Ld is also the same as the expression of Lb with νb = 0, see (13). The expression (14) with νb = 0 is also suitable for the software implementation.
22
5
Results of parameter estimations
In this section, we present results of parameter estimations of Weibull models by the maximum likelihood method. A Weibull law is postulated; a sample of failure data is drawn according to this ”reality”. MLE computations are done on this basis mentioned in slide 24. In all cases, we use the following notation for diagrams of the estimated cumulative distribution functions: Case B (Three parameter Weibull model, with non-negative location parameter) is marked by the ’+’ symbols. Case C (Three parameter Weibull model, with negative location parameter) is marked by the ’-’ symbols. Case D (Two parameter Weibull model) is marked by the ’o’ symbols.
23
5.1
Estimation of 3-parameter Weibull model
In order to verify the computer implementation of the maximum likelihood estimator, we first sampled randomly failure data from a three parameter Weibull distribution and than we analysed this random sample by the MLE using the simulated annealing approach accelerated by the local Nelder-Mead optimizer [4] . We assumed the following parameters of the reality (η, β, ν) = (2, 4, 0). We first estimated parameters of this Weibull model using N = 30, N = 100, N = 1000 and N = 10000 samples. The results of the MLE estimation seems to be correct, see following figures.
24
Notes: The symbol ”Zero” represents the MLE results related to the Weibull model with 2 parameters. The symbol ”Neg” represents the Weibull model with 3 parameters including a negative shift and finally the symbol ”Pos” represents the Weibull with 3 parameters, including a positive shift. Of course, the estimated parameters are the same in this simple case and are very close to the parameter of the Weibull model, especially in case for N = 1000 and N = 10000 .
25
Results of estimations (Figures of distribution functions)
26
27
28
29
30
Results of estimations (Figures of density functions)
31
32
33
34
35
5.2
Estimation of parameters for a software reliability task
The developed software for parameter estimation of Weibull models was also used for reliability analyses of the real failure data related to a large software project (software reliability). The failure data were obtained from the company Sun Microsystems Czech. We analyzed three different sets of failure data. Results of parameter estimations are presented in the following figures:
36
37
38
39
6
Conclusions
The ability of the model to predict the future behavior of the system is a crucial property for the correct maintenance modeling and consequent optimization of the system availability and/or maintenance costs. 1. We presented various experiments showing capability of Weibull models to fit various failure data from a software reliability task. These results are now analyzed.
40
2. We also used the developed software also for the analysis of failure times and/or stopping times (right-censored times) belonging to the secondary water circuit of nuclear plants [1]. We successfully verified the parameters of two parameter Weibull model presented in the PhD. thesis [1] and we also extended results by assuming the three parameter Weibull model. 3. All computations run smoothly, without technical difficulties. We also computed the Fisher information matrix by using the automatic BFGS approach [2]. The precision of the estimated Fisher information matrix is now checked.
41
7
Acknowledgment
The support of the Universit´e Libre de Bruxelles, Belgium related to the ARC Project - ”Advanced supervision and dependability of complex processes: application to power systems” is graciously acknowledged by P. Praks. The research has been also supported by the Ministry of Education, Youth and Sports of the Czech Republic under the Research project 1M06047 ”The Center of Quality and Reliability” (CQR.CZ) and by the K-Space NoE project.
42
References [1] Nicolas Bousquet. Analyse bay´esienne de la dur´ee de vie de composants industriels. PhD thesis, D´epartement de Math´ematiques d’Orsay, Universit´e Paris-Sud 11, 2006. [2] J. V. Burke, A.S. Lewis, and M.L. Overton. Hanso (hybrid algorithm for non-smooth optimization): a matlab package based on bfgs, bundle and gradient sampling methods. Courant Institute of Mathematical Sciences, New York University, 2006. www.cs.nyu.edu/faculty/overton/software/hanso. [3] Jan-Mark Geusebroek and Arnold W. M. Smeulders. A six-stimulus theory for stochastic texture. Int. J. Comput. Vision, 62(1-2):7–16, 2005. [4] A. Hedar and M. Fukushima. Heuristic pattern search and its 43
hybridization with simulated annealing for nonlinear global optimization. Optimization Methods and Software, 19(3 - 4):291 – 308, 2004. [5] P.-E. Labeau. Mod´elisation du vieillissement industriel et impact de la maintenance pr´eventive. In Maintenance et Matrise des Risques, November 2006. [6] Edgar G. Pavia and James J. O’Brien. Weibull statistics of wind speed over the ocean. Journal of Applied Meteorology, 25:1324 – 1332, October 1986. http://coaps.fsu.edu/bios/86-10.pdf. [7] Matsuo Sekine and Yuhai Mao. Weibull Radar Clutter. Institution of Engineering and Technology, 1990. [8] W. Weibull. A statistical theory of the strength of materials. The Royal Swedish Institute of Engineering Research, Stockholm, 151:1– 45, 1939. 44
