ReliaWiki - User contributions [en]

Probability Plotting

2011-06-30T22:30:32Z

Steve Sharp:

One method of calculating the parameter of the exponential distribution is by using probability plotting. To better illustrate this procedure, consider the following example.

 

====Example 1====
 

Let's assume six identical units are reliability tested at the same application and operation
stress levels. All of these units fail during the test after operating for the following times (in hours), <math>{{T}_{i}}</math> : 96, 257, 498, 763, 1051 and 1744.
 
The steps for determining the parameters of the exponential <math>pdf</math> representing the
data, using probability plotting, are as follows:
 
::• Rank the times-to-failure in ascending order as shown next.
 

::<math>\begin{matrix}
\text{Time-to-} & \text{Failure Order Number} \\
\text{failure, hr} & \text{out of a Sample Size of 6} \\
\text{96} & \text{1} \\
\text{257} & \text{2} \\
\text{498} & \text{3} \\
\text{763} & \text{4} \\
\text{1,051} & \text{5} \\
\text{1,744} & \text{6} \\
\end{matrix}</math>

 
::• Obtain their median rank plotting positions.
 
Median rank positions are used instead of other ranking methods because median ranks are at a
specific confidence level (50%).
 
::• The times-to-failure, with their corresponding median ranks, are shown next:

 
::<math>\begin{matrix}
\text{Time-to-} & \text{Median} \\
\text{failure, hr} & \text{Rank, }% \\
\text{96} & \text{10}\text{.91} \\
\text{257} & \text{26}\text{.44} \\
\text{498} & \text{42}\text{.14} \\
\text{763} & \text{57}\text{.86} \\
\text{1,051} & \text{73}\text{.56} \\
\text{1,744} & \text{89}\text{.10} \\
\end{matrix}</math>

 
::• On an exponential probability paper, plot the times on the x-axis and their corresponding
rank value on the y-axis. Fig. 4 displays an example of an exponential probability paper. The
paper is simply a log-linear paper. (The solution is given in Fig. 2.)
 
[[File:ALTA4.1.gif|center]]
 
<center>Fig. 4: Sample exponential probability paper.</center>
 
::• Draw the best possible straight line that goes through the <math>t=0</math> and <math>
(t)=100%</math> point and through the plotted points (as shown in Fig. 5).
 
::• At the <math>Q(t)=63.2%</math> or <math>R(t)=36.8%</math> ordinate point, draw a
straight horizontal line until this line intersects the fitted straight line. Draw a vertical line through this intersection until it crosses the abscissa. The value at the intersection of the abscissa is the estimate of the mean. For this case, <math>\widehat{\mu }=833</math> hr which means that <math>\lambda =\tfrac{1}{\mu }=0.0012</math> . (This is always at 63.2% since <math>(T)=1-{{e}^{-\tfrac{\mu }{\mu }}}=1-{{e}^{-1}}=0.632=63.2%).</math>
 
[[File:ALTA4.2.gif|center]]
 
<center>Fig. 5: Probability plot for Example 1.</center>
 

 
Now any reliability value for any mission time <math>t</math> can be obtained. For example, the
reliability for a mission of 15 hr, or any other time, can now be obtained either from the plot or analytically (i.e. using the equations given in Section <math>5.1.1</math> ).

 
To obtain the value from the plot, draw a vertical line from the abscissa, at <math>t=15</math>
hr, to the fitted line. Draw a horizontal line from this intersection to the ordinate and read
<math>R(t)</math> . In this case, <math>R(t=15)=98.15%</math> . This can also be obtained
analytically, from the exponential reliability function.

 

====MLE Parameter Estimation====

 
The parameter of the exponential distribution can also be estimated using the maximum likelihood estimation (MLE) method. This log-likelihood function is:

 
::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\overset{FI}{\mathop{\underset{i=1}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{\prime \prime }\ln [R_{Li}^{\prime \prime }-R_{Ri}^{\prime \prime }]</math>
 
where:
 
::<math>R_{Li}^{\prime \prime }={{e}^{-\lambda T_{Li}^{\prime \prime }}}</math>

 
::<math>R_{Ri}^{\prime \prime }={{e}^{-\lambda T_{Ri}^{\prime \prime }}}</math>

 
and
 
::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points.
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group.
::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found).
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data.
::• <math>S</math> is the number of groups of suspension data points.
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points.
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group.
::• <math>FI</math> is the number of interval data groups.
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the i <math>^{th}</math> group of data intervals.
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the i <math>^{th}</math> interval.
::• <math>T_{Ri}^{\prime \prime }</math> is the ending of the i <math>^{th}</math> interval.

 
The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0</math> where:
 
::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime }-\overset{FI}{\mathop{\underset{i=1}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{\prime \prime }\frac{T_{Li}^{\prime \prime }R_{Li}^{\prime \prime }-T_{Ri}^{\prime \prime }R_{Ri}^{\prime \prime }}{R_{Li}^{\prime \prime }-R_{Ri}^{\prime \prime }}</math>

 

====Example 2====

 
Using the same data as in the probability plotting example (Example 1), and assuming an exponential distribution, estimate the parameter using the MLE method.
 
''Solution''
 
In this example we have non-grouped data without suspensions. Thus Eqn. (exp-mle) becomes:
 
::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}} \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}} \right) \right]=0</math>

 
Substituting the values for <math>T</math> we get:

 
::<math>\begin{align}
\frac{6}{\lambda }= & 4409,\text{ or:} \\
\lambda = & 0.00136\text{ failure/hr}
\end{align}</math>

Bayesian-Weibull Analysis

2011-06-30T19:05:33Z

Steve Sharp: /* Confidence Bounds on R(T) */

In this section, the Bayesian methods are presented for the two-parameter Weibull distribution. Bayesian concepts were introduced in Chapter 3. This model considers prior knowledge on the shape (β) parameter of the Weibull distribution when it is chosen to be fitted to a given set of data. There are many practical applications for this model, particularly when dealing with small sample sizes and some prior knowledge for the shape parameter is available. For example, when a test is performed, there is often a good understanding about the behavior of the failure mode under investigation, primarily through historical data. At the same time, most reliability tests are performed on a limited number of samples. Under these conditions, it would be very useful to use this prior knowledge with the goal of making more accurate predictions. A common approach for such scenarios is to use the one-parameter Weibull distribution, but this approach is too deterministic, too absolute you may say (and you would be right). The Weibull-Bayesian model in Weibull++ (which is actually a true "WeiBayes" model, unlike the one-parameter Weibull that is commonly referred to as such) offers an alternative to the one-parameter Weibull, by including the variation and uncertainty that might have been observed in the past on the shape parameter. Applying Bayes's rule on the two-parameter Weibull distribution and assuming the prior distributions of β and η are independent, we obtain the following posterior :

::<math> f(\beta ,\eta |Data)=\dfrac{L(\beta ,\eta )\varphi (\beta )\varphi (\eta )}{ \int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta } </math> EQNREF WeibBayes

In this model, η is assumed to follow a noninformative prior distribution with the density function <math> \varphi (\eta )=\dfrac{1}{\eta } </math>. This is called Jeffrey's prior, and is obtained by performing a logarithmic transformation on η. Specifically, since η is always positive, we can assume that ln(η) follows a uniform distribution, ''U''( − ∞, + ∞). Applying Jeffrey's rule [9] which says "in general, an approximate non-informative prior is taken proportional to the square root of Fisher's information", yields <math> \varphi (\eta )=\dfrac{1}{\eta }. </math>

The prior distribution of β, denoted as <math> \varphi (\beta ) </math>, can be selected from the following distributions: normal, lognormal, exponential and uniform. The procedure of performing a Weibull-Bayesian analysis is as follows:

*Collect the times-to-failure data.
*Specify a prior distribution for β (the prior for η is assumed to be 1/η).
*Obtain the posterior from Eqn. (EQNREF WeibBayes ).

In other words, a distribution (the posterior ) is obtained, rather than a point estimate as in classical statistics (i.e., as in the parameter estimation methods described previously in this chapter). Therefore, if a point estimate needs to be reported, a point of the posterior needs to be calculated. Typical points of the posterior distribution used are the mean (expected value) or median. In Weibull++, both options are available and can be chosen from the ''Analysis'' page, under the ''Results As'' area, as shown next.

The expected value of β is obtained by:

::<math> E(\beta )=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\beta \cdot f(\beta ,\eta |Data)d\beta d\eta </math>

Similarly, the expected value of η is obtained by:

::<math> E(\eta )=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\eta \cdot f(\beta ,\eta |Data)d\beta d\eta </math>

The median points are obtained by solving the following equations for <math> \breve{\beta} </math> and <math> \breve{\eta} </math> respectively:

::<math> \int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\breve{\beta}}f(\beta ,\eta |Data)d\beta d\eta =0.5 </math>

and

::<math> \int\nolimits_{0}^{\breve{\eta}}\int\nolimits_{0}^{\infty }f(\beta ,\eta |Data)d\beta d\eta =0.5 </math>

Of course, other points of the posterior distribution can be calculated as well. For example, one may want to calculate the 10th percentile of the joint posterior distribution (w.r.t. one of the parameters). The procedure for obtaining other points of the posterior distribution is similar to the one for obtaining the median values, where instead of 0.5 the percentage of interest is given. This procedure actually provides the confidence bounds on the parameters, which in the Bayesian framework are called ‘‘Credible Bounds‘‘. However, since the engineering interpretation is the same, and to avoid confusion, we refer to them as confidence bounds in this reference and in Weibull++.

== Posterior Distributions for Functions of Parameters ==

As explained in Chapter 3, in Bayesian analysis, all the functions of the parameters are distributed. In other words, a posterior distribution is obtained for functions such as reliability and failure rate, instead of point estimate as in classical statistics. Therefore, in order to obtain a point estimate for these functions, a point on the posterior distributions needs to be calculated. Again, the expected value (mean) or median value are used.

===<math>pdf</math> of the Times-to-Failure ===

The posterior distribution of the failure time is given by:

::<math> f(T|Data)=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }f(T,\beta ,\eta )f(\beta ,\eta |Data)d\eta d\beta </math> EQNREF WeibBayesPDF

where:

::<math> f(T,\beta ,\eta )=\dfrac{\beta }{\eta }\left( \dfrac{T}{\eta }\right) ^{\beta -1}e^{-\left( \dfrac{T}{\eta }\right) ^{\beta }} </math>

For the <math>pdf</math> of the times-to-failure, only the expected value is calculated and reported in Weibull++.

=== Reliability ===

In order to calculate the median value of the reliability function, we first need to obtain posterior of the reliability. Since ''R''(''T'') is a function of β, the density functions of β and ''R''(''T'') have the following relationship:

::<math> \begin{align} f(R|Data,T)dR = & f(\beta |Data)d\beta)\\
= & (\int\nolimits_{0}^{\infty }f(\beta ,\eta |Data)d{\eta}) d{\beta} \\
=& \dfrac{\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }d\beta
\end{align}
</math> EQNREF Rpdf

The median value of the reliability is obtained by solving the following equation w.r.t. <math> \breve{R}: </math>

::<math> \int\nolimits_{0}^{\breve{R}}f(R|Data,T)dR=0.5 </math> EQNREF MedRel

The expected value of the reliability at time is given by:

::<math> R(T|Data)=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }R(T,\beta ,\eta )f(\beta ,\eta |Data)d\eta d\beta </math> where:

::<math> R(T,\beta ,\eta )=e^{-\left( \dfrac{T}{\eta }\right) ^{^{\beta }}} </math>

 

=== Failure Rate ===

The failure rate at time is given by:

::<math> \lambda (T|Data)=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\lambda (T,\beta ,\eta )L(\beta ,\eta )\varphi (\eta )\varphi (\beta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\eta )\varphi (\beta )d\eta d\beta } </math>

where:

::<math> \lambda (T,\beta ,\eta )=\dfrac{\beta }{\eta }\left( \dfrac{T}{\eta }\right) ^{\beta -1} </math>

 

== Note on Calculated Results ==

As mentioned above, in order to obtain point estimates for the parameters of functions of the parameters in Bayesian analysis, the Median or Mean values of the different posterior <math>pdf</math>s are calculated. It is important to note that the Median value is preferable and is the default in Weibull++. This is because the Median value always corresponds to the 50th percentile of the distribution. On the other hand, the Mean is not a fixed point on the distribution, which could cause issues, especially when comparing results across different data sets.

== Confidence Bounds on ''R''(''T'') ==

The confidence bounds calculation under the Weibull-Bayesian analysis is very similar to the Bayesian Confidence Bounds method described in the previous section, with the exception that in the case of the Weibull-Bayesian Analysis the specified prior of β is considered instead of an non-informative prior. The Bayesian one-sided upper bound estimate for ''R''(''T'') is given by:

::<math> \int\nolimits_{0}^{R_{U}(T)}f(R|Data,T)dR=CL </math>

Using Eqns. (EQNREF WeibBayes ) and (EQNREF Rpdf ) the following is obtained:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{T\exp (-\dfrac{\ln (-\ln R_{U})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLRU

Eqn. (EQNREF 1CLRU ) can be solved for ''R''''U''(''T''). The Bayesian one-sided lower bound estimate for <math> \ R(T) </math> is given by:

::<math> \int\nolimits_{0}^{R_{L}(T)}f(R|Data,T)dR=1-CL </math>

Using Eqns. (EQNREF WeibBayes ) and (EQNREF Rpdf ) the following is obtained:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T\exp (-\dfrac{\ln (-\ln R_{L})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=1-CL </math> EQNREF 1CLRL

Eqn. (EQNREF 1CLRL ) can be solved for ''R''''L''(''T''). The Bayesian two-sided bounds estimate for ''R''(''T'') is given by:

::<math> \int\nolimits_{R_{L}(T)}^{R_{U}(T)}f(R|Data,T)dR=CL </math> which is equivalent to:

::<math> \int\nolimits_{0}^{R_{U}(T)}f(R|Data,T)dR=(1+CL)/2 </math>

and

::<math> \int\nolimits_{0}^{R_{L}(T)}f(R|Data,T)dR=(1-CL)/2 </math>

Using the same method for one-sided bounds, ''R''''U''(''T'')and ''R''''L''(''T'') can be computed.

== Confidence Bounds on Time ==

Following the same procedure described for bounds on Reliability, the bounds of time can be calculated, given . The Bayesian one-sided upper bound estimate for ''T''(''R'') is given by:

::<math> \int\nolimits_{0}^{T_{U}(R)}f(T|Data,R)dT=CL </math>

Using Eqns. (EQNREF WeibBayes ) and. (EQNREF WeibBayesPDF ), we obtain:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T_{U}\exp (-\dfrac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLTU

Eqn. (EQNREF 1CLTU ) can be solved for ''T''''U''(''R''). The Bayesian one-sided lower bound estimate for ''T''(''R'') is given by:

::<math> \int\nolimits_{0}^{T_{L}(R)}f(T|Data,R)dT=1-CL </math>

or:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{T_{L}\exp (\dfrac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLTL

Eqn. (EQNREF 1CLTL ) can be solved for ''T''''L''(''R''). The Bayesian two-sided lower bounds estimate for ''T''(''R'') is:

::<math> \int\nolimits_{T_{L}(R)}^{T_{U}(R)}f(T|Data,R)dT=CL </math>

which is equivalent to:

::<math> \int\nolimits_{0}^{T_{U}(R)}f(T|Data,R)dT=(1+CL)/2 </math>

and:

::<math> \int\nolimits_{0}^{T_{L}(R)}f(T|Data,R)dT=(1-CL)/2 </math>

 

=====Example 6=====

A manufacturer has tested prototypes of a modified product. The test was terminated at 2000 hours, with only two failures observed from a sample size of eighteen.

{| border=1 cellspacing=1 align="center"
|-
|Number of State||State of F or S||State End Time
|-
| 1 || F || 1180
|-
| 1 || F || 1842
|-
| 16 || S || 2000
|}

Because of the lack of failure data in the prototype testing, the manufacturer decided to use information gathered from prior tests on this product to increase the confidence in the results of the prototype testing. This decision was made because failure analysis indicated that the failure mode of these two failures is the same as the one observed in previous tests. In other words, it is expected that the shape of the distribution hasn't changed, but hopefully the scale has, indicating longer life. The two-parameter Weibull distribution have been used to model all prior tests results. The list of the estimated β parameter is as follows:

{| border=1 cellspacing=1 align="center"
|-
|Betas Obtained for Similar Mode
|-
| 1.7
|-
| 2.1
|-
| 2.4
|-
|3.1
|-
|3.5
|}

First, in order to fit the data to a Weibull-Bayesian model, a prior distribution for β needs to be determined. Based on the prior tests' β values, the prior distribution for β was found to be a lognormal distribution with μ = 0.9064, σ = 0.3325 (obtained by entering the β values into a Weibull++ ''Standard Folio'' and analyzing it based on the RRX analysis method.)

the test data is entered into a ''Standard Folio'', the Weibull-Bayesian is selected under '' Distribution'' and the β prior distribution is entered after clicking the ''Calculate'' button.

Suppose that the reliability at 3000hr is the metric of interest in this example. This reliability can be obtained using Eqn. (EQNREF MedRel ), resulting in the median value of the posterior of the reliability at 3000hr. Using the ''QCP'', this value is calculated to be 76.97. ( By default Weibull++ returns the median values of the posterior distribution. )

The posterior <math>pdf</math> of the reliability function at 3000hrs can be obtained using Eqn. (EQNREF Rpdf ). In Figure 6-10 the posterior <math>pdf</math> of the reliability at 3000hrs is plotted, with the corresponding median value as well as the 10th percentile value shown. The 10th percentile constitutes the 90 Lower 1-Sided bound on the reliability at 3000hrs, which is calculated to be 50.77.

FIGURE HERE

Notice that the <math>pdf</math> plotted in Fig. 6-10 is of the reliability at 3000hrs, and not the <math>pdf</math> of the times-to-failure data. The <math>pdf</math> of the times-to-failure data can be obtained using Eqn. (EQNREF WeibBayesPDF ) and plotted using Weibull++, as shown next:

FIGURE HERE

{{RS Copyright}}

[[Category:Life_Data_Analysis_Reference]]

Fisher Matrix Confidence Bounds

2011-06-30T18:59:37Z

Steve Sharp: /* Confidence Bounds on Time (Type 1) */

This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [30] and Lloyd and Lipow [24]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds.
===Approximate Estimates of the Mean and Variance of a Function===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e. reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G</math> is presented next.

====Single Parameter Case====
For simplicity, consider a one-parameter distribution represented by a general function, <math>G,</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda </math>: <math>G(\lambda )=1/\lambda =\mu </math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)</math> can be found by:

::<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)</math>

where <math>G(\theta )</math> is some function of <math>\theta </math>, such as the reliability function, and <math>\theta </math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta </math> as <math>n\to \infty </math> . The term <math>O\left( \tfrac{1}{n} \right)</math> is a function of <math>n</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},</math> as <math>n\to \infty .</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}</math> and <math>G(x)=1/x</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}</math>. Thus as <math>n\to \infty </math>, <math>E(G(\widehat{\theta }))=\mu </math> where <math>\mu </math> and <math>\sigma </math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)</math> can then be estimated by:

::<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)</math>

====Two-Parameter Case====

Consider a Weibull distribution with two parameters <math>\beta </math> and <math>\eta </math>. For a given value of <math>T</math>, <math>R(T)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{T}{\eta } \right)}^{\beta }}}}</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)</math>, that:

::<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)</math>

and:

::<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Var(\widehat{\theta_{2}})\\

& +2{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\

& +O(\frac{1}{n^{\tfrac{3}{2}}})
\end{align}

</math>

Note that the derivatives of Eqn. (var) are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{1}},</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.</math>
Parameter Variance and Covariance Determination
The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

::<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}</math>

In the equation above, the first summation is for complete data, the second summation is for right censored data, and the third summation is for interval or left censored data. For more information on these data types, see Chapter 4.
Then the Fisher information matrix is given by:

::<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]</math>

The subscript <math>0</math> indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},</math> the true values of the parameters.
So for a sample of <math>N</math> units where <math>R</math> units have failed, <math>S</math> have been suspended, and <math>P</math> have failed within a time interval, and <math>N=R+M+P,</math> one could obtain the sample local information matrix by:

::<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}</math>

Substituting in the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}</math> and <math>{{\widehat{\theta }}_{2}}</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

::<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}</math>

Then the variance of a function (<math>Var(G)</math>) can be estimated using Eqn. (var). Values for the variance and covariance of the parameters are obtained from Eqn. (Fisher2).
Once they have been obtained, the approximate confidence bounds on the function are given as:

::<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha }}</math> next.

====Approximate Confidence Intervals on the Parameters====
In general, MLE estimates of the parameters are asymptotically normal, meaning for large sample sizes that a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }</math> is the MLE estimator for <math>\theta </math>, in the case of a single parameter distribution, estimated from a large sample of <math>n</math> units and if:

::<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}</math>

then using the normal distribution of <math>z\ \ :</math>

::<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

for large <math>n</math>. We now place confidence bounds on <math>\theta ,</math> at some confidence level <math>\delta </math>, bounded by the two end points <math>{{C}_{1}}</math> and <math>{{C}_{2}}</math> where:

::<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta </math>

From Eqn. (e729):

::<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta </math>

where <math>{{K}_{\alpha }}</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)</math>

Now by simplifying Eqn. (e731), one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta ,</math> at a confidence level <math>\delta ,</math> or:

::<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)</math>

The upper one-sided bounds are given by:

::<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

while the lower one-sided bounds are given by:

::<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

If <math>\widehat{\theta }</math> must be positive, then <math>\ln \widehat{\theta }</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta </math>, then become:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}</math>

The one-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta ,</math> can be found from:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [24] further elaborate on this procedure.

====Confidence Bounds on Time (Type 1)====
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e. y-ordinate or unreliability, <math>Q=1-R)</math> is determined by solving the unreliability function for the time, <math>T</math>, or:

::<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cb). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Note that the procedure is slightly more complicated for distributions with more than one parameter.

====Confidence Bounds on Reliability (Type 2)====
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

::<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cbr). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Once again, the procedure is more complicated for distributions with more than one parameter.

===Beta Binomial Confidence Bounds===
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see Chapter 4). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution.
Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z</math> (rank for the <math>{{j}^{th}}</math> failure):

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where <math>N</math> is the sample size and <math>j</math> is the order number.
The median rank was obtained by solving the following equation for <math>Z</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

The same methodology can then be repeated by changing <math>P</math> for <math>0.50</math> <math>(50%)</math> to our desired confidence level. For <math>P=90%</math> one would formulate the equation as

::<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

Keep in mind that one must be careful to select the appropriate values for <math>P</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P=0.1</math> and once with <math>P=0.9</math>) in order to place the bounds around 80% of the population.
Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.
This non-parametric methodology is only used by Weibull++ when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [20]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

===Likelihood Ratio Confidence Bounds===
====Introduction====
A third method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.
Likelihood ratio confidence bounds are based on the equation:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}</math>

where:
::#<math>L(\theta )</math> is the likelihood function for the unknown parameter vector <math>\theta </math>
::#<math>L(\widehat{\theta })</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }</math>
::#<math>\chi _{\alpha ;k}^{2}</math> is the chi-squared statistic with probability <math>\alpha </math> and <math>k</math> degrees of freedom, where <math>k</math> is the number of quantities jointly estimated
If <math>\delta </math> is the confidence level, then <math>\alpha =\delta </math> for two-sided bounds and <math>\alpha =(2\delta -1)</math> for one-sided. Recall from Chapter 3 that if <math>x</math> is a continuous random variable with <math>pdf</math>:

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>,

where <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,{{x}_{R}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,R</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L.</math> These are represented by the <math>L(\widehat{\theta })</math> term in the denominator of the ratio in Eqn. (lratio1). Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }</math> have been calculated using MLE methods, the only unknown term in Eqn. (lratio1) is the <math>L(\theta )</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta </math> that satisfy Eqn. (lratio1). For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy Eqn. (lratio1). The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta ;</math> but at a given value of <math>\delta </math> there is only a certain region of values for <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> for which Eqn. (lratio1) holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of Eqn. (lratio1).

====Note on Contour Plots in Weibull++====
Contour plots can be used for comparing data sets. Consider two data sets, e.g. old and new design where the engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in a multiple plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e. the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If there is an overlap between the two 95% contours, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. Chapter 12 discusses comparing data sets.

====Confidence Bounds on the Parameters====
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}</math>

The task now becomes to find the values of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> so that the equality in Eqn. (lratio3) is satisfied. Unfortunately, there is no closed-form solution, thus these values must be arrived at numerically. One method of doing this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

=====Example 1=====
Five units were put on a reliability test and experienced failures at 10, 20, 30, 40, and 50 hours. Assuming a Weibull distribution, the MLE parameter estimates are calculated to be <math>\widehat{\beta }=2.2938</math> and <math>\widehat{\eta }=33.9428.</math> Calculate the 90% two-sided confidence bounds on these parameters using the likelihood ratio method.

=====Solution to Example 1=====
The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\widehat{\beta },\widehat{\eta })=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\widehat{\beta }}{\widehat{\eta }}\cdot {{\left( \frac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }-1}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{2.2938}{33.9428}\cdot {{\left( \frac{{{x}_{i}}}{33.9428} \right)}^{1.2938}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{33.9428} \right)}^{2.2938}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & 1.714714\times {{10}^{-9}}
\end{align}</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We then substitute this information into the equation:

::<math>\begin{align}
L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,\eta )-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,\eta )-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

The next step is to find the set of values of <math>\beta </math> and <math>\eta </math> that satisfy this equation, or find the values of <math>\beta </math> and <math>\eta </math> such that <math>L(\beta ,\eta )=4.432926\cdot {{10}^{-10}}.</math>

The solution is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>\eta </math>, and vice versa. The following table gives values of <math>\beta </math> based on given values of <math>\eta </math>.

These data are represented graphically in the following contour plot:

(Note that this plot is generated with degrees of freedom <math>k=1</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2</math>, for use in comparing both parameters simultaneously.) As can be determined from the table, the lowest calculated value for <math>\beta </math> is 1.142, while the highest is 3.950. These represent the two-sided 90% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>\eta </math> below 23 or above 50, these can be considered the 90% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>\eta </math>, we can perform the same procedure as before, but finding the two values of <math>\eta </math> that correspond with a given value of <math>\beta .</math> Using this method, we find that the 90% confidence limits on <math>\eta </math> are 22.474 and 49.967, which are close to the initial estimates of 23 and 50.
Note that the points where <math>\beta </math> are maximized and minimized do not necessarily correspond with the points where <math>\eta </math> are maximized and minimized. This is due to the fact that the contour plot is not symmetrical, so that the parameters will have their extremes at different points.

====Confidence Bounds on Time (Type 1)====
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.
=====Example 2=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the time estimate for a reliability of 50%. The ML estimate for the time at which <math>R(t)=50%</math> is 28.930.
=====Solution to Example 2=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the time estimate of 28.930. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>t</math> and <math>\beta .</math> This is accomplished by using a form of the Weibull reliability equation, <math>R={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}.</math> This can be rearranged in terms of <math>\eta </math>, with <math>R</math> being considered a known variable or:

::<math>\eta =\frac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}}</math>

This can then be substituted into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>t</math> and <math>\beta </math> or:

::<math>\begin{align}
& L(\beta ,t)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,t)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
& \\
L(\beta ,t)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

Note that the likelihood value for <math>L(\widehat{\beta },\widehat{\eta })</math> is the same as it was for Example 1. This is because we are dealing with the same data and parameter estimates or, in other words, the maximum value of the likelihood function did not change. It now remains to find the values of <math>\beta </math> and <math>t</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>t</math>. The following table gives the values of <math>t</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:
As can be determined from the table, the lowest calculated value for <math>t</math> is 17.389, while the highest is 41.714. These represent the 90% two-sided confidence limits on the time at which reliability is equal to 50%.

====Confidence Bounds on Reliability (Type 2)====
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta </math> and <math>R</math>. The likelihood function is once again altered in the same way as before, only now <math>R</math> is considered to be a parameter instead of <math>t</math>, since the value of <math>t</math> must be specified in advance. Once again, this process is best illustrated with an example.

=====Example 3=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the reliability estimate for <math>t=45</math>. The ML estimate for the reliability at <math>t=45</math> is 14.816%.

=====Solution to Example 3=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the reliability estimate of 14.816%. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>R</math> and <math>\beta .</math> This is again accomplished by substituting the Weibull reliability equation into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>R</math> and <math>\beta </math>:

::<math>\begin{align}
& L(\beta ,R)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,R)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,R)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

It now remains to find the values of <math>\beta </math> and <math>R</math> that satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>R</math>. The following table gives the values of <math>R</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:

As can be determined from the table, the lowest calculated value for <math>R</math> is 2.38%, while the highest is 44.26%. These represent the 90% two-sided confidence limits on the reliability at <math>t=45</math>.

===Bayesian Confidence Bounds===
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in Chapter 3.
Bayesian confidence bounds are derived from Bayes rule, which states that:

::<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
::#<math>f(\theta |Data)</math> is the <math>posterior</math> <math>pdf</math> of <math>\theta </math>
::#<math>\theta </math> is the parameter vector of the chosen distribution (i.e. Weibull, lognormal, etc.)
::#<math>L(\bullet )</math> is the likelihood function
::#<math>\varphi (\theta )</math> is the <math>prior</math> <math>pdf</math> of the parameter vector <math>\theta </math>
::#<math>\varsigma </math> is the range of <math>\theta </math>.
In other words, the prior knowledge is provided in the form of the prior <math>pdf</math> of the parameters, which in turn is combined with the sample data in order to obtain the posterior <math>pdf.</math> Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to Chapter 3). It can be seen from Eqn. (BayesRule) that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta }_{1}}</math>. Given a set of sample data, and a prior distribution for <math>{{\theta }_{1}},</math> <math>\varphi ({{\theta }_{1}}),</math> Eqn. (BayesRule) can be written as:

::<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

In other words, we now have the distribution of <math>{{\theta }_{1}}</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta }_{1}}</math> is less than or equal to a value <math>x,</math> <math>P({{\theta }_{1}}\le x)</math> can be obtained by integrating Eqn. (BayesEX), or:

::<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}</math>

Eqn. (IntBayes) essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)</math> is the confidence level and <math>x</math> is the confidence bound. Substituting Eqn. (BayesEX) into Eqn. (IntBayes) yields:

::<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

The only question at this point is what do we use as a prior distribution of <math>{{\theta }_{1}}.</math>. For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform.
Eqn. (BayesCLEX) can be generalized for any distribution having a vector of parameters <math>\theta ,</math> yielding the general equation for calculating Bayesian confidence bounds:

::<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
::#<math>CL</math> is confidence level
::#<math>\theta </math> is the parameter vector
::#<math>L(\bullet )</math> is the likelihood function
::#<math>\varphi (\theta )</math> is the prior <math>pdf</math> of the parameter vector <math>\theta </math>
::#<math>\varsigma </math> is the range of <math>\theta </math>
::#<math>\xi </math> is the range in which <math>\theta </math> changes from <math>\Psi (T,R)</math> till <math>{\theta }'s</math> maximum value or from <math>{\theta }'s</math> minimum value till <math>\Psi (T,R)</math>
::#<math>\Psi (T,R)</math> is function such that if <math>T</math> is given then the bounds are calculated for <math>R</math> and if <math>R</math> is given, then he bounds are calculated for <math>T</math>.
If <math>T</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>R</math> are calculated.
If <math>R</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>T</math> are calculated.
====Confidence Bounds on Time (Type 1)====
For a given failure time distribution and a given reliability <math>R</math>, <math>T(R)</math> is a function of <math>R</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For the two-parameter Weibull distribution:

::<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)</math> is:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

where <math>f(T|Data,R)</math> is the posterior distribution of Time <math>T.</math>
Using Eqn. (T bayes), we have the following:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})</math>

Eqn. (cl) can be rewritten in terms of <math>\eta </math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))</math>

From Eqns. (IntBayes), (BayesCLEX) and (BayesCL), by assuming the priors of <math>\beta </math> and <math>\eta </math> are independent, we then obtain the following relationship:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl2) can be solved for <math>{{T}_{U}}(R)</math>, where:
::#<math>CL</math> is confidence level,
::#<math>\varphi (\beta )</math> is the prior <math>pdf</math> of the parameter <math>\beta </math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.</math>
::#<math>\varphi (\eta )</math> is the prior <math>pdf</math> of the parameter <math>\eta .</math>. For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.</math>
::#<math>L(\bullet )</math> is the likelihood function.
The same method can be used to get the one-sided lower bound of <math>T(R)</math> from:

::<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl5) can be solved to get <math>{{T}_{L}}(R)</math>.
The Bayesian two-sided bounds estimate for <math>T(R)</math> is:

::<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

which is equivalent to:

::<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

and:

::<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)</math> and <math>{{T}_{L}}(R)</math> can be solved.

====Confidence Bounds on Reliability (Type 2)====
For a given failure time distribution and a given time <math>T</math>, <math>R(T)</math> is a function of <math>T</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For example, for two parameter Weibull distribution:

::<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})</math>

The Bayesian one-sided upper bound estimate for <math>R(T)</math> is:

::<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

Similar with the bounds on Time, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl3) can be solved to get <math>{{R}_{U}}(T)</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

::<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR</math>

Using the posterior distribution, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl4) can be solved to get <math>{{R}_{L}}(T)</math>.
The Bayesian two-sided bounds estimate for <math>R(T)</math> is:

::<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

which is equivalent to:

::<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2</math>

and

::<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)</math> and <math>{{R}_{L}}(T)</math> can be solved.

===Simulation Based Bounds===
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds.
SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

=====Example 4=====
The purpose of this example is to determine the best parameter estimation method for a sample of ten units following a Weibull distribution with <math>\beta =2</math> and <math>\eta =100</math> and with complete time-to-failure data for each unit (i.e. no censoring). The number of generated data sets is set to 10,000. The SimuMatic inputs are shown next.

The parameters are estimated using RRX, RRY and MLE. The plotted results generated by SimuMatic are shown next.

Using RRX:

Using RRY:

Using MLE:

The results clearly demonstrate that the median RRX estimate provides the least deviation from the truth for this sample size and data type. However, the MLE outputs are grouped more closely together, as evidenced by the bounds. The previous figures also show the simulation-based bounds, as well as the expected variation due to sampling error.
This experiment can be repeated in SimuMatic using multiple censoring schemes (including Type I and Type II right censoring as well as random censoring) with various distributions. Multiple experiments can be performed with this utility to evaluate assumptions about the appropriate parameter estimation method to use for data sets.

Fisher Matrix Confidence Bounds

2011-06-30T18:59:08Z

Steve Sharp: /* Bayesian Confidence Bounds */

This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [30] and Lloyd and Lipow [24]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds.
===Approximate Estimates of the Mean and Variance of a Function===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e. reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G</math> is presented next.

====Single Parameter Case====
For simplicity, consider a one-parameter distribution represented by a general function, <math>G,</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda </math>: <math>G(\lambda )=1/\lambda =\mu </math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)</math> can be found by:

::<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)</math>

where <math>G(\theta )</math> is some function of <math>\theta </math>, such as the reliability function, and <math>\theta </math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta </math> as <math>n\to \infty </math> . The term <math>O\left( \tfrac{1}{n} \right)</math> is a function of <math>n</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},</math> as <math>n\to \infty .</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}</math> and <math>G(x)=1/x</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}</math>. Thus as <math>n\to \infty </math>, <math>E(G(\widehat{\theta }))=\mu </math> where <math>\mu </math> and <math>\sigma </math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)</math> can then be estimated by:

::<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)</math>

====Two-Parameter Case====

Consider a Weibull distribution with two parameters <math>\beta </math> and <math>\eta </math>. For a given value of <math>T</math>, <math>R(T)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{T}{\eta } \right)}^{\beta }}}}</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)</math>, that:

::<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)</math>

and:

::<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Var(\widehat{\theta_{2}})\\

& +2{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\

& +O(\frac{1}{n^{\tfrac{3}{2}}})
\end{align}

</math>

Note that the derivatives of Eqn. (var) are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{1}},</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.</math>
Parameter Variance and Covariance Determination
The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

::<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}</math>

In the equation above, the first summation is for complete data, the second summation is for right censored data, and the third summation is for interval or left censored data. For more information on these data types, see Chapter 4.
Then the Fisher information matrix is given by:

::<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]</math>

The subscript <math>0</math> indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},</math> the true values of the parameters.
So for a sample of <math>N</math> units where <math>R</math> units have failed, <math>S</math> have been suspended, and <math>P</math> have failed within a time interval, and <math>N=R+M+P,</math> one could obtain the sample local information matrix by:

::<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}</math>

Substituting in the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}</math> and <math>{{\widehat{\theta }}_{2}}</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

::<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}</math>

Then the variance of a function (<math>Var(G)</math>) can be estimated using Eqn. (var). Values for the variance and covariance of the parameters are obtained from Eqn. (Fisher2).
Once they have been obtained, the approximate confidence bounds on the function are given as:

::<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha }}</math> next.

====Approximate Confidence Intervals on the Parameters====
In general, MLE estimates of the parameters are asymptotically normal, meaning for large sample sizes that a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }</math> is the MLE estimator for <math>\theta </math>, in the case of a single parameter distribution, estimated from a large sample of <math>n</math> units and if:

::<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}</math>

then using the normal distribution of <math>z\ \ :</math>

::<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

for large <math>n</math>. We now place confidence bounds on <math>\theta ,</math> at some confidence level <math>\delta </math>, bounded by the two end points <math>{{C}_{1}}</math> and <math>{{C}_{2}}</math> where:

::<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta </math>

From Eqn. (e729):

::<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta </math>

where <math>{{K}_{\alpha }}</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)</math>

Now by simplifying Eqn. (e731), one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta ,</math> at a confidence level <math>\delta ,</math> or:

::<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)</math>

The upper one-sided bounds are given by:

::<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

while the lower one-sided bounds are given by:

::<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

If <math>\widehat{\theta }</math> must be positive, then <math>\ln \widehat{\theta }</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta </math>, then become:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}</math>

The one-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta ,</math> can be found from:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [24] further elaborate on this procedure.

====Confidence Bounds on Time (Type 1)====
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e. y-ordinate or unreliability, <math>Q=1-R)</math> is determined by solving the unreliability function for the time, <math>T</math>, or:

::<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cb). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Note that the procedure is slightly more complicated for distributions with more than one parameter.

====Confidence Bounds on Reliability (Type 2)====
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

::<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cbr). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Once again, the procedure is more complicated for distributions with more than one parameter.

===Beta Binomial Confidence Bounds===
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see Chapter 4). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution.
Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z</math> (rank for the <math>{{j}^{th}}</math> failure):

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where <math>N</math> is the sample size and <math>j</math> is the order number.
The median rank was obtained by solving the following equation for <math>Z</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

The same methodology can then be repeated by changing <math>P</math> for <math>0.50</math> <math>(50%)</math> to our desired confidence level. For <math>P=90%</math> one would formulate the equation as

::<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

Keep in mind that one must be careful to select the appropriate values for <math>P</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P=0.1</math> and once with <math>P=0.9</math>) in order to place the bounds around 80% of the population.
Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.
This non-parametric methodology is only used by Weibull++ when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [20]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

===Likelihood Ratio Confidence Bounds===
====Introduction====
A third method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.
Likelihood ratio confidence bounds are based on the equation:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}</math>

where:
::#<math>L(\theta )</math> is the likelihood function for the unknown parameter vector <math>\theta </math>
::#<math>L(\widehat{\theta })</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }</math>
::#<math>\chi _{\alpha ;k}^{2}</math> is the chi-squared statistic with probability <math>\alpha </math> and <math>k</math> degrees of freedom, where <math>k</math> is the number of quantities jointly estimated
If <math>\delta </math> is the confidence level, then <math>\alpha =\delta </math> for two-sided bounds and <math>\alpha =(2\delta -1)</math> for one-sided. Recall from Chapter 3 that if <math>x</math> is a continuous random variable with <math>pdf</math>:

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>,

where <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,{{x}_{R}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,R</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L.</math> These are represented by the <math>L(\widehat{\theta })</math> term in the denominator of the ratio in Eqn. (lratio1). Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }</math> have been calculated using MLE methods, the only unknown term in Eqn. (lratio1) is the <math>L(\theta )</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta </math> that satisfy Eqn. (lratio1). For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy Eqn. (lratio1). The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta ;</math> but at a given value of <math>\delta </math> there is only a certain region of values for <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> for which Eqn. (lratio1) holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of Eqn. (lratio1).

====Note on Contour Plots in Weibull++====
Contour plots can be used for comparing data sets. Consider two data sets, e.g. old and new design where the engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in a multiple plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e. the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If there is an overlap between the two 95% contours, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. Chapter 12 discusses comparing data sets.

====Confidence Bounds on the Parameters====
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}</math>

The task now becomes to find the values of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> so that the equality in Eqn. (lratio3) is satisfied. Unfortunately, there is no closed-form solution, thus these values must be arrived at numerically. One method of doing this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

=====Example 1=====
Five units were put on a reliability test and experienced failures at 10, 20, 30, 40, and 50 hours. Assuming a Weibull distribution, the MLE parameter estimates are calculated to be <math>\widehat{\beta }=2.2938</math> and <math>\widehat{\eta }=33.9428.</math> Calculate the 90% two-sided confidence bounds on these parameters using the likelihood ratio method.

=====Solution to Example 1=====
The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\widehat{\beta },\widehat{\eta })=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\widehat{\beta }}{\widehat{\eta }}\cdot {{\left( \frac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }-1}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{2.2938}{33.9428}\cdot {{\left( \frac{{{x}_{i}}}{33.9428} \right)}^{1.2938}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{33.9428} \right)}^{2.2938}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & 1.714714\times {{10}^{-9}}
\end{align}</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We then substitute this information into the equation:

::<math>\begin{align}
L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,\eta )-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,\eta )-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

The next step is to find the set of values of <math>\beta </math> and <math>\eta </math> that satisfy this equation, or find the values of <math>\beta </math> and <math>\eta </math> such that <math>L(\beta ,\eta )=4.432926\cdot {{10}^{-10}}.</math>

The solution is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>\eta </math>, and vice versa. The following table gives values of <math>\beta </math> based on given values of <math>\eta </math>.

These data are represented graphically in the following contour plot:

(Note that this plot is generated with degrees of freedom <math>k=1</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2</math>, for use in comparing both parameters simultaneously.) As can be determined from the table, the lowest calculated value for <math>\beta </math> is 1.142, while the highest is 3.950. These represent the two-sided 90% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>\eta </math> below 23 or above 50, these can be considered the 90% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>\eta </math>, we can perform the same procedure as before, but finding the two values of <math>\eta </math> that correspond with a given value of <math>\beta .</math> Using this method, we find that the 90% confidence limits on <math>\eta </math> are 22.474 and 49.967, which are close to the initial estimates of 23 and 50.
Note that the points where <math>\beta </math> are maximized and minimized do not necessarily correspond with the points where <math>\eta </math> are maximized and minimized. This is due to the fact that the contour plot is not symmetrical, so that the parameters will have their extremes at different points.

====Confidence Bounds on Time (Type 1)====
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.
=====Example 2=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the time estimate for a reliability of 50%. The ML estimate for the time at which <math>R(t)=50%</math> is 28.930.
=====Solution to Example 2=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the time estimate of 28.930. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>t</math> and <math>\beta .</math> This is accomplished by using a form of the Weibull reliability equation, <math>R={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}.</math> This can be rearranged in terms of <math>\eta </math>, with <math>R</math> being considered a known variable or:

::<math>\eta =\frac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}}</math>

This can then be substituted into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>t</math> and <math>\beta </math> or:

::<math>\begin{align}
& L(\beta ,t)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,t)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
& \\
L(\beta ,t)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

Note that the likelihood value for <math>L(\widehat{\beta },\widehat{\eta })</math> is the same as it was for Example 1. This is because we are dealing with the same data and parameter estimates or, in other words, the maximum value of the likelihood function did not change. It now remains to find the values of <math>\beta </math> and <math>t</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>t</math>. The following table gives the values of <math>t</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:
As can be determined from the table, the lowest calculated value for <math>t</math> is 17.389, while the highest is 41.714. These represent the 90% two-sided confidence limits on the time at which reliability is equal to 50%.

====Confidence Bounds on Reliability (Type 2)====
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta </math> and <math>R</math>. The likelihood function is once again altered in the same way as before, only now <math>R</math> is considered to be a parameter instead of <math>t</math>, since the value of <math>t</math> must be specified in advance. Once again, this process is best illustrated with an example.

=====Example 3=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the reliability estimate for <math>t=45</math>. The ML estimate for the reliability at <math>t=45</math> is 14.816%.

=====Solution to Example 3=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the reliability estimate of 14.816%. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>R</math> and <math>\beta .</math> This is again accomplished by substituting the Weibull reliability equation into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>R</math> and <math>\beta </math>:

::<math>\begin{align}
& L(\beta ,R)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,R)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,R)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

It now remains to find the values of <math>\beta </math> and <math>R</math> that satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>R</math>. The following table gives the values of <math>R</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:

As can be determined from the table, the lowest calculated value for <math>R</math> is 2.38%, while the highest is 44.26%. These represent the 90% two-sided confidence limits on the reliability at <math>t=45</math>.

===Bayesian Confidence Bounds===
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in Chapter 3.
Bayesian confidence bounds are derived from Bayes rule, which states that:

::<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
::#<math>f(\theta |Data)</math> is the <math>posterior</math> <math>pdf</math> of <math>\theta </math>
::#<math>\theta </math> is the parameter vector of the chosen distribution (i.e. Weibull, lognormal, etc.)
::#<math>L(\bullet )</math> is the likelihood function
::#<math>\varphi (\theta )</math> is the <math>prior</math> <math>pdf</math> of the parameter vector <math>\theta </math>
::#<math>\varsigma </math> is the range of <math>\theta </math>.
In other words, the prior knowledge is provided in the form of the prior <math>pdf</math> of the parameters, which in turn is combined with the sample data in order to obtain the posterior <math>pdf.</math> Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to Chapter 3). It can be seen from Eqn. (BayesRule) that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta }_{1}}</math>. Given a set of sample data, and a prior distribution for <math>{{\theta }_{1}},</math> <math>\varphi ({{\theta }_{1}}),</math> Eqn. (BayesRule) can be written as:

::<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

In other words, we now have the distribution of <math>{{\theta }_{1}}</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta }_{1}}</math> is less than or equal to a value <math>x,</math> <math>P({{\theta }_{1}}\le x)</math> can be obtained by integrating Eqn. (BayesEX), or:

::<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}</math>

Eqn. (IntBayes) essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)</math> is the confidence level and <math>x</math> is the confidence bound. Substituting Eqn. (BayesEX) into Eqn. (IntBayes) yields:

::<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

The only question at this point is what do we use as a prior distribution of <math>{{\theta }_{1}}.</math>. For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform.
Eqn. (BayesCLEX) can be generalized for any distribution having a vector of parameters <math>\theta ,</math> yielding the general equation for calculating Bayesian confidence bounds:

::<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
::#<math>CL</math> is confidence level
::#<math>\theta </math> is the parameter vector
::#<math>L(\bullet )</math> is the likelihood function
::#<math>\varphi (\theta )</math> is the prior <math>pdf</math> of the parameter vector <math>\theta </math>
::#<math>\varsigma </math> is the range of <math>\theta </math>
::#<math>\xi </math> is the range in which <math>\theta </math> changes from <math>\Psi (T,R)</math> till <math>{\theta }'s</math> maximum value or from <math>{\theta }'s</math> minimum value till <math>\Psi (T,R)</math>
::#<math>\Psi (T,R)</math> is function such that if <math>T</math> is given then the bounds are calculated for <math>R</math> and if <math>R</math> is given, then he bounds are calculated for <math>T</math>.
If <math>T</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>R</math> are calculated.
If <math>R</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>T</math> are calculated.
====Confidence Bounds on Time (Type 1)====
For a given failure time distribution and a given reliability <math>R</math>, <math>T(R)</math> is a function of <math>R</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For the two-parameter Weibull distribution:

::<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)</math> is:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

where <math>f(T|Data,R)</math> is the posterior distribution of Time <math>T.</math>
Using Eqn. (T bayes), we have the following:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})</math>

Eqn. (cl) can be rewritten in terms of <math>\eta </math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))</math>

From Eqns. (IntBayes), (BayesCLEX) and (BayesCL), by assuming the priors of <math>\beta </math> and <math>\eta </math> are independent, we then obtain the following relationship:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl2) can be solved for <math>{{T}_{U}}(R)</math>, where:
#<math>CL</math> is confidence level,
#<math>\varphi (\beta )</math> is the prior <math>pdf</math> of the parameter <math>\beta </math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.</math>
#<math>\varphi (\eta )</math> is the prior <math>pdf</math> of the parameter <math>\eta .</math>. For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.</math>
#<math>L(\bullet )</math> is the likelihood function.
The same method can be used to get the one-sided lower bound of <math>T(R)</math> from:

::<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl5) can be solved to get <math>{{T}_{L}}(R)</math>.
The Bayesian two-sided bounds estimate for <math>T(R)</math> is:

::<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

which is equivalent to:

::<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

and:

::<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)</math> and <math>{{T}_{L}}(R)</math> can be solved.
====Confidence Bounds on Reliability (Type 2)====
For a given failure time distribution and a given time <math>T</math>, <math>R(T)</math> is a function of <math>T</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For example, for two parameter Weibull distribution:

::<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})</math>

The Bayesian one-sided upper bound estimate for <math>R(T)</math> is:

::<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

Similar with the bounds on Time, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl3) can be solved to get <math>{{R}_{U}}(T)</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

::<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR</math>

Using the posterior distribution, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl4) can be solved to get <math>{{R}_{L}}(T)</math>.
The Bayesian two-sided bounds estimate for <math>R(T)</math> is:

::<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

which is equivalent to:

::<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2</math>

and

::<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)</math> and <math>{{R}_{L}}(T)</math> can be solved.

===Simulation Based Bounds===
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds.
SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

=====Example 4=====
The purpose of this example is to determine the best parameter estimation method for a sample of ten units following a Weibull distribution with <math>\beta =2</math> and <math>\eta =100</math> and with complete time-to-failure data for each unit (i.e. no censoring). The number of generated data sets is set to 10,000. The SimuMatic inputs are shown next.

The parameters are estimated using RRX, RRY and MLE. The plotted results generated by SimuMatic are shown next.

Using RRX:

Using RRY:

Using MLE:

The results clearly demonstrate that the median RRX estimate provides the least deviation from the truth for this sample size and data type. However, the MLE outputs are grouped more closely together, as evidenced by the bounds. The previous figures also show the simulation-based bounds, as well as the expected variation due to sampling error.
This experiment can be repeated in SimuMatic using multiple censoring schemes (including Type I and Type II right censoring as well as random censoring) with various distributions. Multiple experiments can be performed with this utility to evaluate assumptions about the appropriate parameter estimation method to use for data sets.

Fisher Matrix Confidence Bounds

2011-06-30T18:58:30Z

Steve Sharp: /* Bayesian Confidence Bounds */

This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [30] and Lloyd and Lipow [24]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds.
===Approximate Estimates of the Mean and Variance of a Function===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e. reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G</math> is presented next.

====Single Parameter Case====
For simplicity, consider a one-parameter distribution represented by a general function, <math>G,</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda </math>: <math>G(\lambda )=1/\lambda =\mu </math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)</math> can be found by:

::<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)</math>

where <math>G(\theta )</math> is some function of <math>\theta </math>, such as the reliability function, and <math>\theta </math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta </math> as <math>n\to \infty </math> . The term <math>O\left( \tfrac{1}{n} \right)</math> is a function of <math>n</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},</math> as <math>n\to \infty .</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}</math> and <math>G(x)=1/x</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}</math>. Thus as <math>n\to \infty </math>, <math>E(G(\widehat{\theta }))=\mu </math> where <math>\mu </math> and <math>\sigma </math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)</math> can then be estimated by:

::<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)</math>

====Two-Parameter Case====

Consider a Weibull distribution with two parameters <math>\beta </math> and <math>\eta </math>. For a given value of <math>T</math>, <math>R(T)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{T}{\eta } \right)}^{\beta }}}}</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)</math>, that:

::<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)</math>

and:

::<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Var(\widehat{\theta_{2}})\\

& +2{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\

& +O(\frac{1}{n^{\tfrac{3}{2}}})
\end{align}

</math>

Note that the derivatives of Eqn. (var) are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{1}},</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.</math>
Parameter Variance and Covariance Determination
The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

::<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}</math>

In the equation above, the first summation is for complete data, the second summation is for right censored data, and the third summation is for interval or left censored data. For more information on these data types, see Chapter 4.
Then the Fisher information matrix is given by:

::<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]</math>

The subscript <math>0</math> indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},</math> the true values of the parameters.
So for a sample of <math>N</math> units where <math>R</math> units have failed, <math>S</math> have been suspended, and <math>P</math> have failed within a time interval, and <math>N=R+M+P,</math> one could obtain the sample local information matrix by:

::<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}</math>

Substituting in the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}</math> and <math>{{\widehat{\theta }}_{2}}</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

::<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}</math>

Then the variance of a function (<math>Var(G)</math>) can be estimated using Eqn. (var). Values for the variance and covariance of the parameters are obtained from Eqn. (Fisher2).
Once they have been obtained, the approximate confidence bounds on the function are given as:

::<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha }}</math> next.

====Approximate Confidence Intervals on the Parameters====
In general, MLE estimates of the parameters are asymptotically normal, meaning for large sample sizes that a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }</math> is the MLE estimator for <math>\theta </math>, in the case of a single parameter distribution, estimated from a large sample of <math>n</math> units and if:

::<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}</math>

then using the normal distribution of <math>z\ \ :</math>

::<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

for large <math>n</math>. We now place confidence bounds on <math>\theta ,</math> at some confidence level <math>\delta </math>, bounded by the two end points <math>{{C}_{1}}</math> and <math>{{C}_{2}}</math> where:

::<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta </math>

From Eqn. (e729):

::<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta </math>

where <math>{{K}_{\alpha }}</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)</math>

Now by simplifying Eqn. (e731), one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta ,</math> at a confidence level <math>\delta ,</math> or:

::<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)</math>

The upper one-sided bounds are given by:

::<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

while the lower one-sided bounds are given by:

::<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

If <math>\widehat{\theta }</math> must be positive, then <math>\ln \widehat{\theta }</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta </math>, then become:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}</math>

The one-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta ,</math> can be found from:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [24] further elaborate on this procedure.

====Confidence Bounds on Time (Type 1)====
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e. y-ordinate or unreliability, <math>Q=1-R)</math> is determined by solving the unreliability function for the time, <math>T</math>, or:

::<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cb). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Note that the procedure is slightly more complicated for distributions with more than one parameter.

====Confidence Bounds on Reliability (Type 2)====
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

::<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cbr). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Once again, the procedure is more complicated for distributions with more than one parameter.

===Beta Binomial Confidence Bounds===
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see Chapter 4). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution.
Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z</math> (rank for the <math>{{j}^{th}}</math> failure):

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where <math>N</math> is the sample size and <math>j</math> is the order number.
The median rank was obtained by solving the following equation for <math>Z</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

The same methodology can then be repeated by changing <math>P</math> for <math>0.50</math> <math>(50%)</math> to our desired confidence level. For <math>P=90%</math> one would formulate the equation as

::<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

Keep in mind that one must be careful to select the appropriate values for <math>P</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P=0.1</math> and once with <math>P=0.9</math>) in order to place the bounds around 80% of the population.
Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.
This non-parametric methodology is only used by Weibull++ when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [20]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

===Likelihood Ratio Confidence Bounds===
====Introduction====
A third method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.
Likelihood ratio confidence bounds are based on the equation:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}</math>

where:
::#<math>L(\theta )</math> is the likelihood function for the unknown parameter vector <math>\theta </math>
::#<math>L(\widehat{\theta })</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }</math>
::#<math>\chi _{\alpha ;k}^{2}</math> is the chi-squared statistic with probability <math>\alpha </math> and <math>k</math> degrees of freedom, where <math>k</math> is the number of quantities jointly estimated
If <math>\delta </math> is the confidence level, then <math>\alpha =\delta </math> for two-sided bounds and <math>\alpha =(2\delta -1)</math> for one-sided. Recall from Chapter 3 that if <math>x</math> is a continuous random variable with <math>pdf</math>:

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>,

where <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,{{x}_{R}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,R</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L.</math> These are represented by the <math>L(\widehat{\theta })</math> term in the denominator of the ratio in Eqn. (lratio1). Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }</math> have been calculated using MLE methods, the only unknown term in Eqn. (lratio1) is the <math>L(\theta )</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta </math> that satisfy Eqn. (lratio1). For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy Eqn. (lratio1). The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta ;</math> but at a given value of <math>\delta </math> there is only a certain region of values for <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> for which Eqn. (lratio1) holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of Eqn. (lratio1).

====Note on Contour Plots in Weibull++====
Contour plots can be used for comparing data sets. Consider two data sets, e.g. old and new design where the engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in a multiple plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e. the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If there is an overlap between the two 95% contours, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. Chapter 12 discusses comparing data sets.

====Confidence Bounds on the Parameters====
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}</math>

The task now becomes to find the values of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> so that the equality in Eqn. (lratio3) is satisfied. Unfortunately, there is no closed-form solution, thus these values must be arrived at numerically. One method of doing this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

=====Example 1=====
Five units were put on a reliability test and experienced failures at 10, 20, 30, 40, and 50 hours. Assuming a Weibull distribution, the MLE parameter estimates are calculated to be <math>\widehat{\beta }=2.2938</math> and <math>\widehat{\eta }=33.9428.</math> Calculate the 90% two-sided confidence bounds on these parameters using the likelihood ratio method.

=====Solution to Example 1=====
The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\widehat{\beta },\widehat{\eta })=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\widehat{\beta }}{\widehat{\eta }}\cdot {{\left( \frac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }-1}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{2.2938}{33.9428}\cdot {{\left( \frac{{{x}_{i}}}{33.9428} \right)}^{1.2938}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{33.9428} \right)}^{2.2938}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & 1.714714\times {{10}^{-9}}
\end{align}</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We then substitute this information into the equation:

::<math>\begin{align}
L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,\eta )-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,\eta )-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

The next step is to find the set of values of <math>\beta </math> and <math>\eta </math> that satisfy this equation, or find the values of <math>\beta </math> and <math>\eta </math> such that <math>L(\beta ,\eta )=4.432926\cdot {{10}^{-10}}.</math>

The solution is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>\eta </math>, and vice versa. The following table gives values of <math>\beta </math> based on given values of <math>\eta </math>.

These data are represented graphically in the following contour plot:

(Note that this plot is generated with degrees of freedom <math>k=1</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2</math>, for use in comparing both parameters simultaneously.) As can be determined from the table, the lowest calculated value for <math>\beta </math> is 1.142, while the highest is 3.950. These represent the two-sided 90% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>\eta </math> below 23 or above 50, these can be considered the 90% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>\eta </math>, we can perform the same procedure as before, but finding the two values of <math>\eta </math> that correspond with a given value of <math>\beta .</math> Using this method, we find that the 90% confidence limits on <math>\eta </math> are 22.474 and 49.967, which are close to the initial estimates of 23 and 50.
Note that the points where <math>\beta </math> are maximized and minimized do not necessarily correspond with the points where <math>\eta </math> are maximized and minimized. This is due to the fact that the contour plot is not symmetrical, so that the parameters will have their extremes at different points.

====Confidence Bounds on Time (Type 1)====
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.
=====Example 2=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the time estimate for a reliability of 50%. The ML estimate for the time at which <math>R(t)=50%</math> is 28.930.
=====Solution to Example 2=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the time estimate of 28.930. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>t</math> and <math>\beta .</math> This is accomplished by using a form of the Weibull reliability equation, <math>R={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}.</math> This can be rearranged in terms of <math>\eta </math>, with <math>R</math> being considered a known variable or:

::<math>\eta =\frac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}}</math>

This can then be substituted into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>t</math> and <math>\beta </math> or:

::<math>\begin{align}
& L(\beta ,t)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,t)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
& \\
L(\beta ,t)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

Note that the likelihood value for <math>L(\widehat{\beta },\widehat{\eta })</math> is the same as it was for Example 1. This is because we are dealing with the same data and parameter estimates or, in other words, the maximum value of the likelihood function did not change. It now remains to find the values of <math>\beta </math> and <math>t</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>t</math>. The following table gives the values of <math>t</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:
As can be determined from the table, the lowest calculated value for <math>t</math> is 17.389, while the highest is 41.714. These represent the 90% two-sided confidence limits on the time at which reliability is equal to 50%.

====Confidence Bounds on Reliability (Type 2)====
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta </math> and <math>R</math>. The likelihood function is once again altered in the same way as before, only now <math>R</math> is considered to be a parameter instead of <math>t</math>, since the value of <math>t</math> must be specified in advance. Once again, this process is best illustrated with an example.

=====Example 3=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the reliability estimate for <math>t=45</math>. The ML estimate for the reliability at <math>t=45</math> is 14.816%.

=====Solution to Example 3=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the reliability estimate of 14.816%. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>R</math> and <math>\beta .</math> This is again accomplished by substituting the Weibull reliability equation into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>R</math> and <math>\beta </math>:

::<math>\begin{align}
& L(\beta ,R)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,R)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,R)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

It now remains to find the values of <math>\beta </math> and <math>R</math> that satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>R</math>. The following table gives the values of <math>R</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:

As can be determined from the table, the lowest calculated value for <math>R</math> is 2.38%, while the highest is 44.26%. These represent the 90% two-sided confidence limits on the reliability at <math>t=45</math>.

===Bayesian Confidence Bounds===
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in Chapter 3.
Bayesian confidence bounds are derived from Bayes rule, which states that:

::<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
::#<math>f(\theta |Data)</math> is the <math>posterior</math> <math>pdf</math> of <math>\theta </math>
::#<math>\theta </math> is the parameter vector of the chosen distribution (i.e. Weibull, lognormal, etc.)
::#<math>L(\bullet )</math> is the likelihood function
::#<math>\varphi (\theta )</math> is the <math>prior</math> <math>pdf</math> of the parameter vector <math>\theta </math>
::#<math>\varsigma </math> is the range of <math>\theta </math>.
In other words, the prior knowledge is provided in the form of the prior <math>pdf</math> of the parameters, which in turn is combined with the sample data in order to obtain the posterior <math>pdf.</math> Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to Chapter 3). It can be seen from Eqn. (BayesRule) that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta }_{1}}</math>. Given a set of sample data, and a prior distribution for <math>{{\theta }_{1}},</math> <math>\varphi ({{\theta }_{1}}),</math> Eqn. (BayesRule) can be written as:

::<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

In other words, we now have the distribution of <math>{{\theta }_{1}}</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta }_{1}}</math> is less than or equal to a value <math>x,</math> <math>P({{\theta }_{1}}\le x)</math> can be obtained by integrating Eqn. (BayesEX), or:

::<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}</math>

Eqn. (IntBayes) essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)</math> is the confidence level and <math>x</math> is the confidence bound. Substituting Eqn. (BayesEX) into Eqn. (IntBayes) yields:

::<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

The only question at this point is what do we use as a prior distribution of <math>{{\theta }_{1}}.</math>. For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform.
Eqn. (BayesCLEX) can be generalized for any distribution having a vector of parameters <math>\theta ,</math> yielding the general equation for calculating Bayesian confidence bounds:

::<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>CL</math> is confidence level
#<math>\theta </math> is the parameter vector
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the prior <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>
#<math>\xi </math> is the range in which <math>\theta </math> changes from <math>\Psi (T,R)</math> till <math>{\theta }'s</math> maximum value or from <math>{\theta }'s</math> minimum value till <math>\Psi (T,R)</math>
#<math>\Psi (T,R)</math> is function such that if <math>T</math> is given then the bounds are calculated for <math>R</math> and if <math>R</math> is given, then he bounds are calculated for <math>T</math>.
If <math>T</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>R</math> are calculated.
If <math>R</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>T</math> are calculated.
====Confidence Bounds on Time (Type 1)====
For a given failure time distribution and a given reliability <math>R</math>, <math>T(R)</math> is a function of <math>R</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For the two-parameter Weibull distribution:

::<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)</math> is:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

where <math>f(T|Data,R)</math> is the posterior distribution of Time <math>T.</math>
Using Eqn. (T bayes), we have the following:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})</math>

Eqn. (cl) can be rewritten in terms of <math>\eta </math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))</math>

From Eqns. (IntBayes), (BayesCLEX) and (BayesCL), by assuming the priors of <math>\beta </math> and <math>\eta </math> are independent, we then obtain the following relationship:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl2) can be solved for <math>{{T}_{U}}(R)</math>, where:
#<math>CL</math> is confidence level,
#<math>\varphi (\beta )</math> is the prior <math>pdf</math> of the parameter <math>\beta </math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.</math>
#<math>\varphi (\eta )</math> is the prior <math>pdf</math> of the parameter <math>\eta .</math>. For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.</math>
#<math>L(\bullet )</math> is the likelihood function.
The same method can be used to get the one-sided lower bound of <math>T(R)</math> from:

::<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl5) can be solved to get <math>{{T}_{L}}(R)</math>.
The Bayesian two-sided bounds estimate for <math>T(R)</math> is:

::<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

which is equivalent to:

::<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

and:

::<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)</math> and <math>{{T}_{L}}(R)</math> can be solved.
====Confidence Bounds on Reliability (Type 2)====
For a given failure time distribution and a given time <math>T</math>, <math>R(T)</math> is a function of <math>T</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For example, for two parameter Weibull distribution:

::<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})</math>

The Bayesian one-sided upper bound estimate for <math>R(T)</math> is:

::<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

Similar with the bounds on Time, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl3) can be solved to get <math>{{R}_{U}}(T)</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

::<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR</math>

Using the posterior distribution, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl4) can be solved to get <math>{{R}_{L}}(T)</math>.
The Bayesian two-sided bounds estimate for <math>R(T)</math> is:

::<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

which is equivalent to:

::<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2</math>

and

::<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)</math> and <math>{{R}_{L}}(T)</math> can be solved.

===Simulation Based Bounds===
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds.
SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

=====Example 4=====
The purpose of this example is to determine the best parameter estimation method for a sample of ten units following a Weibull distribution with <math>\beta =2</math> and <math>\eta =100</math> and with complete time-to-failure data for each unit (i.e. no censoring). The number of generated data sets is set to 10,000. The SimuMatic inputs are shown next.

The parameters are estimated using RRX, RRY and MLE. The plotted results generated by SimuMatic are shown next.

Using RRX:

Using RRY:

Using MLE:

The results clearly demonstrate that the median RRX estimate provides the least deviation from the truth for this sample size and data type. However, the MLE outputs are grouped more closely together, as evidenced by the bounds. The previous figures also show the simulation-based bounds, as well as the expected variation due to sampling error.
This experiment can be repeated in SimuMatic using multiple censoring schemes (including Type I and Type II right censoring as well as random censoring) with various distributions. Multiple experiments can be performed with this utility to evaluate assumptions about the appropriate parameter estimation method to use for data sets.

Fisher Matrix Confidence Bounds

2011-06-30T18:57:43Z

Steve Sharp: /* Introduction */

This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [30] and Lloyd and Lipow [24]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds.
===Approximate Estimates of the Mean and Variance of a Function===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e. reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G</math> is presented next.

====Single Parameter Case====
For simplicity, consider a one-parameter distribution represented by a general function, <math>G,</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda </math>: <math>G(\lambda )=1/\lambda =\mu </math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)</math> can be found by:

::<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)</math>

where <math>G(\theta )</math> is some function of <math>\theta </math>, such as the reliability function, and <math>\theta </math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta </math> as <math>n\to \infty </math> . The term <math>O\left( \tfrac{1}{n} \right)</math> is a function of <math>n</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},</math> as <math>n\to \infty .</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}</math> and <math>G(x)=1/x</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}</math>. Thus as <math>n\to \infty </math>, <math>E(G(\widehat{\theta }))=\mu </math> where <math>\mu </math> and <math>\sigma </math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)</math> can then be estimated by:

::<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)</math>

====Two-Parameter Case====

Consider a Weibull distribution with two parameters <math>\beta </math> and <math>\eta </math>. For a given value of <math>T</math>, <math>R(T)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{T}{\eta } \right)}^{\beta }}}}</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)</math>, that:

::<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)</math>

and:

::<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Var(\widehat{\theta_{2}})\\

& +2{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\

& +O(\frac{1}{n^{\tfrac{3}{2}}})
\end{align}

</math>

Note that the derivatives of Eqn. (var) are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{1}},</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.</math>
Parameter Variance and Covariance Determination
The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

::<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}</math>

In the equation above, the first summation is for complete data, the second summation is for right censored data, and the third summation is for interval or left censored data. For more information on these data types, see Chapter 4.
Then the Fisher information matrix is given by:

::<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]</math>

The subscript <math>0</math> indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},</math> the true values of the parameters.
So for a sample of <math>N</math> units where <math>R</math> units have failed, <math>S</math> have been suspended, and <math>P</math> have failed within a time interval, and <math>N=R+M+P,</math> one could obtain the sample local information matrix by:

::<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}</math>

Substituting in the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}</math> and <math>{{\widehat{\theta }}_{2}}</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

::<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}</math>

Then the variance of a function (<math>Var(G)</math>) can be estimated using Eqn. (var). Values for the variance and covariance of the parameters are obtained from Eqn. (Fisher2).
Once they have been obtained, the approximate confidence bounds on the function are given as:

::<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha }}</math> next.

====Approximate Confidence Intervals on the Parameters====
In general, MLE estimates of the parameters are asymptotically normal, meaning for large sample sizes that a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }</math> is the MLE estimator for <math>\theta </math>, in the case of a single parameter distribution, estimated from a large sample of <math>n</math> units and if:

::<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}</math>

then using the normal distribution of <math>z\ \ :</math>

::<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

for large <math>n</math>. We now place confidence bounds on <math>\theta ,</math> at some confidence level <math>\delta </math>, bounded by the two end points <math>{{C}_{1}}</math> and <math>{{C}_{2}}</math> where:

::<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta </math>

From Eqn. (e729):

::<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta </math>

where <math>{{K}_{\alpha }}</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)</math>

Now by simplifying Eqn. (e731), one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta ,</math> at a confidence level <math>\delta ,</math> or:

::<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)</math>

The upper one-sided bounds are given by:

::<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

while the lower one-sided bounds are given by:

::<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

If <math>\widehat{\theta }</math> must be positive, then <math>\ln \widehat{\theta }</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta </math>, then become:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}</math>

The one-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta ,</math> can be found from:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [24] further elaborate on this procedure.

====Confidence Bounds on Time (Type 1)====
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e. y-ordinate or unreliability, <math>Q=1-R)</math> is determined by solving the unreliability function for the time, <math>T</math>, or:

::<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cb). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Note that the procedure is slightly more complicated for distributions with more than one parameter.

====Confidence Bounds on Reliability (Type 2)====
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

::<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cbr). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Once again, the procedure is more complicated for distributions with more than one parameter.

===Beta Binomial Confidence Bounds===
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see Chapter 4). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution.
Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z</math> (rank for the <math>{{j}^{th}}</math> failure):

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where <math>N</math> is the sample size and <math>j</math> is the order number.
The median rank was obtained by solving the following equation for <math>Z</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

The same methodology can then be repeated by changing <math>P</math> for <math>0.50</math> <math>(50%)</math> to our desired confidence level. For <math>P=90%</math> one would formulate the equation as

::<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

Keep in mind that one must be careful to select the appropriate values for <math>P</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P=0.1</math> and once with <math>P=0.9</math>) in order to place the bounds around 80% of the population.
Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.
This non-parametric methodology is only used by Weibull++ when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [20]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

===Likelihood Ratio Confidence Bounds===
====Introduction====
A third method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.
Likelihood ratio confidence bounds are based on the equation:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}</math>

where:
::#<math>L(\theta )</math> is the likelihood function for the unknown parameter vector <math>\theta </math>
::#<math>L(\widehat{\theta })</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }</math>
::#<math>\chi _{\alpha ;k}^{2}</math> is the chi-squared statistic with probability <math>\alpha </math> and <math>k</math> degrees of freedom, where <math>k</math> is the number of quantities jointly estimated
If <math>\delta </math> is the confidence level, then <math>\alpha =\delta </math> for two-sided bounds and <math>\alpha =(2\delta -1)</math> for one-sided. Recall from Chapter 3 that if <math>x</math> is a continuous random variable with <math>pdf</math>:

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>,

where <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,{{x}_{R}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,R</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L.</math> These are represented by the <math>L(\widehat{\theta })</math> term in the denominator of the ratio in Eqn. (lratio1). Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }</math> have been calculated using MLE methods, the only unknown term in Eqn. (lratio1) is the <math>L(\theta )</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta </math> that satisfy Eqn. (lratio1). For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy Eqn. (lratio1). The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta ;</math> but at a given value of <math>\delta </math> there is only a certain region of values for <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> for which Eqn. (lratio1) holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of Eqn. (lratio1).

====Note on Contour Plots in Weibull++====
Contour plots can be used for comparing data sets. Consider two data sets, e.g. old and new design where the engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in a multiple plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e. the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If there is an overlap between the two 95% contours, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. Chapter 12 discusses comparing data sets.

====Confidence Bounds on the Parameters====
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}</math>

The task now becomes to find the values of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> so that the equality in Eqn. (lratio3) is satisfied. Unfortunately, there is no closed-form solution, thus these values must be arrived at numerically. One method of doing this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

=====Example 1=====
Five units were put on a reliability test and experienced failures at 10, 20, 30, 40, and 50 hours. Assuming a Weibull distribution, the MLE parameter estimates are calculated to be <math>\widehat{\beta }=2.2938</math> and <math>\widehat{\eta }=33.9428.</math> Calculate the 90% two-sided confidence bounds on these parameters using the likelihood ratio method.

=====Solution to Example 1=====
The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\widehat{\beta },\widehat{\eta })=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\widehat{\beta }}{\widehat{\eta }}\cdot {{\left( \frac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }-1}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{2.2938}{33.9428}\cdot {{\left( \frac{{{x}_{i}}}{33.9428} \right)}^{1.2938}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{33.9428} \right)}^{2.2938}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & 1.714714\times {{10}^{-9}}
\end{align}</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We then substitute this information into the equation:

::<math>\begin{align}
L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,\eta )-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,\eta )-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

The next step is to find the set of values of <math>\beta </math> and <math>\eta </math> that satisfy this equation, or find the values of <math>\beta </math> and <math>\eta </math> such that <math>L(\beta ,\eta )=4.432926\cdot {{10}^{-10}}.</math>

The solution is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>\eta </math>, and vice versa. The following table gives values of <math>\beta </math> based on given values of <math>\eta </math>.

These data are represented graphically in the following contour plot:

(Note that this plot is generated with degrees of freedom <math>k=1</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2</math>, for use in comparing both parameters simultaneously.) As can be determined from the table, the lowest calculated value for <math>\beta </math> is 1.142, while the highest is 3.950. These represent the two-sided 90% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>\eta </math> below 23 or above 50, these can be considered the 90% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>\eta </math>, we can perform the same procedure as before, but finding the two values of <math>\eta </math> that correspond with a given value of <math>\beta .</math> Using this method, we find that the 90% confidence limits on <math>\eta </math> are 22.474 and 49.967, which are close to the initial estimates of 23 and 50.
Note that the points where <math>\beta </math> are maximized and minimized do not necessarily correspond with the points where <math>\eta </math> are maximized and minimized. This is due to the fact that the contour plot is not symmetrical, so that the parameters will have their extremes at different points.

====Confidence Bounds on Time (Type 1)====
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.
=====Example 2=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the time estimate for a reliability of 50%. The ML estimate for the time at which <math>R(t)=50%</math> is 28.930.
=====Solution to Example 2=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the time estimate of 28.930. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>t</math> and <math>\beta .</math> This is accomplished by using a form of the Weibull reliability equation, <math>R={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}.</math> This can be rearranged in terms of <math>\eta </math>, with <math>R</math> being considered a known variable or:

::<math>\eta =\frac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}}</math>

This can then be substituted into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>t</math> and <math>\beta </math> or:

::<math>\begin{align}
& L(\beta ,t)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,t)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
& \\
L(\beta ,t)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

Note that the likelihood value for <math>L(\widehat{\beta },\widehat{\eta })</math> is the same as it was for Example 1. This is because we are dealing with the same data and parameter estimates or, in other words, the maximum value of the likelihood function did not change. It now remains to find the values of <math>\beta </math> and <math>t</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>t</math>. The following table gives the values of <math>t</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:
As can be determined from the table, the lowest calculated value for <math>t</math> is 17.389, while the highest is 41.714. These represent the 90% two-sided confidence limits on the time at which reliability is equal to 50%.

====Confidence Bounds on Reliability (Type 2)====
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta </math> and <math>R</math>. The likelihood function is once again altered in the same way as before, only now <math>R</math> is considered to be a parameter instead of <math>t</math>, since the value of <math>t</math> must be specified in advance. Once again, this process is best illustrated with an example.

=====Example 3=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the reliability estimate for <math>t=45</math>. The ML estimate for the reliability at <math>t=45</math> is 14.816%.

=====Solution to Example 3=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the reliability estimate of 14.816%. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>R</math> and <math>\beta .</math> This is again accomplished by substituting the Weibull reliability equation into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>R</math> and <math>\beta </math>:

::<math>\begin{align}
& L(\beta ,R)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,R)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,R)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

It now remains to find the values of <math>\beta </math> and <math>R</math> that satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>R</math>. The following table gives the values of <math>R</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:

As can be determined from the table, the lowest calculated value for <math>R</math> is 2.38%, while the highest is 44.26%. These represent the 90% two-sided confidence limits on the reliability at <math>t=45</math>.

===Bayesian Confidence Bounds===
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in Chapter 3.
Bayesian confidence bounds are derived from Bayes rule, which states that:

::<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>f(\theta |Data)</math> is the <math>posterior</math> <math>pdf</math> of <math>\theta </math>
#<math>\theta </math> is the parameter vector of the chosen distribution (i.e. Weibull, lognormal, etc.)
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the <math>prior</math> <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>.
In other words, the prior knowledge is provided in the form of the prior <math>pdf</math> of the parameters, which in turn is combined with the sample data in order to obtain the posterior <math>pdf.</math> Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to Chapter 3). It can be seen from Eqn. (BayesRule) that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta }_{1}}</math>. Given a set of sample data, and a prior distribution for <math>{{\theta }_{1}},</math> <math>\varphi ({{\theta }_{1}}),</math> Eqn. (BayesRule) can be written as:

::<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

In other words, we now have the distribution of <math>{{\theta }_{1}}</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta }_{1}}</math> is less than or equal to a value <math>x,</math> <math>P({{\theta }_{1}}\le x)</math> can be obtained by integrating Eqn. (BayesEX), or:

::<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}</math>

Eqn. (IntBayes) essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)</math> is the confidence level and <math>x</math> is the confidence bound. Substituting Eqn. (BayesEX) into Eqn. (IntBayes) yields:

::<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

The only question at this point is what do we use as a prior distribution of <math>{{\theta }_{1}}.</math>. For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform.
Eqn. (BayesCLEX) can be generalized for any distribution having a vector of parameters <math>\theta ,</math> yielding the general equation for calculating Bayesian confidence bounds:

::<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>CL</math> is confidence level
#<math>\theta </math> is the parameter vector
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the prior <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>
#<math>\xi </math> is the range in which <math>\theta </math> changes from <math>\Psi (T,R)</math> till <math>{\theta }'s</math> maximum value or from <math>{\theta }'s</math> minimum value till <math>\Psi (T,R)</math>
#<math>\Psi (T,R)</math> is function such that if <math>T</math> is given then the bounds are calculated for <math>R</math> and if <math>R</math> is given, then he bounds are calculated for <math>T</math>.
If <math>T</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>R</math> are calculated.
If <math>R</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>T</math> are calculated.
====Confidence Bounds on Time (Type 1)====
For a given failure time distribution and a given reliability <math>R</math>, <math>T(R)</math> is a function of <math>R</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For the two-parameter Weibull distribution:

::<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)</math> is:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

where <math>f(T|Data,R)</math> is the posterior distribution of Time <math>T.</math>
Using Eqn. (T bayes), we have the following:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})</math>

Eqn. (cl) can be rewritten in terms of <math>\eta </math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))</math>

From Eqns. (IntBayes), (BayesCLEX) and (BayesCL), by assuming the priors of <math>\beta </math> and <math>\eta </math> are independent, we then obtain the following relationship:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl2) can be solved for <math>{{T}_{U}}(R)</math>, where:
#<math>CL</math> is confidence level,
#<math>\varphi (\beta )</math> is the prior <math>pdf</math> of the parameter <math>\beta </math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.</math>
#<math>\varphi (\eta )</math> is the prior <math>pdf</math> of the parameter <math>\eta .</math>. For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.</math>
#<math>L(\bullet )</math> is the likelihood function.
The same method can be used to get the one-sided lower bound of <math>T(R)</math> from:

::<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl5) can be solved to get <math>{{T}_{L}}(R)</math>.
The Bayesian two-sided bounds estimate for <math>T(R)</math> is:

::<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

which is equivalent to:

::<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

and:

::<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)</math> and <math>{{T}_{L}}(R)</math> can be solved.
====Confidence Bounds on Reliability (Type 2)====
For a given failure time distribution and a given time <math>T</math>, <math>R(T)</math> is a function of <math>T</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For example, for two parameter Weibull distribution:

::<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})</math>

The Bayesian one-sided upper bound estimate for <math>R(T)</math> is:

::<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

Similar with the bounds on Time, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl3) can be solved to get <math>{{R}_{U}}(T)</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

::<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR</math>

Using the posterior distribution, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl4) can be solved to get <math>{{R}_{L}}(T)</math>.
The Bayesian two-sided bounds estimate for <math>R(T)</math> is:

::<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

which is equivalent to:

::<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2</math>

and

::<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)</math> and <math>{{R}_{L}}(T)</math> can be solved.

===Simulation Based Bounds===
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds.
SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

=====Example 4=====
The purpose of this example is to determine the best parameter estimation method for a sample of ten units following a Weibull distribution with <math>\beta =2</math> and <math>\eta =100</math> and with complete time-to-failure data for each unit (i.e. no censoring). The number of generated data sets is set to 10,000. The SimuMatic inputs are shown next.

The parameters are estimated using RRX, RRY and MLE. The plotted results generated by SimuMatic are shown next.

Using RRX:

Using RRY:

Using MLE:

The results clearly demonstrate that the median RRX estimate provides the least deviation from the truth for this sample size and data type. However, the MLE outputs are grouped more closely together, as evidenced by the bounds. The previous figures also show the simulation-based bounds, as well as the expected variation due to sampling error.
This experiment can be repeated in SimuMatic using multiple censoring schemes (including Type I and Type II right censoring as well as random censoring) with various distributions. Multiple experiments can be performed with this utility to evaluate assumptions about the appropriate parameter estimation method to use for data sets.

Appendix: Log-Likelihood Equations

2011-06-30T18:51:57Z

Steve Sharp:

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)
::• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval failure data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)
::• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
 
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.
 
Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

::<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)
::• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

::<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

::<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

::<math>\lambda =0</math>

or:

::<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)
::• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>{{F}_{i}}</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

::<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

::<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)
::• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>Q</math> is the number of subpopulations
::• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)
::• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)
::• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of groups of interval data points
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

::<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)
::• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval failure data group
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)
::• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval failure data groups,
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)
::• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval failure data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

::<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)
::• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval failure data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Time-Dependent System Reliability (Analytical)

2011-06-30T16:00:42Z

Steve Sharp:

In the previous chapter, different system configuration types were examined, as well as different methods for obtaining the system's reliability function analytically. Because the reliabilities in the problems presented were treated as probabilities (e.g. <math>P(A)</math> , <math>{{R}_{i}}</math> ), the reliability values and equations presented were referred to as static (not time-dependent). Thus, in the prior chapter, the life distributions of the components were not incorporated in the process of calculating the system reliability. In this chapter, time dependency in the reliability function will be introduced. We will develop the models necessary to observe the reliability over the life of the system, instead of at just one point in time. In addition, performance measures such as failure rate, MTTF and warranty time will be estimated for the entire system. The methods of obtaining the reliability function analytically remain identical to the ones presented in the previous chapter, with the exception that the reliabilities will be functions of time. In other words, instead of dealing with <math>{{R}_{i}}</math> , we will use <math>{{R}_{i}}(t)</math> . All examples in this chapter assume that no repairs are performed on the components.
= Sections =
#[[Analytical Life Predictions]]
#[[Approximating the System CDF]]
#[[Duty Cycle]]
#[[Load Sharing]]
#[[Standby Components]]
#[[Note Regarding Numerical Integration Solutions]]

Probability Plotting

2011-06-29T22:21:44Z

Steve Sharp:

One method of calculating the parameter of the exponential distribution is by using probability plotting. To better illustrate this procedure, consider the following example.

 

====Example 1====
 

Let's assume six identical units are reliability tested at the same application and operation
stress levels. All of these units fail during the test after operating for the following times (in hours), <math>{{T}_{i}}</math> : 96, 257, 498, 763, 1051 and 1744.
 
The steps for determining the parameters of the exponential <math>pdf</math> representing the
data, using probability plotting, are as follows:
 
:• Rank the times-to-failure in ascending order as shown next.
 

::<math>\begin{matrix}
\text{Time-to-} & \text{Failure Order Number} \\
\text{failure, hr} & \text{out of a Sample Size of 6} \\
\text{96} & \text{1} \\
\text{257} & \text{2} \\
\text{498} & \text{3} \\
\text{763} & \text{4} \\
\text{1,051} & \text{5} \\
\text{1,744} & \text{6} \\
\end{matrix}</math>

 
:• Obtain their median rank plotting positions.
 
Median rank positions are used instead of other ranking methods because median ranks are at a
specific confidence level (50%).
 
:• The times-to-failure, with their corresponding median ranks, are shown next:

 
::<math>\begin{matrix}
\text{Time-to-} & \text{Median} \\
\text{failure, hr} & \text{Rank, }% \\
\text{96} & \text{10}\text{.91} \\
\text{257} & \text{26}\text{.44} \\
\text{498} & \text{42}\text{.14} \\
\text{763} & \text{57}\text{.86} \\
\text{1,051} & \text{73}\text{.56} \\
\text{1,744} & \text{89}\text{.10} \\
\end{matrix}</math>

 
:• On an exponential probability paper, plot the times on the x-axis and their corresponding
rank value on the y-axis. Fig. 4 displays an example of an exponential probability paper. The
paper is simply a log-linear paper. (The solution is given in Fig. 2.)
 
[[File:ALTA4.1.gif|center]]
 
<center>Fig. 4: Sample exponential probability paper.</center>
 
:• Draw the best possible straight line that goes through the <math>t=0</math> and <math>
(t)=100%</math> point and through the plotted points (as shown in Fig. 5).
 
:• At the <math>Q(t)=63.2%</math> or <math>R(t)=36.8%</math> ordinate point, draw a
straight horizontal line until this line intersects the fitted straight line. Draw a vertical line through this intersection until it crosses the abscissa. The value at the intersection of the abscissa is the estimate of the mean. For this case, <math>\widehat{\mu }=833</math> hr which means that <math>\lambda =\tfrac{1}{\mu }=0.0012</math> . (This is always at 63.2% since <math>(T)=1-{{e}^{-\tfrac{\mu }{\mu }}}=1-{{e}^{-1}}=0.632=63.2%).</math>
 
[[File:ALTA4.2.gif|center]]
 
<center>Fig. 5: Probability plot for Example 1.</center>
 
 
Now any reliability value for any mission time <math>t</math> can be obtained. For example, the
reliability for a mission of 15 hr, or any other time, can now be obtained either from the plot or analytically (i.e. using the equations given in Section <math>5.1.1</math> ).

 
To obtain the value from the plot, draw a vertical line from the abscissa, at <math>t=15</math>
hr, to the fitted line. Draw a horizontal line from this intersection to the ordinate and read
<math>R(t)</math> . In this case, <math>R(t=15)=98.15%</math> . This can also be obtained
analytically, from the exponential reliability function.

 

====MLE Parameter Estimation====

 
The parameter of the exponential distribution can also be estimated using the maximum likelihood estimation (MLE) method. This log-likelihood function is:

 
::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\overset{FI}{\mathop{\underset{i=1}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{\prime \prime }\ln [R_{Li}^{\prime \prime }-R_{Ri}^{\prime \prime }]</math>
 
where:
 
::<math>R_{Li}^{\prime \prime }={{e}^{-\lambda T_{Li}^{\prime \prime }}}</math>

 
::<math>R_{Ri}^{\prime \prime }={{e}^{-\lambda T_{Ri}^{\prime \prime }}}</math>

 
and:
 
:• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points.
 
:• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group.
 
:• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found).
 
:• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data.
 
:• <math>S</math> is the number of groups of suspension data points.
 
:• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points.
 
:• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group.
 
:• <math>FI</math> is the number of interval data groups.
 
:• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the i <math>^{th}</math> group of data intervals.
 
:• <math>T_{Li}^{\prime \prime }</math> is the beginning of the i <math>^{th}</math> interval.
 
:• <math>T_{Ri}^{\prime \prime }</math> is the ending of the i <math>^{th}</math> interval.
 
 
The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0</math> where:
 
::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime }-\overset{FI}{\mathop{\underset{i=1}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{\prime \prime }\frac{T_{Li}^{\prime \prime }R_{Li}^{\prime \prime }-T_{Ri}^{\prime \prime }R_{Ri}^{\prime \prime }}{R_{Li}^{\prime \prime }-R_{Ri}^{\prime \prime }}</math>

 

====Example 2====

 
Using the same data as in the probability plotting example (Example 1), and assuming an exponential distribution, estimate the parameter using the MLE method.
 
''Solution''
 
In this example we have non-grouped data without suspensions. Thus Eqn. (exp-mle) becomes:
 
::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}} \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}} \right) \right]=0</math>

 
Substituting the values for <math>T</math> we get:

 
::<math>\begin{align}
& \frac{6}{\lambda }= & 4409,\text{ or:} \\
& \lambda = & 0.00136\text{ failure/hr}
\end{align}</math>

Appendix A: Generating Random Numbers from a Distribution

2011-06-29T22:02:44Z

Steve Sharp:

Simulation involves generating random numbers that belong to a specific distribution. We will illustrate this methodology using the Weibull distribution.
= Sections =
#[[Generating Random Times from a Weibull Distribution]]
#[[Conditional]]
#[[Regarding BlockSim's Random Number Generator (RNG)]]

Basics of System Reliability Analysis

2011-06-29T21:10:48Z

Steve Sharp: Created page with '=Sections= #Overview #Basic Terminology'

=Sections=
#[[Overview]]
#[[Basic Terminology]]

Lognormal Statistical Properties

2011-06-29T17:13:09Z

Steve Sharp:

====The Mean or MTTF====
The mean of the lognormal distribution, <math>\mu </math> , is given by [18]:

::<math>\mu ={{e}^{{\mu }'+\tfrac{1}{2}\sigma _{{{T}'}}^{2}}}</math>

The mean of the natural logarithms of the times-to-failure, <math>\mu'</math> , in terms of <math>\bar{T}</math> and <math>{{\sigma }_{T}}</math> is givgen by:

::<math>{\mu }'=\ln \left( {\bar{T}} \right)-\frac{1}{2}\ln \left( \frac{\sigma _{T}^{2}}{{{{\bar{T}}}^{2}}}+1 \right)</math>

====The Median====
The median of the lognormal distribution, <math>\breve{T}</math> , is given by [18]:

::<math>\breve{T}={{e}^{{{\mu }'}}}</math>

====The Mode====
The mode of the lognormal distribution, <math>\tilde{T}</math> , is given by [1]:

::<math>\tilde{T}={{e}^{{\mu }'-\sigma _{{{T}'}}^{2}}}</math>

====The Standard Deviation====
The standard deviation of the lognormal distribution, <math>{{\sigma }_{T}}</math> , is given by [18]:

::<math>{{\sigma }_{T}}=\sqrt{\left( {{e}^{2{\mu }'+\sigma _{{{T}'}}^{2}}} \right)\left( {{e}^{\sigma _{{{T}'}}^{2}}}-1 \right)}</math>

The standard deviation of the natural logarithms of the times-to-failure, <math>{{\sigma }_{{{T}'}}}</math> , in terms of <math>\bar{T}</math> and <math>{{\sigma }_{T}}</math> is given by:

::<math>{{\sigma }_{{{T}'}}}=\sqrt{\ln \left( \frac{\sigma _{T}^{2}}{{{{\bar{T}}}^{2}}}+1 \right)}</math>

====The Lognormal Reliability Function====
The reliability for a mission of time <math>T</math> , starting at age 0, for the lognormal distribution is determined by:

::<math>R(T)=\int_{T}^{\infty }f(t)dt</math>

or:

::<math>R(T)=\int_{{{T}^{^{\prime }}}}^{\infty }\frac{1}{{{\sigma }_{{{T}'}}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{t-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}^{2}}}}dt</math>

As with the normal distribution, there is no closed-form solution for the lognormal reliability function. Solutions can be obtained via the use of standard normal tables. Since the application automatically solves for the reliability we will not discuss manual solution methods. For interested readers, full explanations can be found in the references.

Bayesian-Weibull Analysis

2011-06-29T16:57:44Z

Steve Sharp:

In this section, the Bayesian methods are presented for the two-parameter Weibull distribution. Bayesian concepts were introduced in Chapter 3. This model considers prior knowledge on the shape (β) parameter of the Weibull distribution when it is chosen to be fitted to a given set of data. There are many practical applications for this model, particularly when dealing with small sample sizes and some prior knowledge for the shape parameter is available. For example, when a test is performed, there is often a good understanding about the behavior of the failure mode under investigation, primarily through historical data. At the same time, most reliability tests are performed on a limited number of samples. Under these conditions, it would be very useful to use this prior knowledge with the goal of making more accurate predictions. A common approach for such scenarios is to use the one-parameter Weibull distribution, but this approach is too deterministic, too absolute you may say (and you would be right). The Weibull-Bayesian model in Weibull++ (which is actually a true "WeiBayes" model, unlike the one-parameter Weibull that is commonly referred to as such) offers an alternative to the one-parameter Weibull, by including the variation and uncertainty that might have been observed in the past on the shape parameter. Applying Bayes's rule on the two-parameter Weibull distribution and assuming the prior distributions of β and η are independent, we obtain the following posterior :

::<math> f(\beta ,\eta |Data)=\dfrac{L(\beta ,\eta )\varphi (\beta )\varphi (\eta )}{ \int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta } </math> EQNREF WeibBayes

In this model, η is assumed to follow a noninformative prior distribution with the density function <math> \varphi (\eta )=\dfrac{1}{\eta } </math>. This is called Jeffrey's prior, and is obtained by performing a logarithmic transformation on η. Specifically, since η is always positive, we can assume that ln(η) follows a uniform distribution, ''U''( − ∞, + ∞). Applying Jeffrey's rule [9] which says "in general, an approximate non-informative prior is taken proportional to the square root of Fisher's information", yields <math> \varphi (\eta )=\dfrac{1}{\eta }. </math>

The prior distribution of β, denoted as <math> \varphi (\beta ) </math>, can be selected from the following distributions: normal, lognormal, exponential and uniform. The procedure of performing a Weibull-Bayesian analysis is as follows:

*Collect the times-to-failure data.
*Specify a prior distribution for β (the prior for η is assumed to be 1/η).
*Obtain the posterior from Eqn. (EQNREF WeibBayes ).

In other words, a distribution (the posterior ) is obtained, rather than a point estimate as in classical statistics (i.e., as in the parameter estimation methods described previously in this chapter). Therefore, if a point estimate needs to be reported, a point of the posterior needs to be calculated. Typical points of the posterior distribution used are the mean (expected value) or median. In Weibull++, both options are available and can be chosen from the ''Analysis'' page, under the ''Results As'' area, as shown next.

The expected value of β is obtained by:

::<math> E(\beta )=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\beta \cdot f(\beta ,\eta |Data)d\beta d\eta </math>

Similarly, the expected value of η is obtained by:

::<math> E(\eta )=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\eta \cdot f(\beta ,\eta |Data)d\beta d\eta </math>

The median points are obtained by solving the following equations for <math> \breve{\beta} </math> and <math> \breve{\eta} </math> respectively:

::<math> \int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\breve{\beta}}f(\beta ,\eta |Data)d\beta d\eta =0.5 </math>

and

::<math> \int\nolimits_{0}^{\breve{\eta}}\int\nolimits_{0}^{\infty }f(\beta ,\eta |Data)d\beta d\eta =0.5 </math>

Of course, other points of the posterior distribution can be calculated as well. For example, one may want to calculate the 10th percentile of the joint posterior distribution (w.r.t. one of the parameters). The procedure for obtaining other points of the posterior distribution is similar to the one for obtaining the median values, where instead of 0.5 the percentage of interest is given. This procedure actually provides the confidence bounds on the parameters, which in the Bayesian framework are called ‘‘Credible Bounds‘‘. However, since the engineering interpretation is the same, and to avoid confusion, we refer to them as confidence bounds in this reference and in Weibull++.

== Posterior Distributions for Functions of Parameters ==

As explained in Chapter 3, in Bayesian analysis, all the functions of the parameters are distributed. In other words, a posterior distribution is obtained for functions such as reliability and failure rate, instead of point estimate as in classical statistics. Therefore, in order to obtain a point estimate for these functions, a point on the posterior distributions needs to be calculated. Again, the expected value (mean) or median value are used.

===<math>pdf</math> of the Times-to-Failure ===

The posterior distribution of the failure time is given by:

::<math> f(T|Data)=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }f(T,\beta ,\eta )f(\beta ,\eta |Data)d\eta d\beta </math> EQNREF WeibBayesPDF

where:

::<math> f(T,\beta ,\eta )=\dfrac{\beta }{\eta }\left( \dfrac{T}{\eta }\right) ^{\beta -1}e^{-\left( \dfrac{T}{\eta }\right) ^{\beta }} </math>

For the <math>pdf</math> of the times-to-failure, only the expected value is calculated and reported in Weibull++.

=== Reliability ===

In order to calculate the median value of the reliability function, we first need to obtain posterior of the reliability. Since ''R''(''T'') is a function of β, the density functions of β and ''R''(''T'') have the following relationship:

::<math> \begin{align} f(R|Data,T)dR = & f(\beta |Data)d\beta)\\
= & (\int\nolimits_{0}^{\infty }f(\beta ,\eta |Data)d{\eta}) d{\beta} \\
=& \dfrac{\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }d\beta
\end{align}
</math> EQNREF Rpdf

The median value of the reliability is obtained by solving the following equation w.r.t. <math> \breve{R}: </math>

::<math> \int\nolimits_{0}^{\breve{R}}f(R|Data,T)dR=0.5 </math> EQNREF MedRel

The expected value of the reliability at time is given by:

::<math> R(T|Data)=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }R(T,\beta ,\eta )f(\beta ,\eta |Data)d\eta d\beta </math> where:

::<math> R(T,\beta ,\eta )=e^{-\left( \dfrac{T}{\eta }\right) ^{^{\beta }}} </math>

 

=== Failure Rate ===

The failure rate at time is given by:

::<math> \lambda (T|Data)=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\lambda (T,\beta ,\eta )L(\beta ,\eta )\varphi (\eta )\varphi (\beta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\eta )\varphi (\beta )d\eta d\beta } </math>

where:

::<math> \lambda (T,\beta ,\eta )=\dfrac{\beta }{\eta }\left( \dfrac{T}{\eta }\right) ^{\beta -1} </math>

 

== Note on Calculated Results ==

As mentioned above, in order to obtain point estimates for the parameters of functions of the parameters in Bayesian analysis, the Median or Mean values of the different posterior <math>pdf</math>s are calculated. It is important to note that the Median value is preferable and is the default in Weibull++. This is because the Median value always corresponds to the 50th percentile of the distribution. On the other hand, the Mean is not a fixed point on the distribution, which could cause issues, especially when comparing results across different data sets.

== Confidence Bounds on ''R''(''T'') ==

The confidence bounds calculation under the Weibull-Bayesian analysis is very similar to the Bayesian Confidence Bounds method described in the previous section, with the exception that in the case of the Weibull-Bayesian Analysis the specified prior of β is considered instead of an non-informative prior. The Bayesian one-sided upper bound estimate for ''R''(''T'') is given by:

::<math> \int\nolimits_{0}^{R_{U}(T)}f(R|Data,T)dR=CL </math>

Using Eqns. (EQNREF WeibBayes ) and (EQNREF Rpdf ) the following is obtained:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{T\exp (-\dfrac{\ln (-\ln R_{U})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLRU

Eqn. (EQNREF 1CLRU ) can be solved for ''R''''U''(''T''). The Bayesian one-sided lower bound estimate for <math> \ R(T) </math> is given by:

:<math> \int\nolimits_{0}^{R_{L}(T)}f(R|Data,T)dR=1-CL </math>

Using Eqns. (EQNREF WeibBayes ) and (EQNREF Rpdf ) the following is obtained:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T\exp (-\dfrac{\ln (-\ln R_{L})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=1-CL </math> EQNREF 1CLRL

Eqn. (EQNREF 1CLRL ) can be solved for ''R''''L''(''T''). The Bayesian two-sided bounds estimate for ''R''(''T'') is given by:

::<math> \int\nolimits_{R_{L}(T)}^{R_{U}(T)}f(R|Data,T)dR=CL </math> which is equivalent to:

::<math> \int\nolimits_{0}^{R_{U}(T)}f(R|Data,T)dR=(1+CL)/2 </math>

and

::<math> \int\nolimits_{0}^{R_{L}(T)}f(R|Data,T)dR=(1-CL)/2 </math>

Using the same method for one-sided bounds, ''R''''U''(''T'')and ''R''''L''(''T'') can be computed.

== Confidence Bounds on Time ==

Following the same procedure described for bounds on Reliability, the bounds of time can be calculated, given . The Bayesian one-sided upper bound estimate for ''T''(''R'') is given by:

::<math> \int\nolimits_{0}^{T_{U}(R)}f(T|Data,R)dT=CL </math>

Using Eqns. (EQNREF WeibBayes ) and. (EQNREF WeibBayesPDF ), we obtain:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T_{U}\exp (-\dfrac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLTU

Eqn. (EQNREF 1CLTU ) can be solved for ''T''''U''(''R''). The Bayesian one-sided lower bound estimate for ''T''(''R'') is given by:

::<math> \int\nolimits_{0}^{T_{L}(R)}f(T|Data,R)dT=1-CL </math>

or:

::<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{T_{L}\exp (\dfrac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLTL

Eqn. (EQNREF 1CLTL ) can be solved for ''T''''L''(''R''). The Bayesian two-sided lower bounds estimate for ''T''(''R'') is:

::<math> \int\nolimits_{T_{L}(R)}^{T_{U}(R)}f(T|Data,R)dT=CL </math>

which is equivalent to:

::<math> \int\nolimits_{0}^{T_{U}(R)}f(T|Data,R)dT=(1+CL)/2 </math>

and:

::<math> \int\nolimits_{0}^{T_{L}(R)}f(T|Data,R)dT=(1-CL)/2 </math>

 

=====Example 6=====

A manufacturer has tested prototypes of a modified product. The test was terminated at 2000 hours, with only two failures observed from a sample size of eighteen.

{| border=1 cellspacing=1 align="center"
|-
|Number of State||State of F or S||State End Time
|-
| 1 || F || 1180
|-
| 1 || F || 1842
|-
| 16 || S || 2000
|}

Because of the lack of failure data in the prototype testing, the manufacturer decided to use information gathered from prior tests on this product to increase the confidence in the results of the prototype testing. This decision was made because failure analysis indicated that the failure mode of these two failures is the same as the one observed in previous tests. In other words, it is expected that the shape of the distribution hasn't changed, but hopefully the scale has, indicating longer life. The two-parameter Weibull distribution have been used to model all prior tests results. The list of the estimated β parameter is as follows:

{| border=1 cellspacing=1 align="center"
|-
|Betas Obtained for Similar Mode
|-
| 1.7
|-
| 2.1
|-
| 2.4
|-
|3.1
|-
|3.5
|}

First, in order to fit the data to a Weibull-Bayesian model, a prior distribution for β needs to be determined. Based on the prior tests' β values, the prior distribution for β was found to be a lognormal distribution with μ = 0.9064, σ = 0.3325 (obtained by entering the β values into a Weibull++ ''Standard Folio'' and analyzing it based on the RRX analysis method.)

the test data is entered into a ''Standard Folio'', the Weibull-Bayesian is selected under '' Distribution'' and the β prior distribution is entered after clicking the ''Calculate'' button.

Suppose that the reliability at 3000hr is the metric of interest in this example. This reliability can be obtained using Eqn. (EQNREF MedRel ), resulting in the median value of the posterior of the reliability at 3000hr. Using the ''QCP'', this value is calculated to be 76.97. ( By default Weibull++ returns the median values of the posterior distribution. )

The posterior <math>pdf</math> of the reliability function at 3000hrs can be obtained using Eqn. (EQNREF Rpdf ). In Figure 6-10 the posterior <math>pdf</math> of the reliability at 3000hrs is plotted, with the corresponding median value as well as the 10th percentile value shown. The 10th percentile constitutes the 90 Lower 1-Sided bound on the reliability at 3000hrs, which is calculated to be 50.77.

FIGURE HERE

Notice that the <math>pdf</math> plotted in Fig. 6-10 is of the reliability at 3000hrs, and not the <math>pdf</math> of the times-to-failure data. The <math>pdf</math> of the times-to-failure data can be obtained using Eqn. (EQNREF WeibBayesPDF ) and plotted using Weibull++, as shown next:

FIGURE HERE

{{RS Copyright}}

[[Category:Life_Data_Analysis_Reference]]

Template:Likelihood Ratio Confidence Bounds

2011-06-29T16:54:08Z

Steve Sharp:

As covered in Chapter 5, the likelihood confidence bounds are calculated by finding values for θ1 and θ2 that satisfy:

::<math> -2\cdot \text{ln}\left( \frac{L(\theta _{1},\theta _{2})}{L(\hat{\theta }_{1}, \hat{\theta }_{2})}\right) =\chi _{\alpha ;1}^{2} EQNREF lratio2 </math>

This equation can be rewritten as:

::<math> L(\theta _{1},\theta _{2})=L(\hat{\theta }_{1},\hat{\theta } _{2})\cdot e^{\frac{-\chi _{\alpha ;1}^{2}}{2}} EQNREF lratio3 </math>

For complete data, the likelihood function for the Weibull distribution is given by:

::<math> L(\beta ,\eta )=\prod_{i=1}^{N}f(x_{i};\beta ,\eta )=\prod_{i=1}^{N}\frac{ \beta }{\eta }\cdot \left( \frac{x_{i}}{\eta }\right) ^{\beta -1}\cdot e^{-\left( \frac{x_{i}}{\eta }\right) ^{\beta }} </math>

For a given value of α, values for β and η can be found which represent the maximum and minimum values that satisfy Eqn. (\ref {lratio3}). These represent the confidence bounds for the parameters at a confidence level δ, where α = δ for two-sided bounds and α = 2δ − 1 for one-sided.

Similarly, the bounds on time and reliability can be found by substituting the Weibull reliability equation into the likelihood function so that it is in terms of β and time or reliability, as discussed in Chapter 5. The likelihood ratio equation used to solve for bounds on time (Type 1) is:

::<math> L(\beta ,t)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] </math>

The likelihood ratio equation used to solve for bounds on reliability (Type 2) is:

::<math> L(\beta ,R)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] </math>

Fisher Matrix Confidence Bounds

2011-06-29T16:19:37Z

Steve Sharp:

This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [30] and Lloyd and Lipow [24]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds.
===Approximate Estimates of the Mean and Variance of a Function===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e. reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G</math> is presented next.

====Single Parameter Case====
For simplicity, consider a one-parameter distribution represented by a general function, <math>G,</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda </math>: <math>G(\lambda )=1/\lambda =\mu </math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)</math> can be found by:

::<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)</math>

where <math>G(\theta )</math> is some function of <math>\theta </math>, such as the reliability function, and <math>\theta </math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta </math> as <math>n\to \infty </math> . The term <math>O\left( \tfrac{1}{n} \right)</math> is a function of <math>n</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},</math> as <math>n\to \infty .</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}</math> and <math>G(x)=1/x</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}</math>. Thus as <math>n\to \infty </math>, <math>E(G(\widehat{\theta }))=\mu </math> where <math>\mu </math> and <math>\sigma </math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)</math> can then be estimated by:

::<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)</math>

====Two-Parameter Case====

Consider a Weibull distribution with two parameters <math>\beta </math> and <math>\eta </math>. For a given value of <math>T</math>, <math>R(T)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{T}{\eta } \right)}^{\beta }}}}</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)</math>, that:

::<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)</math>

and:

::<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Var(\widehat{\theta_{2}})\\

& +2{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\

& +O(\frac{1}{n^{\tfrac{3}{2}}})
\end{align}

</math>

Note that the derivatives of Eqn. (var) are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{1}},</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.</math>
Parameter Variance and Covariance Determination
The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

::<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}</math>

In the equation above, the first summation is for complete data, the second summation is for right censored data, and the third summation is for interval or left censored data. For more information on these data types, see Chapter 4.
Then the Fisher information matrix is given by:

::<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]</math>

The subscript <math>0</math> indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},</math> the true values of the parameters.
So for a sample of <math>N</math> units where <math>R</math> units have failed, <math>S</math> have been suspended, and <math>P</math> have failed within a time interval, and <math>N=R+M+P,</math> one could obtain the sample local information matrix by:

::<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}</math>

Substituting in the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}</math> and <math>{{\widehat{\theta }}_{2}}</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

::<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}</math>

Then the variance of a function (<math>Var(G)</math>) can be estimated using Eqn. (var). Values for the variance and covariance of the parameters are obtained from Eqn. (Fisher2).
Once they have been obtained, the approximate confidence bounds on the function are given as:

::<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha }}</math> next.

====Approximate Confidence Intervals on the Parameters====
In general, MLE estimates of the parameters are asymptotically normal, meaning for large sample sizes that a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }</math> is the MLE estimator for <math>\theta </math>, in the case of a single parameter distribution, estimated from a large sample of <math>n</math> units and if:

::<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}</math>

then using the normal distribution of <math>z\ \ :</math>

::<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

for large <math>n</math>. We now place confidence bounds on <math>\theta ,</math> at some confidence level <math>\delta </math>, bounded by the two end points <math>{{C}_{1}}</math> and <math>{{C}_{2}}</math> where:

::<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta </math>

From Eqn. (e729):

::<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta </math>

where <math>{{K}_{\alpha }}</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)</math>

Now by simplifying Eqn. (e731), one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta ,</math> at a confidence level <math>\delta ,</math> or:

::<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)</math>

The upper one-sided bounds are given by:

::<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

while the lower one-sided bounds are given by:

::<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

If <math>\widehat{\theta }</math> must be positive, then <math>\ln \widehat{\theta }</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta </math>, then become:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}</math>

The one-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta ,</math> can be found from:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [24] further elaborate on this procedure.

====Confidence Bounds on Time (Type 1)====
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e. y-ordinate or unreliability, <math>Q=1-R)</math> is determined by solving the unreliability function for the time, <math>T</math>, or:

::<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cb). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Note that the procedure is slightly more complicated for distributions with more than one parameter.

====Confidence Bounds on Reliability (Type 2)====
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

::<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cbr). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Once again, the procedure is more complicated for distributions with more than one parameter.

===Beta Binomial Confidence Bounds===
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see Chapter 4). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution.
Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z</math> (rank for the <math>{{j}^{th}}</math> failure):

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where <math>N</math> is the sample size and <math>j</math> is the order number.
The median rank was obtained by solving the following equation for <math>Z</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

The same methodology can then be repeated by changing <math>P</math> for <math>0.50</math> <math>(50%)</math> to our desired confidence level. For <math>P=90%</math> one would formulate the equation as

::<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

Keep in mind that one must be careful to select the appropriate values for <math>P</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P=0.1</math> and once with <math>P=0.9</math>) in order to place the bounds around 80% of the population.
Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.
This non-parametric methodology is only used by Weibull++ when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [20]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

===Likelihood Ratio Confidence Bounds===
====Introduction====
A third method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.
Likelihood ratio confidence bounds are based on the equation:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}</math>

where:
#<math>L(\theta )</math> is the likelihood function for the unknown parameter vector <math>\theta </math>
#<math>L(\widehat{\theta })</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }</math>
#<math>\chi _{\alpha ;k}^{2}</math> is the chi-squared statistic with probability <math>\alpha </math> and <math>k</math> degrees of freedom, where <math>k</math> is the number of quantities jointly estimated
If <math>\delta </math> is the confidence level, then <math>\alpha =\delta </math> for two-sided bounds and <math>\alpha =(2\delta -1)</math> for one-sided. Recall from Chapter 3 that if <math>x</math> is a continuous random variable with <math>pdf</math>:

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>,

where <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,{{x}_{R}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,R</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L.</math> These are represented by the <math>L(\widehat{\theta })</math> term in the denominator of the ratio in Eqn. (lratio1). Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }</math> have been calculated using MLE methods, the only unknown term in Eqn. (lratio1) is the <math>L(\theta )</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta </math> that satisfy Eqn. (lratio1). For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy Eqn. (lratio1). The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta ;</math> but at a given value of <math>\delta </math> there is only a certain region of values for <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> for which Eqn. (lratio1) holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of Eqn. (lratio1).

====Note on Contour Plots in Weibull++====
Contour plots can be used for comparing data sets. Consider two data sets, e.g. old and new design where the engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in a multiple plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e. the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If there is an overlap between the two 95% contours, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. Chapter 12 discusses comparing data sets.

====Confidence Bounds on the Parameters====
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}</math>

The task now becomes to find the values of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> so that the equality in Eqn. (lratio3) is satisfied. Unfortunately, there is no closed-form solution, thus these values must be arrived at numerically. One method of doing this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

=====Example 1=====
Five units were put on a reliability test and experienced failures at 10, 20, 30, 40, and 50 hours. Assuming a Weibull distribution, the MLE parameter estimates are calculated to be <math>\widehat{\beta }=2.2938</math> and <math>\widehat{\eta }=33.9428.</math> Calculate the 90% two-sided confidence bounds on these parameters using the likelihood ratio method.

=====Solution to Example 1=====
The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\widehat{\beta },\widehat{\eta })=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\widehat{\beta }}{\widehat{\eta }}\cdot {{\left( \frac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }-1}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{2.2938}{33.9428}\cdot {{\left( \frac{{{x}_{i}}}{33.9428} \right)}^{1.2938}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{33.9428} \right)}^{2.2938}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & 1.714714\times {{10}^{-9}}
\end{align}</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We then substitute this information into the equation:

::<math>\begin{align}
L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,\eta )-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,\eta )-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

The next step is to find the set of values of <math>\beta </math> and <math>\eta </math> that satisfy this equation, or find the values of <math>\beta </math> and <math>\eta </math> such that <math>L(\beta ,\eta )=4.432926\cdot {{10}^{-10}}.</math>

The solution is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>\eta </math>, and vice versa. The following table gives values of <math>\beta </math> based on given values of <math>\eta </math>.

These data are represented graphically in the following contour plot:

(Note that this plot is generated with degrees of freedom <math>k=1</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2</math>, for use in comparing both parameters simultaneously.) As can be determined from the table, the lowest calculated value for <math>\beta </math> is 1.142, while the highest is 3.950. These represent the two-sided 90% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>\eta </math> below 23 or above 50, these can be considered the 90% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>\eta </math>, we can perform the same procedure as before, but finding the two values of <math>\eta </math> that correspond with a given value of <math>\beta .</math> Using this method, we find that the 90% confidence limits on <math>\eta </math> are 22.474 and 49.967, which are close to the initial estimates of 23 and 50.
Note that the points where <math>\beta </math> are maximized and minimized do not necessarily correspond with the points where <math>\eta </math> are maximized and minimized. This is due to the fact that the contour plot is not symmetrical, so that the parameters will have their extremes at different points.

====Confidence Bounds on Time (Type 1)====
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.
=====Example 2=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the time estimate for a reliability of 50%. The ML estimate for the time at which <math>R(t)=50%</math> is 28.930.
=====Solution to Example 2=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the time estimate of 28.930. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>t</math> and <math>\beta .</math> This is accomplished by using a form of the Weibull reliability equation, <math>R={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}.</math> This can be rearranged in terms of <math>\eta </math>, with <math>R</math> being considered a known variable or:

::<math>\eta =\frac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}}</math>

This can then be substituted into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>t</math> and <math>\beta </math> or:

::<math>\begin{align}
& L(\beta ,t)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,t)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
& \\
L(\beta ,t)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

Note that the likelihood value for <math>L(\widehat{\beta },\widehat{\eta })</math> is the same as it was for Example 1. This is because we are dealing with the same data and parameter estimates or, in other words, the maximum value of the likelihood function did not change. It now remains to find the values of <math>\beta </math> and <math>t</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>t</math>. The following table gives the values of <math>t</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:
As can be determined from the table, the lowest calculated value for <math>t</math> is 17.389, while the highest is 41.714. These represent the 90% two-sided confidence limits on the time at which reliability is equal to 50%.

====Confidence Bounds on Reliability (Type 2)====
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta </math> and <math>R</math>. The likelihood function is once again altered in the same way as before, only now <math>R</math> is considered to be a parameter instead of <math>t</math>, since the value of <math>t</math> must be specified in advance. Once again, this process is best illustrated with an example.

=====Example 3=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the reliability estimate for <math>t=45</math>. The ML estimate for the reliability at <math>t=45</math> is 14.816%.

=====Solution to Example 3=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the reliability estimate of 14.816%. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>R</math> and <math>\beta .</math> This is again accomplished by substituting the Weibull reliability equation into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>R</math> and <math>\beta </math>:

::<math>\begin{align}
& L(\beta ,R)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

::<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

::<math>L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,R)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,R)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

It now remains to find the values of <math>\beta </math> and <math>R</math> that satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>R</math>. The following table gives the values of <math>R</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:

As can be determined from the table, the lowest calculated value for <math>R</math> is 2.38%, while the highest is 44.26%. These represent the 90% two-sided confidence limits on the reliability at <math>t=45</math>.

===Bayesian Confidence Bounds===
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in Chapter 3.
Bayesian confidence bounds are derived from Bayes rule, which states that:

::<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>f(\theta |Data)</math> is the <math>posterior</math> <math>pdf</math> of <math>\theta </math>
#<math>\theta </math> is the parameter vector of the chosen distribution (i.e. Weibull, lognormal, etc.)
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the <math>prior</math> <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>.
In other words, the prior knowledge is provided in the form of the prior <math>pdf</math> of the parameters, which in turn is combined with the sample data in order to obtain the posterior <math>pdf.</math> Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to Chapter 3). It can be seen from Eqn. (BayesRule) that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta }_{1}}</math>. Given a set of sample data, and a prior distribution for <math>{{\theta }_{1}},</math> <math>\varphi ({{\theta }_{1}}),</math> Eqn. (BayesRule) can be written as:

::<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

In other words, we now have the distribution of <math>{{\theta }_{1}}</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta }_{1}}</math> is less than or equal to a value <math>x,</math> <math>P({{\theta }_{1}}\le x)</math> can be obtained by integrating Eqn. (BayesEX), or:

::<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}</math>

Eqn. (IntBayes) essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)</math> is the confidence level and <math>x</math> is the confidence bound. Substituting Eqn. (BayesEX) into Eqn. (IntBayes) yields:

::<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

The only question at this point is what do we use as a prior distribution of <math>{{\theta }_{1}}.</math>. For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform.
Eqn. (BayesCLEX) can be generalized for any distribution having a vector of parameters <math>\theta ,</math> yielding the general equation for calculating Bayesian confidence bounds:

::<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>CL</math> is confidence level
#<math>\theta </math> is the parameter vector
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the prior <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>
#<math>\xi </math> is the range in which <math>\theta </math> changes from <math>\Psi (T,R)</math> till <math>{\theta }'s</math> maximum value or from <math>{\theta }'s</math> minimum value till <math>\Psi (T,R)</math>
#<math>\Psi (T,R)</math> is function such that if <math>T</math> is given then the bounds are calculated for <math>R</math> and if <math>R</math> is given, then he bounds are calculated for <math>T</math>.
If <math>T</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>R</math> are calculated.
If <math>R</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>T</math> are calculated.
====Confidence Bounds on Time (Type 1)====
For a given failure time distribution and a given reliability <math>R</math>, <math>T(R)</math> is a function of <math>R</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For the two-parameter Weibull distribution:

::<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)</math> is:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

where <math>f(T|Data,R)</math> is the posterior distribution of Time <math>T.</math>
Using Eqn. (T bayes), we have the following:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})</math>

Eqn. (cl) can be rewritten in terms of <math>\eta </math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))</math>

From Eqns. (IntBayes), (BayesCLEX) and (BayesCL), by assuming the priors of <math>\beta </math> and <math>\eta </math> are independent, we then obtain the following relationship:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl2) can be solved for <math>{{T}_{U}}(R)</math>, where:
#<math>CL</math> is confidence level,
#<math>\varphi (\beta )</math> is the prior <math>pdf</math> of the parameter <math>\beta </math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.</math>
#<math>\varphi (\eta )</math> is the prior <math>pdf</math> of the parameter <math>\eta .</math>. For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.</math>
#<math>L(\bullet )</math> is the likelihood function.
The same method can be used to get the one-sided lower bound of <math>T(R)</math> from:

::<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl5) can be solved to get <math>{{T}_{L}}(R)</math>.
The Bayesian two-sided bounds estimate for <math>T(R)</math> is:

::<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

which is equivalent to:

::<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

and:

::<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)</math> and <math>{{T}_{L}}(R)</math> can be solved.
====Confidence Bounds on Reliability (Type 2)====
For a given failure time distribution and a given time <math>T</math>, <math>R(T)</math> is a function of <math>T</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For example, for two parameter Weibull distribution:

::<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})</math>

The Bayesian one-sided upper bound estimate for <math>R(T)</math> is:

::<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

Similar with the bounds on Time, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl3) can be solved to get <math>{{R}_{U}}(T)</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

::<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR</math>

Using the posterior distribution, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl4) can be solved to get <math>{{R}_{L}}(T)</math>.
The Bayesian two-sided bounds estimate for <math>R(T)</math> is:

::<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

which is equivalent to:

::<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2</math>

and

::<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)</math> and <math>{{R}_{L}}(T)</math> can be solved.

===Simulation Based Bounds===
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds.
SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

=====Example 4=====
The purpose of this example is to determine the best parameter estimation method for a sample of ten units following a Weibull distribution with <math>\beta =2</math> and <math>\eta =100</math> and with complete time-to-failure data for each unit (i.e. no censoring). The number of generated data sets is set to 10,000. The SimuMatic inputs are shown next.

The parameters are estimated using RRX, RRY and MLE. The plotted results generated by SimuMatic are shown next.

Using RRX:

Using RRY:

Using MLE:

The results clearly demonstrate that the median RRX estimate provides the least deviation from the truth for this sample size and data type. However, the MLE outputs are grouped more closely together, as evidenced by the bounds. The previous figures also show the simulation-based bounds, as well as the expected variation due to sampling error.
This experiment can be repeated in SimuMatic using multiple censoring schemes (including Type I and Type II right censoring as well as random censoring) with various distributions. Multiple experiments can be performed with this utility to evaluate assumptions about the appropriate parameter estimation method to use for data sets.

Median Ranks

2011-06-29T16:08:26Z

Steve Sharp:

Median ranks are used to obtain an estimate of the unreliability, <math>Q({{T}_{j}}),</math> for each failure at a <math>50%</math> confidence level. In the case of grouped data, the ranks are estimated for each group of failures, instead of each failure.
For example, when using a group of 10 failures at 100 hours, 10 at 200 hours and 10 at 300 hours, Weibull++ estimates the median ranks (<math>Z</math> values) by solving the cumulative binomial equation with the appropriate values for order number and total number of test units.
For 10 failures at 100 hours, the median rank, <math>Z,</math> is estimated by using:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

with:

::<math>N=30,\text{ }J=10</math>

where one <math>Z</math> is obtained for the group, to represent the probability of 10 failures occurring out of 30.
For 10 failures at 200 hours, <math>Z</math> is estimated by using:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where:

::<math>N=30,\text{ }J=20</math>

to represent the probability of 20 failures out of 30.
For 10 failures at 300 hours, <math>Z</math> is estimated by using:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where:

::<math>N=30,\text{ }J=30</math>

to represent the probability of 30 failures out of 30.

Appendix: Log-Likelihood Equations

2011-06-29T16:06:35Z

Steve Sharp:

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

::<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

::<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

::<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

::<math>\lambda =0</math>

or:

::<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>{{F}_{i}}</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

::<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

::<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>Q</math> is the number of subpopulations

• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of groups of interval data points

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

::<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data group

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups,

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

::<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Appendix: Maximum Likelihood Estimation Example

2011-06-29T15:59:38Z

Steve Sharp: /* Illustrating the MLE Method Using the Exponential Distribution */

If <math>x</math> is a continuous random variable with <math>pdf\ \ :</math>

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}),</math>

where <math>{{\theta }_{1}},</math> <math>{{\theta }_{2}},</math><math>...,</math> <math>{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,</math> <math>{{x}_{N}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,N</math>

The logarithmic likelihood function is:

::<math>\Lambda =\ln L=\underset{i=1}{\overset{N}{\mathop \sum }}\,\ln f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L</math> or <math>\Lambda .</math>
By maximizing <math>\Lambda ,</math> which is much easier to work with than <math>L</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are the simultaneous solutions of <math>k</math> equations such that:

::<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>

Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other, and in no way suggests that the solution is wrong.

====Illustrating the MLE Method Using the Exponential Distribution====
*To estimate <math>\widehat{\lambda }</math> for a sample of <math>n</math> units (all tested to failure), first obtain the likelihood function:

::<math>\begin{align}
L(\lambda |{{t}_{1}},{{t}_{2}},...,{{t}_{n}})= & \underset{i=1}{\overset{n}{\mathop \prod }}\,f({{t}_{i}}) \\
= & \underset{i=1}{\overset{n}{\mathop \prod }}\,\lambda {{e}^{-\lambda {{t}_{i}}}} \\
= & {{\lambda }^{n}}\cdot {{e}^{-\lambda \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{t}_{i}}}}
\end{align}</math>

*Take the natural log of both sides:

::<math>\Lambda =\ln (L)=n\ln (\lambda )-\lambda \underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}.</math>

*Obtain <math>\tfrac{\partial \Lambda }{\partial \lambda }</math>, and set it equal to zero:

::<math>\frac{\partial \Lambda }{\partial \lambda }=\frac{n}{\lambda }-\underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}=0</math>

*Solve for <math>\widehat{\lambda }</math> or:

::<math>\hat{\lambda }=\frac{n}{\underset{i=1}{\overset{n}{\mathop{\sum }}}\,{{t}_{i}}}</math>

====Notes About <math>\widehat{\lambda }</math>====
Note that the value of <math>\widehat{\lambda }</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda </math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda </math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in Chapter 5.)

====Illustrating the MLE Method Using Normal Distribution====
To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},</math> for the normal distribution, start with the ::<math>pdf</math> of the normal distribution which is given by:

::<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

If <math>{{T}_{1}},{{T}_{2}},...,{{T}_{N}}</math> are known times-to-failure (and with no suspensions), then the likelihood function is given by:

::<math>L({{T}_{1}},{{T}_{2}},...,{{T}_{N}}|\bar{T},{{\sigma }_{T}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left[ \frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} \right]</math>

::<math>L=\frac{1}{{{({{\sigma }_{T}}\sqrt{2\pi })}^{N}}}{{e}^{-\tfrac{1}{2}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

then:

::<math>\Lambda =\ln L=-\frac{N}{2}\ln (2\pi )-N\ln {{\sigma }_{T}}-\frac{1}{2}\underset{i=1}{\overset{N}{\mathop \sum }}\,\left( \frac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)_{}^{2}</math>

Then taking the partial derivatives of <math>\Lambda </math> with respect to each one of the parameters and setting them equal to zero yields:

::<math>\frac{\partial (\Lambda )}{\partial \bar{T}}=\frac{1}{\sigma _{T}^{2}}\underset{i=1}{\overset{N}{\mathop \sum }}\,({{T}_{i}}-\bar{T})=0</math>

and:

::<math>\frac{\partial (\Lambda )}{\partial {{\sigma }_{T}}}=-\frac{N}{{{\sigma }_{T}}}+\frac{1}{\sigma _{T}^{3}}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}=0</math>

Solving Eqns. (dldt) and (dlds) simultaneously yields:

::<math>\bar{T}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

::<math>\begin{align}
& \hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
& & \\
& {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

It should be noted that these solutions are only valid for data with no suspensions, i.e. all units are tested to failure. In the case where suspensions are present or all units are not tested to failure, the methodology changes and the problem becomes much more complicated.

====Illustrating with an Example of the Normal Distribution====
If we had five units that failed at 10, 20, 30, 40 and 50 hours, the mean would be:

::<math>\begin{align}
\bar{T}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}} \\
= & \frac{10+20+30+40+50}{5} \\
= & 30
\end{align}</math>

The standard deviation estimate then would be:

::<math>\begin{align}
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \\
= & \sqrt{\frac{{{(10-30)}^{2}}+{{(20-30)}^{2}}+{{(30-30)}^{2}}+{{(40-30)}^{2}}+{{(50-30)}^{2}}}{5}}, \\
= & 14.1421
\end{align}</math>

A look at the likelihood function surface plot in Figure A-1 reveals that both of these values are the maximum values of the function.

This three-dimensional plot represents the likelihood function. As can be seen from the plot, the maximum likelihood estimates for the two parameters correspond with the peak or maximum of the likelihood function surface.

Appendix: Maximum Likelihood Estimation Example

2011-06-29T15:59:18Z

Steve Sharp:

If <math>x</math> is a continuous random variable with <math>pdf\ \ :</math>

::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}),</math>

where <math>{{\theta }_{1}},</math> <math>{{\theta }_{2}},</math><math>...,</math> <math>{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,</math> <math>{{x}_{N}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

::<math>i=1,2,...,N</math>

The logarithmic likelihood function is:

::<math>\Lambda =\ln L=\underset{i=1}{\overset{N}{\mathop \sum }}\,\ln f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L</math> or <math>\Lambda .</math>
By maximizing <math>\Lambda ,</math> which is much easier to work with than <math>L</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are the simultaneous solutions of <math>k</math> equations such that:

::<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>

Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other, and in no way suggests that the solution is wrong.

====Illustrating the MLE Method Using the Exponential Distribution====
*To estimate <math>\widehat{\lambda }</math> for a sample of <math>n</math> units (all tested to failure), first obtain the likelihood function:

::<math>\begin{align}
L(\lambda |{{t}_{1}},{{t}_{2}},...,{{t}_{n}})= & \underset{i=1}{\overset{n}{\mathop \prod }}\,f({{t}_{i}}) \\
= & \underset{i=1}{\overset{n}{\mathop \prod }}\,\lambda {{e}^{-\lambda {{t}_{i}}}} \\
= & {{\lambda }^{n}}\cdot {{e}^{-\lambda \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{t}_{i}}}}
\end{align}</math>

*Take the natural log of both sides:

::<math>\Lambda =\ln (L)=n\ln (\lambda )-\lambda \underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}.</math>

*Obtain <math>\tfrac{\partial \Lambda }{\partial \lambda }</math>, and set it equal to zero:

::<math>\frac{\partial \Lambda }{\partial \lambda }=\frac{n}{\lambda }-\underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}=0</math>

*Solve for <math>\widehat{\lambda }</math> or:

<math>\hat{\lambda }=\frac{n}{\underset{i=1}{\overset{n}{\mathop{\sum }}}\,{{t}_{i}}}</math>

====Notes About <math>\widehat{\lambda }</math>====
Note that the value of <math>\widehat{\lambda }</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda </math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda </math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in Chapter 5.)

====Illustrating the MLE Method Using Normal Distribution====
To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},</math> for the normal distribution, start with the ::<math>pdf</math> of the normal distribution which is given by:

::<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

If <math>{{T}_{1}},{{T}_{2}},...,{{T}_{N}}</math> are known times-to-failure (and with no suspensions), then the likelihood function is given by:

::<math>L({{T}_{1}},{{T}_{2}},...,{{T}_{N}}|\bar{T},{{\sigma }_{T}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left[ \frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} \right]</math>

::<math>L=\frac{1}{{{({{\sigma }_{T}}\sqrt{2\pi })}^{N}}}{{e}^{-\tfrac{1}{2}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

then:

::<math>\Lambda =\ln L=-\frac{N}{2}\ln (2\pi )-N\ln {{\sigma }_{T}}-\frac{1}{2}\underset{i=1}{\overset{N}{\mathop \sum }}\,\left( \frac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)_{}^{2}</math>

Then taking the partial derivatives of <math>\Lambda </math> with respect to each one of the parameters and setting them equal to zero yields:

::<math>\frac{\partial (\Lambda )}{\partial \bar{T}}=\frac{1}{\sigma _{T}^{2}}\underset{i=1}{\overset{N}{\mathop \sum }}\,({{T}_{i}}-\bar{T})=0</math>

and:

::<math>\frac{\partial (\Lambda )}{\partial {{\sigma }_{T}}}=-\frac{N}{{{\sigma }_{T}}}+\frac{1}{\sigma _{T}^{3}}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}=0</math>

Solving Eqns. (dldt) and (dlds) simultaneously yields:

::<math>\bar{T}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

::<math>\begin{align}
& \hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
& & \\
& {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

It should be noted that these solutions are only valid for data with no suspensions, i.e. all units are tested to failure. In the case where suspensions are present or all units are not tested to failure, the methodology changes and the problem becomes much more complicated.

====Illustrating with an Example of the Normal Distribution====
If we had five units that failed at 10, 20, 30, 40 and 50 hours, the mean would be:

::<math>\begin{align}
\bar{T}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}} \\
= & \frac{10+20+30+40+50}{5} \\
= & 30
\end{align}</math>

The standard deviation estimate then would be:

::<math>\begin{align}
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \\
= & \sqrt{\frac{{{(10-30)}^{2}}+{{(20-30)}^{2}}+{{(30-30)}^{2}}+{{(40-30)}^{2}}+{{(50-30)}^{2}}}{5}}, \\
= & 14.1421
\end{align}</math>

A look at the likelihood function surface plot in Figure A-1 reveals that both of these values are the maximum values of the function.

This three-dimensional plot represents the likelihood function. As can be seen from the plot, the maximum likelihood estimates for the two parameters correspond with the peak or maximum of the likelihood function surface.

Appendix E: References

2011-06-29T15:09:27Z

Steve Sharp: Created page with '1) Aitchison, J., Jr. and Brown, J.A.C., The Lognormal Distribution, Cambridge University Press, New York, 176 pp., 1957. 2) Cramer, H., Mathematical Methods of Statist…'

1) Aitchison, J., Jr. and Brown, J.A.C., The Lognormal Distribution, Cambridge University Press, New York, 176 pp., 1957.
 
 
2) Cramer, H., Mathematical Methods of Statistics, Princeton University Press, Princeton, NJ, 1946.
 
 
3) Davis, D.J., An Analysis of Some Failure Data, J. Am. Stat. Assoc., Vol. 47, p. 113, 1952.
 
 
4) Dietrich, D., SIE 530 Engineering Statistics Lecture Notes, The University of Arizona, Tucson, Arizona.
 
 
5) Dudewicz, E.J., An Analysis of Some Failure Data, J. Am. Stat. Assoc., Vol. 47, p. 113, 1952.
 
 
6) Dudewicz, E.J., and Mishra, Satya N., Modern Mathematical Statistics, John Wiley & Sons, Inc., New York, 1988.
 
 
7) Evans, Ralph A., The Lognormal Distribution is Not a Wearout Distribution, Reliability Group Newsletter, IEEE, Inc., 345 East 47 St., New York, N.Y. 10017, p. 9, Vol. XV, Issue 1, January 1970.
 
 
8) Gottfried, Paul, Wear-out, Reliability Group Newsletter, IEEE, Inc., 345 East 47 St., New York, N.Y. 10017, p. 7, Vol. XV, Issue 3, July 1970.
 
 
9) Glasstone, S., Laidler, K. J., and Eyring, H. E., The Theory of Rate Processes, McGraw Hill, NY, 1941.
 
 
10) Hahn, Gerald J., and Shapiro, Samuel S., Statistical Models in Engineering, John Wiley & Sons, Inc., New York, 355 pp., 1967.
 
 
11) Hald, A., Statistical Theory with Engineering Applications, John Wiley & Sons, Inc., New York, 783 pp., 1952.
 
 
12) Hald, A., Statistical Tables and Formulas, John Wiley & Sons, Inc., New York, 97 pp., 1952.
 
 
13) Hirose, Hideo, Maximum Likelihood Estimation in the 3-parameter Weibull Distribution - A Look through the Generalized Extreme-value Distribution IEEE Transactions on Dielectrics and Electrical Insulation, Vol. 3, No. 1, pp. 43-55, February 1996.
 
 
14) Johnson, Leonard G., The Median Ranks of Sample Values in their Population With an Application to Certain Fatigue Studies, Industrial Mathematics, Vol. 2, 1951.
 
 
15) Johnson, Leonard G., The Statistical Treatment of Fatigue Experiment, Elsevier Publishing Company, New York, 144 pp., 1964.
 
 
16) Kao, J.H.K., A New Life Quality Measure for Electron Tubes, IRE Transaction on Reliability and Quality Control, PGRQC 13, pp. 15-22, July 1958.
 
 
17) Kapur, K.C., and Lamberson, L.R., Reliability in Engineering Design, John Wiley & Sons, Inc., New York, 586 pp., 1977.
 
 
18) Kececioglu, Dimitri, Reliability Engineering Handbook, Prentice Hall, Inc., New Jersey, Vol. 1, 1991.
 
 
19) Kececioglu, Dimitri, Reliability & Life Testing Handbook, Prentice Hall, Inc., New Jersey, Vol. 1 and 2, 1993 and 1994.
 
 
20) Kececioglu, Dimitri, and Sun, Feng-Bin, Environmental Stress Screening - Its Quantification, Optimization and Management, Prentice Hall PTR, New Jersey, 1995.
 
 
21) Kececioglu, Dimitri, and Sun, Feng-Bin, Burn-In Testing - Its Quantification and Optimization, Prentice Hall PTR, New Jersey, 1997.
 
 
22) Leemis Lawrence M., Reliability - Probabilistic Models and Statistical Methods, Prentice Hall, Inc., Englewood Cliffs, New Jersey, 1995.
 
 
23) Lieblein, J., and Zelen, M., Statistical Investigation of the Fatigue Life of Deep-Groove Ball Bearings, Journal of Research, National Bereau of Standards, Vol. 57, p. 273, 1956.
 
 
24) Lloyd, David K., and Lipow Myron, Reliability: Management Methods and Mathematics, 1962, Prentice Hall, Englewood Cliffs, New Jersey.
 
 
25) Mann, Nancy R., Schafer, Ray. E., and Singpurwalla, Nozer D., Methods for Statistical Analysis of Reliability and Life Data, John Wiley & Sons, Inc., New York, 1974.
 
 
26) Meeker, William Q., and Escobar, Luis A., Statistical Methods for Reliability Data, John Wiley & Sons, Inc., New York, 1998.
 
 
27) Nelson, Wayne, Applied Life Data Analysis, John Wiley & Sons, Inc., New York, 1982.
 
 
28) Nelson, Wayne, Accelerated Testing: Statistical Models Test Plans and Data Analyses John Wiley & Sons, Inc., New York, 1990.
 
 
29) Perry, J. N., Semiconductor Burn-in and Weibull Statistics, Semiconductor Reliability, Vol. 2, Engineering Publishers, Elizabeth, N.J., pp. 8-90, 1962.
 
 
30) Procassini, A. A., and Romano, A., Transistor Reliability Estimates Improve with Weibull Distribution Function, Motorola Military Products Division, Engineering Bulletin, Vol. 9, No. 2, pp. 16-18, 1961.
 
 
31) ReliaSoft Corporation, Life Data Analysis Reference, ReliaSoft Publishing, Tucson, AZ, 2000.
 
 
32) Striny, Kurt M., and Schelling, Arthur W., Reliability Evaluation of Aluminum-Metalized MOS Dynamic RAMS in Plastic Packages in High Humidity and Temperature Environments, IEEE 31 Electronic Components Conference, pp. 238-244, 1981.
 
 
33) Weibull, Wallodi, A Statistical Representation of Fatigue Failure in Solids, Transactions on the Royal Institute of Technology, No. 27, Stockholm, 1949.
 
 
34) Weibull, Wallodi, A Statistical Distribution Function of Wide Applicability, Journal of Applied Mechanics, Vol. 18, pp. 293-297, 1951.
 
 
35) Wingo, Dallas R., Solution of the Three-Parameter Weibull Equations by Constrained Modified Quasilinearization Progressively Censored Samples , IEEE Transactions on Reliability, Vol. R-22, No. 2, pp. 96-100, June 1973.
 
 
36) Meeker, William Q., and Hahn, Gerlad J., Volume 1: How To Plan An Accelerated Life Test - Some Practical Guidelines, American Society For Quality Control, Milwaukee, 1985.
 
 
37) Escobar, Luis A. and Meeker, William Q., Planning Accelerated Life Tests with Two or More Experimental Factors, Technometrics, Vol. 37, No. 4, pp. 411-427, 1995.
 
 
38) Meeker, William Q., A Comparison of Accelerated Life Test Plans for Weibull and Lognormal Distributions and Type I Censoring, Technometrics, Vol. 26, No. 2, pp. 157-171, 1984.

Appendix C: Benchmark Examples

2011-06-29T15:07:03Z

Steve Sharp:

In this section, five published examples are presented for comparison purposes. ReliaSoft's R&D validated the ALTA software with hundreds of data sets and methods. ALTA also cross-validates each provided solution by independently re-evaluating the second partial derivatives based on the estimated parameters each time a calculation is performed. These partials will be equal to zero when a solution is reached. Double precision is used throughout ALTA.

=Sections=
#[[New22 Example 1]]
#[[New22 Example 2]]
#[[New22 Example 3]]
#[[New22 Example 4]]
#[[New22 Example 5]]

Appendix C: Benchmark Examples

2011-06-29T15:06:48Z

Steve Sharp:

In this section, five published examples are presented for comparison purposes. ReliaSoft's R&D validated the ALTA software with hundreds of data sets and methods. ALTA also cross-validates each provided solution by independently re-evaluating the second partial derivatives based on the estimated parameters each time a calculation is performed. These partials will be equal to zero when a solution is reached. Double precision is used throughout ALTA.

=Sections=
#[[New21 Example 1]]
#[[New21 Example 2]]
#[[New210 Example 3]]
#[[New21 Example 4]]
#[[New210 Example 5]]

Appendix C: Benchmark Examples

2011-06-29T15:06:33Z

Steve Sharp:

In this section, five published examples are presented for comparison purposes. ReliaSoft's R&D validated the ALTA software with hundreds of data sets and methods. ALTA also cross-validates each provided solution by independently re-evaluating the second partial derivatives based on the estimated parameters each time a calculation is performed. These partials will be equal to zero when a solution is reached. Double precision is used throughout ALTA.

=Sections=
#[[New21 Example 1]]
#[[New21 Example 2]]
#[[New22 Example 3]]
#[[New21 Example 4]]
#[[New22 Example 5]]

Appendix C: Benchmark Examples

2011-06-29T15:06:20Z

Steve Sharp:

In this section, five published examples are presented for comparison purposes. ReliaSoft's R&D validated the ALTA software with hundreds of data sets and methods. ALTA also cross-validates each provided solution by independently re-evaluating the second partial derivatives based on the estimated parameters each time a calculation is performed. These partials will be equal to zero when a solution is reached. Double precision is used throughout ALTA.

=Sections=
#[[New21 Example 1]]
#[[New21 Example 2]]
#[[New21 Example 3]]
#[[New21 Example 4]]
#[[New21 Example 5]]

Appendix C: Benchmark Examples

2011-06-29T15:05:48Z

Steve Sharp: Created page with 'In this section, five published examples are presented for comparison purposes. ReliaSoft's R&D validated the ALTA software with hundreds of data sets and methods. ALTA also cros…'

In this section, five published examples are presented for comparison purposes. ReliaSoft's R&D validated the ALTA software with hundreds of data sets and methods. ALTA also cross-validates each provided solution by independently re-evaluating the second partial derivatives based on the estimated parameters each time a calculation is performed. These partials will be equal to zero when a solution is reached. Double precision is used throughout ALTA.

=Sections=
#[[Example 1]]
#[[Example 2]]
#[[Example 3]]
#[[Example 4]]
#[[Example 5]]

Appendix B: Parameter Estimation

2011-06-29T00:06:58Z

Steve Sharp: Created page with 'Once a life distribution and a life-stress relationship have been selected, the parameters (i.e. the variables that govern the characteristics of the ) need to be determined. Sev…'

Once a life distribution and a life-stress relationship have been selected, the parameters (i.e. the variables that govern the characteristics of the ) need to be determined. Several parameter estimation methods, including probability plotting, least squares, and maximum likelihood, are available. This appendix will present an overview of these methods. Because the least squares method for analyzing accelerated life data is very limiting, it will be covered very briefly in this appendix. Interested readers can refer to Nelson [28] for a more detailed discussion of the least squares parameter estimation method.
=Sections=
#[[Graphical Method]]
#[[Maximum Likelihood (MLE) Parameter Estimation]]
#[[MLE of Accelerated Life Data]]
#[[Analysis of Censored Data]]
#[[Conclusions]]

Appendix A: Brief Statistical Background

2011-06-29T00:02:45Z

Steve Sharp:

=Sections=
#[[Brief Statistical Background]]
#[[New21 Distributions]]
#[[Confidence Intervals (or Bounds)]]
#[[Confidence Limits Determination]]

Appendix A: Brief Statistical Background

2011-06-29T00:02:34Z

Steve Sharp: Created page with '=Sections= #Brief Statistical Background #Distributions #Confidence Intervals (or Bounds) #Confidence Limits Determination'

=Sections=
#[[Brief Statistical Background]]
#[[Distributions]]
#[[Confidence Intervals (or Bounds)]]
#[[Confidence Limits Determination]]

Probability Plotting

2011-06-28T21:45:31Z

Steve Sharp: Created page with 'One method of calculating the parameter of the exponential distribution is by using probability plotting. To better illustrate this procedure, consider the following example. <b…'

One method of calculating the parameter of the exponential distribution is by using probability plotting. To better illustrate this procedure, consider the following example.

 

====Example 1====
 

Let's assume six identical units are reliability tested at the same application and operation
stress levels. All of these units fail during the test after operating for the following times (in hours), <math>{{T}_{i}}</math> : 96, 257, 498, 763, 1051 and 1744.
 
The steps for determining the parameters of the exponential <math>pdf</math> representing the
data, using probability plotting, are as follows:
 
• Rank the times-to-failure in ascending order as shown next.
 

::<math>\begin{matrix}
\text{Time-to-} & \text{Failure Order Number} \\
\text{failure, hr} & \text{out of a Sample Size of 6} \\
\text{96} & \text{1} \\
\text{257} & \text{2} \\
\text{498} & \text{3} \\
\text{763} & \text{4} \\
\text{1,051} & \text{5} \\
\text{1,744} & \text{6} \\
\end{matrix}</math>

 
• Obtain their median rank plotting positions.
 
Median rank positions are used instead of other ranking methods because median ranks are at a
specific confidence level (50%).
 
• The times-to-failure, with their corresponding median ranks, are shown next:

 
::<math>\begin{matrix}
\text{Time-to-} & \text{Median} \\
\text{failure, hr} & \text{Rank, }% \\
\text{96} & \text{10}\text{.91} \\
\text{257} & \text{26}\text{.44} \\
\text{498} & \text{42}\text{.14} \\
\text{763} & \text{57}\text{.86} \\
\text{1,051} & \text{73}\text{.56} \\
\text{1,744} & \text{89}\text{.10} \\
\end{matrix}</math>

 
• On an exponential probability paper, plot the times on the x-axis and their corresponding
rank value on the y-axis. Fig. 4 displays an example of an exponential probability paper. The
paper is simply a log-linear paper. (The solution is given in Fig. 2.)
 
[[File:ALTA4.1.gif|center]]
 
::Fig. 4: Sample exponential probability paper.
 
• Draw the best possible straight line that goes through the <math>t=0</math> and <math>
(t)=100%</math> point and through the plotted points (as shown in Fig. 5).
 
• At the <math>Q(t)=63.2%</math> or <math>R(t)=36.8%</math> ordinate point, draw a
straight horizontal line until this line intersects the fitted straight line. Draw a vertical line through this intersection until it crosses the abscissa. The value at the intersection of the abscissa is the estimate of the mean. For this case, <math>\widehat{\mu }=833</math> hr which means that <math>\lambda =\tfrac{1}{\mu }=0.0012</math> . (This is always at 63.2% since <math>(T)=1-{{e}^{-\tfrac{\mu }{\mu }}}=1-{{e}^{-1}}=0.632=63.2%).</math>
 
[[File:ALTA4.2.gif|center]]
 
::Fig. 5: Probability plot for Example 1.
 
 
Now any reliability value for any mission time <math>t</math> can be obtained. For example, the
reliability for a mission of 15 hr, or any other time, can now be obtained either from the plot or analytically (i.e. using the equations given in Section <math>5.1.1</math> ).

 
To obtain the value from the plot, draw a vertical line from the abscissa, at <math>t=15</math>
hr, to the fitted line. Draw a horizontal line from this intersection to the ordinate and read
<math>R(t)</math> . In this case, <math>R(t=15)=98.15%</math> . This can also be obtained
analytically, from the exponential reliability function.

 

====MLE Parameter Estimation====

 
The parameter of the exponential distribution can also be estimated using the maximum likelihood estimation (MLE) method. This log-likelihood function is:

 
::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\overset{FI}{\mathop{\underset{i=1}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{\prime \prime }\ln [R_{Li}^{\prime \prime }-R_{Ri}^{\prime \prime }]</math>
 
where:
 
::<math>R_{Li}^{\prime \prime }={{e}^{-\lambda T_{Li}^{\prime \prime }}}</math>

 
::<math>R_{Ri}^{\prime \prime }={{e}^{-\lambda T_{Ri}^{\prime \prime }}}</math>

 
and:
 
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points.
 
• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group.
 
• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found).
 
• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data.
 
• <math>S</math> is the number of groups of suspension data points.
 
• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points.
 
• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group.
 
• <math>FI</math> is the number of interval data groups.
 
• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the i <math>^{th}</math> group of data intervals.
 
• <math>T_{Li}^{\prime \prime }</math> is the beginning of the i <math>^{th}</math> interval.
 
• <math>T_{Ri}^{\prime \prime }</math> is the ending of the i <math>^{th}</math> interval.
 
 
The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0</math> where:
 
::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime }-\overset{FI}{\mathop{\underset{i=1}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{\prime \prime }\frac{T_{Li}^{\prime \prime }R_{Li}^{\prime \prime }-T_{Ri}^{\prime \prime }R_{Ri}^{\prime \prime }}{R_{Li}^{\prime \prime }-R_{Ri}^{\prime \prime }}</math>

 

====Example 2====

 
Using the same data as in the probability plotting example (Example 1), and assuming an exponential distribution, estimate the parameter using the MLE method.
 
''Solution''
 
In this example we have non-grouped data without suspensions. Thus Eqn. (exp-mle) becomes:
 
::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}} \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}} \right) \right]=0</math>

 
Substituting the values for <math>T</math> we get:

 
::<math>\begin{align}
& \frac{6}{\lambda }= & 4409,\text{ or:} \\
& \lambda = & 0.00136\text{ failure/hr}
\end{align}</math>

Mixed Weibull Distribution

2011-06-28T17:09:38Z

Steve Sharp: Created page with 'The mixed Weibull distribution (also known as a multimodal Weibull) is used to model data that do not fall on a straight line on a Weibull probability plot. Data of this type, pa…'

The mixed Weibull distribution (also known as a multimodal Weibull) is used to model data that do not fall on a straight line on a Weibull probability plot. Data of this type, particularly if the data points follow an S-shape on the probability plot, may be indicative of more than one failure mode at work in the population of failure times. Field data from a given mixed population may frequently represent multiple failure modes. The necessity of determining the life regions where these failure modes occur is apparent when it is realized that the times-to-failure for each mode may follow a distinct Weibull distribution, thus requiring individual mathematical treatment. Another reason is that each failure mode may require a different design change to improve the component's reliability [19].

A decreasing failure rate is usually encountered during the early life period of components when the substandard components fail and are removed from the population. The failure rate continues to decrease until all such substandard components fail and are removed. This corresponds to a decreasing failure rate. The Weibull distribution having <math>\beta <1</math> is often used to depict this life characteristic.

A second type of failure prevails when the components fail by chance alone and their failure rate is nearly constant. This can be caused by sudden, unpredictable stress applications that have a stress level above those to which the product is designed. Such failures tend to occur throughout the life of a component. The distributions most often used to describe this failure rate characteristic are the exponential distribution and the Weibull distribution with <math>\beta \approx 1</math> .

A third type of failure is characterized by a failure rate that increases as operating hours are accumulated. Usually, wear has started to set in and this brings the component's performance out of specification. As age increases further, this wear-out process removes more and more components until all components fail. The normal distribution and the Weibull distribution with a <math>\beta >1</math> have been successfully used to model the times-to-failure distribution during the wear-out period.

Several different failure modes may occur during the various life periods. A methodology is needed to identify these failure modes and determine their failure distributions and reliabilities. This section presents a procedure whereby the proportion of units failing in each mode is determined and their contribution to the reliability of the component is quantified. From this reliability expression, the remaining major reliability functions, the probability density, the failure rate and the conditional-reliability functions are calculated to complete the reliability analysis of such mixed populations.

Lognormal Statistical Properties

2011-06-28T17:02:13Z

Steve Sharp: Created page with '====The Mean or MTTF==== The mean of the lognormal distribution, <math>\mu </math> , is given by [18]: <math>\mu ={{e}^{{\mu }'+\tfrac{1}{2}\sigma _{{{T}'}}^{2}}}</math> The…'

====The Mean or MTTF====
The mean of the lognormal distribution, <math>\mu </math> , is given by [18]:

<math>\mu ={{e}^{{\mu }'+\tfrac{1}{2}\sigma _{{{T}'}}^{2}}}</math>

The mean of the natural logarithms of the times-to-failure, <math>\mu'</math> , in terms of <math>\bar{T}</math> and <math>{{\sigma }_{T}}</math> is givgen by:

<math>{\mu }'=\ln \left( {\bar{T}} \right)-\frac{1}{2}\ln \left( \frac{\sigma _{T}^{2}}{{{{\bar{T}}}^{2}}}+1 \right)</math>

====The Median====
The median of the lognormal distribution, <math>\breve{T}</math> , is given by [18]:

<math>\breve{T}={{e}^{{{\mu }'}}}</math>

====The Mode====
The mode of the lognormal distribution, <math>\tilde{T}</math> , is given by [1]:

<math>\tilde{T}={{e}^{{\mu }'-\sigma _{{{T}'}}^{2}}}</math>

====The Standard Deviation====
The standard deviation of the lognormal distribution, <math>{{\sigma }_{T}}</math> , is given by [18]:

<math>{{\sigma }_{T}}=\sqrt{\left( {{e}^{2{\mu }'+\sigma _{{{T}'}}^{2}}} \right)\left( {{e}^{\sigma _{{{T}'}}^{2}}}-1 \right)}</math>

The standard deviation of the natural logarithms of the times-to-failure, <math>{{\sigma }_{{{T}'}}}</math> , in terms of <math>\bar{T}</math> and <math>{{\sigma }_{T}}</math> is given by:

<math>{{\sigma }_{{{T}'}}}=\sqrt{\ln \left( \frac{\sigma _{T}^{2}}{{{{\bar{T}}}^{2}}}+1 \right)}</math>

====The Lognormal Reliability Function====
The reliability for a mission of time <math>T</math> , starting at age 0, for the lognormal distribution is determined by:

<math>R(T)=\int_{T}^{\infty }f(t)dt</math>

or:

<math>R(T)=\int_{{{T}^{^{\prime }}}}^{\infty }\frac{1}{{{\sigma }_{{{T}'}}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{t-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}^{2}}}}dt</math>

As with the normal distribution, there is no closed-form solution for the lognormal reliability function. Solutions can be obtained via the use of standard normal tables. Since the application automatically solves for the reliability we will not discuss manual solution methods. For interested readers, full explanations can be found in the references.

Bayesian-Weibull Analysis

2011-06-28T16:54:15Z

Steve Sharp: Created page with 'In this section, the Bayesian methods are presented for the two-parameter Weibull distribution. Bayesian concepts were introduced in Chapter 3. This model considers prior knowled…'

In this section, the Bayesian methods are presented for the two-parameter Weibull distribution. Bayesian concepts were introduced in Chapter 3. This model considers prior knowledge on the shape (β) parameter of the Weibull distribution when it is chosen to be fitted to a given set of data. There are many practical applications for this model, particularly when dealing with small sample sizes and some prior knowledge for the shape parameter is available. For example, when a test is performed, there is often a good understanding about the behavior of the failure mode under investigation, primarily through historical data. At the same time, most reliability tests are performed on a limited number of samples. Under these conditions, it would be very useful to use this prior knowledge with the goal of making more accurate predictions. A common approach for such scenarios is to use the one-parameter Weibull distribution, but this approach is too deterministic, too absolute you may say (and you would be right). The Weibull-Bayesian model in Weibull++ (which is actually a true "WeiBayes" model, unlike the one-parameter Weibull that is commonly referred to as such) offers an alternative to the one-parameter Weibull, by including the variation and uncertainty that might have been observed in the past on the shape parameter. Applying Bayes's rule on the two-parameter Weibull distribution and assuming the prior distributions of β and η are independent, we obtain the following posterior :

<math> f(\beta ,\eta |Data)=\dfrac{L(\beta ,\eta )\varphi (\beta )\varphi (\eta )}{ \int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta } </math> EQNREF WeibBayes

In this model, η is assumed to follow a noninformative prior distribution with the density function <math> \varphi (\eta )=\dfrac{1}{\eta } </math>. This is called Jeffrey's prior, and is obtained by performing a logarithmic transformation on η. Specifically, since η is always positive, we can assume that ln(η) follows a uniform distribution, ''U''( − ∞, + ∞). Applying Jeffrey's rule [9] which says "in general, an approximate non-informative prior is taken proportional to the square root of Fisher's information", yields <math> \varphi (\eta )=\dfrac{1}{\eta }. </math>

The prior distribution of β, denoted as <math> \varphi (\beta ) </math>, can be selected from the following distributions: normal, lognormal, exponential and uniform. The procedure of performing a Weibull-Bayesian analysis is as follows:

*Collect the times-to-failure data.
*Specify a prior distribution for β (the prior for η is assumed to be 1/η).
*Obtain the posterior from Eqn. (EQNREF WeibBayes ).

In other words, a distribution (the posterior ) is obtained, rather than a point estimate as in classical statistics (i.e., as in the parameter estimation methods described previously in this chapter). Therefore, if a point estimate needs to be reported, a point of the posterior needs to be calculated. Typical points of the posterior distribution used are the mean (expected value) or median. In Weibull++, both options are available and can be chosen from the ''Analysis'' page, under the ''Results As'' area, as shown next.

The expected value of β is obtained by:

<math> E(\beta )=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\beta \cdot f(\beta ,\eta |Data)d\beta d\eta </math>

Similarly, the expected value of η is obtained by:

<math> E(\eta )=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\eta \cdot f(\beta ,\eta |Data)d\beta d\eta </math>

The median points are obtained by solving the following equations for <math> \breve{\beta} </math> and <math> \breve{\eta} </math> respectively:

<math> \int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\breve{\beta}}f(\beta ,\eta |Data)d\beta d\eta =0.5 </math>

and

<math> \int\nolimits_{0}^{\breve{\eta}}\int\nolimits_{0}^{\infty }f(\beta ,\eta |Data)d\beta d\eta =0.5 </math>

Of course, other points of the posterior distribution can be calculated as well. For example, one may want to calculate the 10th percentile of the joint posterior distribution (w.r.t. one of the parameters). The procedure for obtaining other points of the posterior distribution is similar to the one for obtaining the median values, where instead of 0.5 the percentage of interest is given. This procedure actually provides the confidence bounds on the parameters, which in the Bayesian framework are called ‘‘Credible Bounds‘‘. However, since the engineering interpretation is the same, and to avoid confusion, we refer to them as confidence bounds in this reference and in Weibull++.

== Posterior Distributions for Functions of Parameters ==

As explained in Chapter 3, in Bayesian analysis, all the functions of the parameters are distributed. In other words, a posterior distribution is obtained for functions such as reliability and failure rate, instead of point estimate as in classical statistics. Therefore, in order to obtain a point estimate for these functions, a point on the posterior distributions needs to be calculated. Again, the expected value (mean) or median value are used.

===<math>pdf</math> of the Times-to-Failure ===

The posterior distribution of the failure time is given by:

<math> f(T|Data)=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }f(T,\beta ,\eta )f(\beta ,\eta |Data)d\eta d\beta </math> EQNREF WeibBayesPDF

where:

<math> f(T,\beta ,\eta )=\dfrac{\beta }{\eta }\left( \dfrac{T}{\eta }\right) ^{\beta -1}e^{-\left( \dfrac{T}{\eta }\right) ^{\beta }} </math>

For the <math>pdf</math> of the times-to-failure, only the expected value is calculated and reported in Weibull++.

=== Reliability ===

In order to calculate the median value of the reliability function, we first need to obtain posterior of the reliability. Since ''R''(''T'') is a function of β, the density functions of β and ''R''(''T'') have the following relationship:

<math> \begin{align} f(R|Data,T)dR = & f(\beta |Data)d\beta)\\
= & (\int\nolimits_{0}^{\infty }f(\beta ,\eta |Data)d{\eta}) d{\beta} \\
=& \dfrac{\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }d\beta
\end{align}
</math> EQNREF Rpdf

The median value of the reliability is obtained by solving the following equation w.r.t. <math> \breve{R}: </math>

<math> \int\nolimits_{0}^{\breve{R}}f(R|Data,T)dR=0.5 </math> EQNREF MedRel

The expected value of the reliability at time is given by:

<math> R(T|Data)=\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }R(T,\beta ,\eta )f(\beta ,\eta |Data)d\eta d\beta </math> where:

<math> R(T,\beta ,\eta )=e^{-\left( \dfrac{T}{\eta }\right) ^{^{\beta }}} </math>

 

=== Failure Rate ===

The failure rate at time is given by:

<math> \lambda (T|Data)=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }\lambda (T,\beta ,\eta )L(\beta ,\eta )\varphi (\eta )\varphi (\beta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\eta )\varphi (\beta )d\eta d\beta } </math>

where:

<math> \lambda (T,\beta ,\eta )=\dfrac{\beta }{\eta }\left( \dfrac{T}{\eta }\right) ^{\beta -1} </math>

 

== Note on Calculated Results ==

As mentioned above, in order to obtain point estimates for the parameters of functions of the parameters in Bayesian analysis, the Median or Mean values of the different posterior <math>pdf</math>s are calculated. It is important to note that the Median value is preferable and is the default in Weibull++. This is because the Median value always corresponds to the 50th percentile of the distribution. On the other hand, the Mean is not a fixed point on the distribution, which could cause issues, especially when comparing results across different data sets.

== Confidence Bounds on ''R''(''T'') ==

The confidence bounds calculation under the Weibull-Bayesian analysis is very similar to the Bayesian Confidence Bounds method described in the previous section, with the exception that in the case of the Weibull-Bayesian Analysis the specified prior of β is considered instead of an non-informative prior. The Bayesian one-sided upper bound estimate for ''R''(''T'') is given by:

<math> \int\nolimits_{0}^{R_{U}(T)}f(R|Data,T)dR=CL </math>

Using Eqns. (EQNREF WeibBayes ) and (EQNREF Rpdf ) the following is obtained:

<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{T\exp (-\dfrac{\ln (-\ln R_{U})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLRU

Eqn. (EQNREF 1CLRU ) can be solved for ''R''''U''(''T''). The Bayesian one-sided lower bound estimate for <math> \ R(T) </math> is given by:

<math> \int\nolimits_{0}^{R_{L}(T)}f(R|Data,T)dR=1-CL </math>

Using Eqns. (EQNREF WeibBayes ) and (EQNREF Rpdf ) the following is obtained:

<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T\exp (-\dfrac{\ln (-\ln R_{L})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=1-CL </math> EQNREF 1CLRL

Eqn. (EQNREF 1CLRL ) can be solved for ''R''''L''(''T''). The Bayesian two-sided bounds estimate for ''R''(''T'') is given by:

<math> \int\nolimits_{R_{L}(T)}^{R_{U}(T)}f(R|Data,T)dR=CL </math> which is equivalent to:

<math> \int\nolimits_{0}^{R_{U}(T)}f(R|Data,T)dR=(1+CL)/2 </math>

and

<math> \int\nolimits_{0}^{R_{L}(T)}f(R|Data,T)dR=(1-CL)/2 </math>

Using the same method for one-sided bounds, ''R''''U''(''T'')and ''R''''L''(''T'') can be computed.

== Confidence Bounds on Time ==

Following the same procedure described for bounds on Reliability, the bounds of time can be calculated, given . The Bayesian one-sided upper bound estimate for ''T''(''R'') is given by:

<math> \int\nolimits_{0}^{T_{U}(R)}f(T|Data,R)dT=CL </math>

Using Eqns. (EQNREF WeibBayes ) and. (EQNREF WeibBayesPDF ), we obtain:

<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T_{U}\exp (-\dfrac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLTU

Eqn. (EQNREF 1CLTU ) can be solved for ''T''''U''(''R''). The Bayesian one-sided lower bound estimate for ''T''(''R'') is given by:

<math> \int\nolimits_{0}^{T_{L}(R)}f(T|Data,R)dT=1-CL </math>

or:

<math> \dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{T_{L}\exp (\dfrac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }=CL </math> EQNREF 1CLTL

Eqn. (EQNREF 1CLTL ) can be solved for ''T''''L''(''R''). The Bayesian two-sided lower bounds estimate for ''T''(''R'') is:

<math> \int\nolimits_{T_{L}(R)}^{T_{U}(R)}f(T|Data,R)dT=CL </math>

which is equivalent to:

<math> \int\nolimits_{0}^{T_{U}(R)}f(T|Data,R)dT=(1+CL)/2 </math>

and:

<math> \int\nolimits_{0}^{T_{L}(R)}f(T|Data,R)dT=(1-CL)/2 </math>

 

=====Example 6=====

A manufacturer has tested prototypes of a modified product. The test was terminated at 2000 hours, with only two failures observed from a sample size of eighteen.

{| border=1 cellspacing=1 align="center"
|-
|Number of State||State of F or S||State End Time
|-
| 1 || F || 1180
|-
| 1 || F || 1842
|-
| 16 || S || 2000
|}

Because of the lack of failure data in the prototype testing, the manufacturer decided to use information gathered from prior tests on this product to increase the confidence in the results of the prototype testing. This decision was made because failure analysis indicated that the failure mode of these two failures is the same as the one observed in previous tests. In other words, it is expected that the shape of the distribution hasn't changed, but hopefully the scale has, indicating longer life. The two-parameter Weibull distribution have been used to model all prior tests results. The list of the estimated β parameter is as follows:

{| border=1 cellspacing=1 align="center"
|-
|Betas Obtained for Similar Mode
|-
| 1.7
|-
| 2.1
|-
| 2.4
|-
|3.1
|-
|3.5
|}

First, in order to fit the data to a Weibull-Bayesian model, a prior distribution for β needs to be determined. Based on the prior tests' β values, the prior distribution for β was found to be a lognormal distribution with μ = 0.9064, σ = 0.3325 (obtained by entering the β values into a Weibull++ ''Standard Folio'' and analyzing it based on the RRX analysis method.)

the test data is entered into a ''Standard Folio'', the Weibull-Bayesian is selected under '' Distribution'' and the β prior distribution is entered after clicking the ''Calculate'' button.

Suppose that the reliability at 3000hr is the metric of interest in this example. This reliability can be obtained using Eqn. (EQNREF MedRel ), resulting in the median value of the posterior of the reliability at 3000hr. Using the ''QCP'', this value is calculated to be 76.97. ( By default Weibull++ returns the median values of the posterior distribution. )

The posterior <math>pdf</math> of the reliability function at 3000hrs can be obtained using Eqn. (EQNREF Rpdf ). In Figure 6-10 the posterior <math>pdf</math> of the reliability at 3000hrs is plotted, with the corresponding median value as well as the 10th percentile value shown. The 10th percentile constitutes the 90 Lower 1-Sided bound on the reliability at 3000hrs, which is calculated to be 50.77.

FIGURE HERE

Notice that the <math>pdf</math> plotted in Fig. 6-10 is of the reliability at 3000hrs, and not the <math>pdf</math> of the times-to-failure data. The <math>pdf</math> of the times-to-failure data can be obtained using Eqn. (EQNREF WeibBayesPDF ) and plotted using Weibull++, as shown next:

FIGURE HERE

{{RS Copyright}}

[[Category:Life_Data_Analysis_Reference]]

Template:Likelihood Ratio Confidence Bounds

2011-06-28T16:53:12Z

Steve Sharp: Created page with 'As covered in Chapter 5, the likelihood confidence bounds are calculated by finding values for θ1 and θ2</sub…'

As covered in Chapter 5, the likelihood confidence bounds are calculated by finding values for θ1 and θ2 that satisfy:

<math> -2\cdot \text{ln}\left( \frac{L(\theta _{1},\theta _{2})}{L(\hat{\theta }_{1}, \hat{\theta }_{2})}\right) =\chi _{\alpha ;1}^{2} EQNREF lratio2 </math>

This equation can be rewritten as:

<math> L(\theta _{1},\theta _{2})=L(\hat{\theta }_{1},\hat{\theta } _{2})\cdot e^{\frac{-\chi _{\alpha ;1}^{2}}{2}} EQNREF lratio3 </math>

For complete data, the likelihood function for the Weibull distribution is given by:

<math> L(\beta ,\eta )=\prod_{i=1}^{N}f(x_{i};\beta ,\eta )=\prod_{i=1}^{N}\frac{ \beta }{\eta }\cdot \left( \frac{x_{i}}{\eta }\right) ^{\beta -1}\cdot e^{-\left( \frac{x_{i}}{\eta }\right) ^{\beta }} </math>

For a given value of α, values for β and η can be found which represent the maximum and minimum values that satisfy Eqn. (\ref {lratio3}). These represent the confidence bounds for the parameters at a confidence level δ, where α = δ for two-sided bounds and α = 2δ − 1 for one-sided.

Similarly, the bounds on time and reliability can be found by substituting the Weibull reliability equation into the likelihood function so that it is in terms of β and time or reliability, as discussed in Chapter 5. The likelihood ratio equation used to solve for bounds on time (Type 1) is:

<math> L(\beta ,t)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] </math>

The likelihood ratio equation used to solve for bounds on reliability (Type 2) is:

<math> L(\beta ,R)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] </math>

Fisher Matrix Confidence Bounds

2011-06-28T00:04:25Z

Steve Sharp: Created page with 'This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher mat…'

This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiply censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [30] and Lloyd and Lipow [24]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds.
===Approximate Estimates of the Mean and Variance of a Function===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e. reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G</math> is presented next.

====Single Parameter Case====
For simplicity, consider a one-parameter distribution represented by a general function, <math>G,</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda </math>: <math>G(\lambda )=1/\lambda =\mu </math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)</math> can be found by:

<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)</math>

where <math>G(\theta )</math> is some function of <math>\theta </math>, such as the reliability function, and <math>\theta </math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta </math> as <math>n\to \infty </math> . The term <math>O\left( \tfrac{1}{n} \right)</math> is a function of <math>n</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},</math> as <math>n\to \infty .</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}</math> and <math>G(x)=1/x</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}</math>. Thus as <math>n\to \infty </math>, <math>E(G(\widehat{\theta }))=\mu </math> where <math>\mu </math> and <math>\sigma </math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)</math> can then be estimated by:

<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)</math>

====Two-Parameter Case====

Consider a Weibull distribution with two parameters <math>\beta </math> and <math>\eta </math>. For a given value of <math>T</math>, <math>R(T)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{T}{\eta } \right)}^{\beta }}}}</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)</math>, that:

<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)</math>

and:

<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Var(\widehat{\theta_{2}})\\

& +2{(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}})^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}{(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}})^2}_{{\widehat{\theta_{2}}}={\theta_{1}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\

& +O(\frac{1}{n^{\tfrac{3}{2}}})
\end{align}

</math>

Note that the derivatives of Eqn. (var) are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{1}},</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.</math>
Parameter Variance and Covariance Determination
The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}</math>

In the equation above, the first summation is for complete data, the second summation is for right censored data, and the third summation is for interval or left censored data. For more information on these data types, see Chapter 4.
Then the Fisher information matrix is given by:

<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]</math>

The subscript <math>0</math> indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},</math> the true values of the parameters.
So for a sample of <math>N</math> units where <math>R</math> units have failed, <math>S</math> have been suspended, and <math>P</math> have failed within a time interval, and <math>N=R+M+P,</math> one could obtain the sample local information matrix by:

<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}</math>

Substituting in the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}</math> and <math>{{\widehat{\theta }}_{2}}</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}</math>

Then the variance of a function (<math>Var(G)</math>) can be estimated using Eqn. (var). Values for the variance and covariance of the parameters are obtained from Eqn. (Fisher2).
Once they have been obtained, the approximate confidence bounds on the function are given as:

<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha }}</math> next.

====Approximate Confidence Intervals on the Parameters====
In general, MLE estimates of the parameters are asymptotically normal, meaning for large sample sizes that a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }</math> is the MLE estimator for <math>\theta </math>, in the case of a single parameter distribution, estimated from a large sample of <math>n</math> units and if:

<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}</math>

then using the normal distribution of <math>z\ \ :</math>

<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

for large <math>n</math>. We now place confidence bounds on <math>\theta ,</math> at some confidence level <math>\delta </math>, bounded by the two end points <math>{{C}_{1}}</math> and <math>{{C}_{2}}</math> where:

<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta </math>

From Eqn. (e729):

<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta </math>

where <math>{{K}_{\alpha }}</math> is defined by:

<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)</math>

Now by simplifying Eqn. (e731), one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta ,</math> at a confidence level <math>\delta ,</math> or:

<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)</math>

The upper one-sided bounds are given by:

<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

while the lower one-sided bounds are given by:

<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}</math>

If <math>\widehat{\theta }</math> must be positive, then <math>\ln \widehat{\theta }</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta </math>, then become:

<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}</math>

The one-sided approximate confidence bounds on the parameter <math>\theta </math>, at confidence level <math>\delta ,</math> can be found from:

<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [24] further elaborate on this procedure.

====Confidence Bounds on Time (Type 1)====
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e. y-ordinate or unreliability, <math>Q=1-R)</math> is determined by solving the unreliability function for the time, <math>T</math>, or:

<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cb). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Note that the procedure is slightly more complicated for distributions with more than one parameter.

====Confidence Bounds on Reliability (Type 2)====
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }</math> and substituting these values into Eqn. (cbr). The bounds on <math>\widehat{\lambda }</math> were determined using Eqns. (cblmu) and (cblml), with its variance obtained from Eqn. (Fisher2). Once again, the procedure is more complicated for distributions with more than one parameter.

===Beta Binomial Confidence Bounds===
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see Chapter 4). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution.
Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z</math> (rank for the <math>{{j}^{th}}</math> failure):

<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where <math>N</math> is the sample size and <math>j</math> is the order number.
The median rank was obtained by solving the following equation for <math>Z</math>:

<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

The same methodology can then be repeated by changing <math>P</math> for <math>0.50</math> <math>(50%)</math> to our desired confidence level. For <math>P=90%</math> one would formulate the equation as

<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

Keep in mind that one must be careful to select the appropriate values for <math>P</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P=0.1</math> and once with <math>P=0.9</math>) in order to place the bounds around 80% of the population.
Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.
This non-parametric methodology is only used by Weibull++ when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [20]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

===Likelihood Ratio Confidence Bounds===
====Introduction====
A third method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.
Likelihood ratio confidence bounds are based on the equation:

<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}</math>

where:
#<math>L(\theta )</math> is the likelihood function for the unknown parameter vector <math>\theta </math>
#<math>L(\widehat{\theta })</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }</math>
#<math>\chi _{\alpha ;k}^{2}</math> is the chi-squared statistic with probability <math>\alpha </math> and <math>k</math> degrees of freedom, where <math>k</math> is the number of quantities jointly estimated
If <math>\delta </math> is the confidence level, then <math>\alpha =\delta </math> for two-sided bounds and <math>\alpha =(2\delta -1)</math> for one-sided. Recall from Chapter 3 that if <math>x</math> is a continuous random variable with <math>pdf</math>:

<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>,

where <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,{{x}_{R}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

<math>i=1,2,...,R</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L.</math> These are represented by the <math>L(\widehat{\theta })</math> term in the denominator of the ratio in Eqn. (lratio1). Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }</math> have been calculated using MLE methods, the only unknown term in Eqn. (lratio1) is the <math>L(\theta )</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta </math> that satisfy Eqn. (lratio1). For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy Eqn. (lratio1). The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta ;</math> but at a given value of <math>\delta </math> there is only a certain region of values for <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> for which Eqn. (lratio1) holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of Eqn. (lratio1).

====Note on Contour Plots in Weibull++====
Contour plots can be used for comparing data sets. Consider two data sets, e.g. old and new design where the engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in a multiple plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e. the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If there is an overlap between the two 95% contours, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. Chapter 12 discusses comparing data sets.

====Confidence Bounds on the Parameters====
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy:

<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}</math>

This equation can be rewritten as:

<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}</math>

The task now becomes to find the values of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math> so that the equality in Eqn. (lratio3) is satisfied. Unfortunately, there is no closed-form solution, thus these values must be arrived at numerically. One method of doing this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

=====Example 1=====
Five units were put on a reliability test and experienced failures at 10, 20, 30, 40, and 50 hours. Assuming a Weibull distribution, the MLE parameter estimates are calculated to be <math>\widehat{\beta }=2.2938</math> and <math>\widehat{\eta }=33.9428.</math> Calculate the 90% two-sided confidence bounds on these parameters using the likelihood ratio method.

=====Solution to Example 1=====
The first step is to calculate the likelihood function for the parameter estimates:

<math>\begin{align}
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\widehat{\beta },\widehat{\eta })=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\widehat{\beta }}{\widehat{\eta }}\cdot {{\left( \frac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }-1}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{\widehat{\eta }} \right)}^{\widehat{\beta }}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{2.2938}{33.9428}\cdot {{\left( \frac{{{x}_{i}}}{33.9428} \right)}^{1.2938}}\cdot {{e}^{-{{\left( \tfrac{{{x}_{i}}}{33.9428} \right)}^{2.2938}}}} \\
\\
L(\widehat{\beta },\widehat{\eta })= & 1.714714\times {{10}^{-9}}
\end{align}</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

<math>L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We then substitute this information into the equation:

<math>\begin{align}
L(\beta ,\eta )-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,\eta )-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,\eta )-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

The next step is to find the set of values of <math>\beta </math> and <math>\eta </math> that satisfy this equation, or find the values of <math>\beta </math> and <math>\eta </math> such that <math>L(\beta ,\eta )=4.432926\cdot {{10}^{-10}}.</math>

The solution is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>\eta </math>, and vice versa. The following table gives values of <math>\beta </math> based on given values of <math>\eta </math>.

These data are represented graphically in the following contour plot:

(Note that this plot is generated with degrees of freedom <math>k=1</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2</math>, for use in comparing both parameters simultaneously.) As can be determined from the table, the lowest calculated value for <math>\beta </math> is 1.142, while the highest is 3.950. These represent the two-sided 90% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>\eta </math> below 23 or above 50, these can be considered the 90% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>\eta </math>, we can perform the same procedure as before, but finding the two values of <math>\eta </math> that correspond with a given value of <math>\beta .</math> Using this method, we find that the 90% confidence limits on <math>\eta </math> are 22.474 and 49.967, which are close to the initial estimates of 23 and 50.
Note that the points where <math>\beta </math> are maximized and minimized do not necessarily correspond with the points where <math>\eta </math> are maximized and minimized. This is due to the fact that the contour plot is not symmetrical, so that the parameters will have their extremes at different points.

====Confidence Bounds on Time (Type 1)====
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta }_{1}}</math> and <math>{{\theta }_{2}}</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.
=====Example 2=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the time estimate for a reliability of 50%. The ML estimate for the time at which <math>R(t)=50%</math> is 28.930.
=====Solution to Example 2=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the time estimate of 28.930. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>t</math> and <math>\beta .</math> This is accomplished by using a form of the Weibull reliability equation, <math>R={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}.</math> This can be rearranged in terms of <math>\eta </math>, with <math>R</math> being considered a known variable or:

<math>\eta =\frac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}}</math>

This can then be substituted into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>t</math> and <math>\beta </math> or:

<math>\begin{align}
& L(\beta ,t)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

<math>L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

<math>\begin{align}
L(\beta ,t)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,t)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
& \\
L(\beta ,t)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

Note that the likelihood value for <math>L(\widehat{\beta },\widehat{\eta })</math> is the same as it was for Example 1. This is because we are dealing with the same data and parameter estimates or, in other words, the maximum value of the likelihood function did not change. It now remains to find the values of <math>\beta </math> and <math>t</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>t</math>. The following table gives the values of <math>t</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:
As can be determined from the table, the lowest calculated value for <math>t</math> is 17.389, while the highest is 41.714. These represent the 90% two-sided confidence limits on the time at which reliability is equal to 50%.

====Confidence Bounds on Reliability (Type 2)====
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta </math> and <math>R</math>. The likelihood function is once again altered in the same way as before, only now <math>R</math> is considered to be a parameter instead of <math>t</math>, since the value of <math>t</math> must be specified in advance. Once again, this process is best illustrated with an example.

=====Example 3=====
For the data given in Example 1, determine the 90% two-sided confidence bounds on the reliability estimate for <math>t=45</math>. The ML estimate for the reliability at <math>t=45</math> is 14.816%.

=====Solution to Example 3=====
In this example, we are trying to determine the 90% two-sided confidence bounds on the reliability estimate of 14.816%. As was mentioned, we need to rewrite Eqn. (lrbexample) so that it is in terms of <math>R</math> and <math>\beta .</math> This is again accomplished by substituting the Weibull reliability equation into the <math>\eta </math> term in Eqn. (lrbexample) to form a likelihood equation in terms of <math>R</math> and <math>\beta </math>:

<math>\begin{align}
& L(\beta ,R)= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\beta ,t,R) \\
& &
\end{align}</math>

<math>=\underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{\beta }{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)}\cdot {{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta -1}}\cdot \text{exp}\left[ -{{\left( \frac{{{x}_{i}}}{\left( \tfrac{t}{{{(-\text{ln}(R))}^{\tfrac{1}{\beta }}}} \right)} \right)}^{\beta }} \right]</math>

where <math>{{x}_{i}}</math> are the original time-to-failure data points. We can now rearrange Eqn. (lratio3) to the form:

<math>L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0</math>

Since our specified confidence level, <math>\delta </math>, is 90%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.9;1}^{2}=2.705543.</math> We can now substitute this information into the equation:

<math>\begin{align}
L(\beta ,R)-L(\widehat{\beta },\widehat{\eta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
\\
L(\beta ,R)-1.714714\times {{10}^{-9}}\cdot {{e}^{\tfrac{-2.705543}{2}}}= & 0 \\
\\
L(\beta ,R)-4.432926\cdot {{10}^{-10}}= & 0
\end{align}</math>

It now remains to find the values of <math>\beta </math> and <math>R</math> that satisfy this equation. This is an iterative process that requires setting the value of <math>\beta </math> and finding the appropriate values of <math>R</math>. The following table gives the values of <math>R</math> based on given values of <math>\beta </math>.

These points are represented graphically in the following contour plot:

As can be determined from the table, the lowest calculated value for <math>R</math> is 2.38%, while the highest is 44.26%. These represent the 90% two-sided confidence limits on the reliability at <math>t=45</math>.

===Bayesian Confidence Bounds===
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in Chapter 3.
Bayesian confidence bounds are derived from Bayes rule, which states that:

<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>f(\theta |Data)</math> is the <math>posterior</math> <math>pdf</math> of <math>\theta </math>
#<math>\theta </math> is the parameter vector of the chosen distribution (i.e. Weibull, lognormal, etc.)
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the <math>prior</math> <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>.
In other words, the prior knowledge is provided in the form of the prior <math>pdf</math> of the parameters, which in turn is combined with the sample data in order to obtain the posterior <math>pdf.</math> Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to Chapter 3). It can be seen from Eqn. (BayesRule) that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta }_{1}}</math>. Given a set of sample data, and a prior distribution for <math>{{\theta }_{1}},</math> <math>\varphi ({{\theta }_{1}}),</math> Eqn. (BayesRule) can be written as:

<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

In other words, we now have the distribution of <math>{{\theta }_{1}}</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta }_{1}}</math> is less than or equal to a value <math>x,</math> <math>P({{\theta }_{1}}\le x)</math> can be obtained by integrating Eqn. (BayesEX), or:

<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}</math>

Eqn. (IntBayes) essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)</math> is the confidence level and <math>x</math> is the confidence bound. Substituting Eqn. (BayesEX) into Eqn. (IntBayes) yields:

<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}</math>

The only question at this point is what do we use as a prior distribution of <math>{{\theta }_{1}}.</math>. For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform.
Eqn. (BayesCLEX) can be generalized for any distribution having a vector of parameters <math>\theta ,</math> yielding the general equation for calculating Bayesian confidence bounds:

<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }</math>

where:
#<math>CL</math> is confidence level
#<math>\theta </math> is the parameter vector
#<math>L(\bullet )</math> is the likelihood function
#<math>\varphi (\theta )</math> is the prior <math>pdf</math> of the parameter vector <math>\theta </math>
#<math>\varsigma </math> is the range of <math>\theta </math>
#<math>\xi </math> is the range in which <math>\theta </math> changes from <math>\Psi (T,R)</math> till <math>{\theta }'s</math> maximum value or from <math>{\theta }'s</math> minimum value till <math>\Psi (T,R)</math>
#<math>\Psi (T,R)</math> is function such that if <math>T</math> is given then the bounds are calculated for <math>R</math> and if <math>R</math> is given, then he bounds are calculated for <math>T</math>.
If <math>T</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>R</math> are calculated.
If <math>R</math> is given, then from Eqn. (BayesCL) and <math>\Psi </math> and for a given <math>CL,</math> the bounds on <math>T</math> are calculated.
====Confidence Bounds on Time (Type 1)====
For a given failure time distribution and a given reliability <math>R</math>, <math>T(R)</math> is a function of <math>R</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For the two-parameter Weibull distribution:

<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)</math> is:

<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

where <math>f(T|Data,R)</math> is the posterior distribution of Time <math>T.</math>
Using Eqn. (T bayes), we have the following:

<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})</math>

Eqn. (cl) can be rewritten in terms of <math>\eta </math> as:

<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))</math>

From Eqns. (IntBayes), (BayesCLEX) and (BayesCL), by assuming the priors of <math>\beta </math> and <math>\eta </math> are independent, we then obtain the following relationship:

<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl2) can be solved for <math>{{T}_{U}}(R)</math>, where:
#<math>CL</math> is confidence level,
#<math>\varphi (\beta )</math> is the prior <math>pdf</math> of the parameter <math>\beta </math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.</math>
#<math>\varphi (\eta )</math> is the prior <math>pdf</math> of the parameter <math>\eta .</math>. For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.</math>
#<math>L(\bullet )</math> is the likelihood function.
The same method can be used to get the one-sided lower bound of <math>T(R)</math> from:

<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl5) can be solved to get <math>{{T}_{L}}(R)</math>.
The Bayesian two-sided bounds estimate for <math>T(R)</math> is:

<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

which is equivalent to:

<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT</math>

and:

<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)</math> and <math>{{T}_{L}}(R)</math> can be solved.
====Confidence Bounds on Reliability (Type 2)====
For a given failure time distribution and a given time <math>T</math>, <math>R(T)</math> is a function of <math>T</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. Bounds, for the case of other distributions, can be obtained in similar fashion. For example, for two parameter Weibull distribution:

<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})</math>

The Bayesian one-sided upper bound estimate for <math>R(T)</math> is:

<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

Similar with the bounds on Time, the following is obtained:

<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl3) can be solved to get <math>{{R}_{U}}(T)</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR</math>

Using the posterior distribution, the following is obtained:

<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }</math>

Eqn. (cl4) can be solved to get <math>{{R}_{L}}(T)</math>.
The Bayesian two-sided bounds estimate for <math>R(T)</math> is:

<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR</math>

which is equivalent to:

<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2</math>

and

<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)</math> and <math>{{R}_{L}}(T)</math> can be solved.

===Simulation Based Bounds===
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds.
SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

=====Example 4=====
The purpose of this example is to determine the best parameter estimation method for a sample of ten units following a Weibull distribution with <math>\beta =2</math> and <math>\eta =100</math> and with complete time-to-failure data for each unit (i.e. no censoring). The number of generated data sets is set to 10,000. The SimuMatic inputs are shown next.

The parameters are estimated using RRX, RRY and MLE. The plotted results generated by SimuMatic are shown next.

Using RRX:

Using RRY:

Using MLE:

The results clearly demonstrate that the median RRX estimate provides the least deviation from the truth for this sample size and data type. However, the MLE outputs are grouped more closely together, as evidenced by the bounds. The previous figures also show the simulation-based bounds, as well as the expected variation due to sampling error.
This experiment can be repeated in SimuMatic using multiple censoring schemes (including Type I and Type II right censoring as well as random censoring) with various distributions. Multiple experiments can be performed with this utility to evaluate assumptions about the appropriate parameter estimation method to use for data sets.

Data & Data Types

2011-06-27T23:54:19Z

Steve Sharp: Created page with 'Statistical models rely extensively on data to make predictions. In our case, the models are the ''statistical distributions'' and the data are the '' life data''or '' times-to-f…'

Statistical models rely extensively on data to make predictions. In our case, the models are the ''statistical distributions'' and the data are the '' life data''or '' times-to-failure data'' of our product. The accuracy of any prediction is directly proportional to the quality, accuracy and completeness of the supplied data. Good data, along with the appropriate model choice, usually results in good predictions. Bad, or insufficient data, will almost always result in bad predictions.
In the analysis of life data, we want to use all available data which sometimes is incomplete or includes uncertainty as to when a failure occurred. To accomplish this, we separate life data into two categories: complete (all information is available) or censored (some of the information is missing). This chapter details these data classification methods.
===Data Classification===
Most types of non-life data, as well as some life data, are what we term as ''complete data''. Complete data means that the value of each sample unit is observed or known. In many cases, life data contains uncertainty as to when exactly an event happened (''i.e.''when the unit failed). Data containing such uncertainty as to exactly when the event happened is termed as ''censored data''.

====Complete Data====
Complete data means that the value of each sample unit is observed or known. For example, if we had to compute the average test score for a sample of ten students, complete data would consist of the known score for each student. Likewise in the case of life data analysis, our data set (if complete) would be composed of the times-to-failure of all units in our sample. For example, if we tested five units and they all failed (and their times-to-failure were recorded), we would then have complete information as to the time of each failure in the sample.

Complete data is much easier to work with than censored data. For example, it would be much harder to compute the average test score of the students if our data set were not complete, i.e.the average test score given scores of 30, 80, 60, 90, 95, three scores greater than 50, a score that is less than 70 and a score that is between 60 and 80.
====Censored Data ====
In many cases when life data are analyzed, all of the units in the sample may not have failed (i.e. the event of interest was not observed) or the exact times-to-failure of all the units are not known. This type of data is commonly called ''censored data''. There are three types of possible censoring schemes, right censored (also called suspended data), interval censored and left censored.

=====Right Censored (Suspended)=====
The most common case of censoring is what is referred to as ''right censored data'', or ''suspended data''. In the case of life data, these data sets are composed of units that did not fail. For example, if we tested five units and only three had failed by the end of the test, we would have suspended data (or right censored data) for the two unfailed units. The term ''right censored'' implies that the event of interest (i.e. the time-to-failure) is to the right of our data point. In other words, if the units were to keep on operating, the failure would occur at some time after our data point (or to the right on the time scale).

=====Interval Censored=====
The second type of censoring is commonly called ''interval censored data''. Interval censored data reflects uncertainty as to the exact times the units failed within an interval. This type of data frequently comes from tests or situations where the objects of interest are not constantly monitored. If we are running a test on five units and inspecting them every 100 hours, we only know that a unit failed or did not fail between inspections. More specifically, if we inspect a certain unit at 100 hours and find it is operating and then perform another inspection at 200 hours to find that the unit is no longer operating, we know that a failure occurred in the interval between 100 and 200 hours. In other words, the only information we have is that it failed in a certain interval of time. This is also called ''inspection data'' by some authors.

=====Left Censored=====
The third type of censoring is similar to the interval censoring and is called ''left censored data''. In left censored data, a failure time is only known to be before a certain time. For instance, we may know that a certain unit failed sometime before 100 hours but not exactly when. In other words, it could have failed any time between 0 and 100 hours. This is identical to ''interval censored data''in which the starting time for the interval is zero.

====Data Types and Weibull++====
Weibull++ allows you to use all of the above data types in a single data set. In other words, a data set can contain complete data, right censored data, interval censored data and left censored data. An overview of this is presented in this section.

====Grouped Data and Weibull++====
All of the previously mentioned data types can also be put into groups. This is simply a way of collecting units with identical failure or censoring times. If ten units were put on test with the first four units failing at 10, 20, 30 and 40 hours respectively, and then the test were terminated after the fourth failure, you can group the last six units as a group of six suspensions at 40 hours. Weibull++ allows you to enter all types of data as groups, as shown in the following figure.

Depending on the analysis method chosen, ''i.e.'' regression or maximum likelihood, Weibull++ treats grouped data differently. This was done by design to allow for more options and flexibility. Appendix B describes how Weibull++ treats grouped data.

====Classifying Data in Weibull++====
In Weibull++, data classifications are specified using data types. A single data set can contain any or all of the mentioned censoring schemes. Weibull++, through the use of the New Project Wizard and the New Data Sheet Wizard features, simplifies the choice of the appropriate data type for your data. Weibull++ uses the logic tree shown in Fig. 4-1 in deciding which is the appropriate data type for your data.

===Analysis & Parameter Estimation Methods for Censored Data===
In Chapter 3 we discussed parameter estimation methods for complete data. We will expand on that approach in this section by including estimation methods for the different types of censoring. The basic methods are still based on the same principles covered in Chapter 3, but modified to take into account the fact that some of the data points are censored. For example, assume that you were asked to find the mean (average) of 10, 20, a value that is between 25 and 40, a value that is greater than 30 and a value that is less than 50. In this case, the familiar method of determining the average is no longer applicable and special methods will need to be employed to handle the censored data in this data set.

ReliaSoft's Alternate Ranking Method (RRM) Step-by-Step Example

2011-06-27T23:49:22Z

Steve Sharp: Created page with 'This section illustrates the ReliaSoft ranking method (RRM), which is an iterative improvement on the standard ranking method (SRM). This method is illustrated in this section us…'

This section illustrates the ReliaSoft ranking method (RRM), which is an iterative improvement on the standard ranking method (SRM). This method is illustrated in this section using an example for the two-parameter Weibull distribution. This method can also be easily generalized for other models.

Consider the following test data, as shown in the following Table B.1.

When Using Maximum Likelihood

2011-06-27T23:45:08Z

Steve Sharp: Created page with 'When using maximum likelihood methods, each individual time is explicitly used in the calculation of the parameters, thus there is no difference in the entry of a group of 10 uni…'

When using maximum likelihood methods, each individual time is explicitly used in the calculation of the parameters, thus there is no difference in the entry of a group of 10 units failing at 100 hours and 10 individual entries of 100 hours. This is inherent in the standard MLE method. In other words, no matter how the data were entered (i.e. as grouped or non-grouped) the results will be identical. When using maximum likelihood, we highly recommend entering redundant data in groups, as this significantly speeds up the calculations.

Median Ranks

2011-06-27T23:44:38Z

Steve Sharp: Created page with 'Median ranks are used to obtain an estimate of the unreliability, <math>Q({{T}_{j}}),</math> for each failure at a <math>50%</math> confidence level. In the case of grouped data,…'

Median ranks are used to obtain an estimate of the unreliability, <math>Q({{T}_{j}}),</math> for each failure at a <math>50%</math> confidence level. In the case of grouped data, the ranks are estimated for each group of failures, instead of each failure.
For example, when using a group of 10 failures at 100 hours, 10 at 200 hours and 10 at 300 hours, Weibull++ estimates the median ranks (<math>Z</math> values) by solving the cumulative binomial equation with the appropriate values for order number and total number of test units.
For 10 failures at 100 hours, the median rank, <math>Z,</math> is estimated by using:

<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

with:

<math>N=30,\text{ }J=10</math>

where one <math>Z</math> is obtained for the group, to represent the probability of 10 failures occurring out of 30.
For 10 failures at 200 hours, <math>Z</math> is estimated by using:

<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where:

<math>N=30,\text{ }J=20</math>

to represent the probability of 20 failures out of 30.
For 10 failures at 300 hours, <math>Z</math> is estimated by using:

<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}</math>

where:

<math>N=30,\text{ }J=30</math>

to represent the probability of 30 failures out of 30.

When Using Rank Regression (Least Squares)

2011-06-27T23:44:07Z

Steve Sharp: Created page with 'When using grouped data, Weibull++ plots the data point corresponding to the highest rank position in each group. In other words, given 3 groups of 10 units, each failing at 100,…'

When using grouped data, Weibull++ plots the data point corresponding to the highest rank position in each group. In other words, given 3 groups of 10 units, each failing at 100, 200, and 300 hours respectively, the three plotted points will be the end point of each group, or the 10th rank position out of 30, the 20th rank position out of 30 and the 30th rank position out of 30. This procedure is identical to standard procedures for using grouped data [19]. In cases where grouped data is used, it is assumed that the failures occurred at some time in the interval between the previous and current time to failure. In our example, this would be the same as saying that 10 units have failed in the interval between zero and 100 hours, another 10 units failed in the interval between 100 and 200 hours, and in the interval from 200 to 300 hours another 10 units failed. The rank regression analysis automatically takes this into account. If this assumption of interval failure is incorrect and 10 units failed exactly at 100 hours, 10 failed exactly at 200 hours and 10 failed exactly at 300 hours, it is recommended that you enter the data as non-grouped when using rank regression, or select the Use all data if grouped option from the Folio Control Panel's Set Analysis tab.

Appendix: Log-Likelihood Equations

2011-06-27T23:40:48Z

Steve Sharp: /* Logistic Log-Likelihood Functions and their Partials */

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

<math>\lambda =0</math>

or:

<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>{{F}_{i}}</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>Q</math> is the number of subpopulations

• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of groups of interval data points

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data group

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups,

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Appendix: Log-Likelihood Equations

2011-06-27T23:39:50Z

Steve Sharp: /* The One-Parameter Exponential */

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

<math>\lambda =0</math>

or:

<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>{{F}_{i}}</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>Q</math> is the number of subpopulations

• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of groups of interval data points

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data group

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& & -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups,

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Appendix: Log-Likelihood Equations

2011-06-27T23:39:23Z

Steve Sharp: /* The Three-Parameter Weibull */

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

<math>\lambda =0</math>

or:

<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>{{F}_{i}}</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>Q</math> is the number of subpopulations

• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of groups of interval data points

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data group

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& & -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups,

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Appendix: Log-Likelihood Equations

2011-06-27T23:38:35Z

Steve Sharp: /* The Two-Parameter Weibull */

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& & \\
& & +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& & -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& & -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& & +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

<math>\lambda =0</math>

or:

<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>{{F}_{i}}</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>Q</math> is the number of subpopulations

• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of groups of interval data points

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data group

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& & -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups,

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Appendix: Log-Likelihood Equations

2011-06-27T23:37:42Z

Steve Sharp: Created page with 'This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed…'

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& & -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& & +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& & \\
& & +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\beta </math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• <math>\eta </math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>\gamma </math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& & -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& & -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& & +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].
Non-regularity occurs when <math>\beta \le 2.</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.</math> When <math>1<\beta <2,</math> MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where <math>\gamma </math> is estimated using non-linear regression. Once <math>\gamma </math> is obtained, the MLE estimates of <math>\widehat{\beta }</math> and <math>\widehat{\eta }</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

where,

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

and:

<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

or Eqn. (expll2) will be equal to zero only if either:

<math>\lambda =0</math>

or:

<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the mean parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>{{F}_{i}}</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}</math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>{\mu }'</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• <math>{{\sigma }_{{{T}'}}}</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0</math>:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math></math>

where:

<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}}</math>

and:

<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:
<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}</math>

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>Q</math> is the number of subpopulations

• <math>{{\rho }_{k}}</math> is the proportionality of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• <math>{{\beta }_{k}}</math> is the Weibull shape parameter of the <math>{{k}^{th}}</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• <math>{{\eta }_{k}}</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of groups of interval data points

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a group of parameters:

<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)</math>

so that:

<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\eta </math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data group

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
& \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& & -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups,

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

or

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}</math>

where:

• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• <math>\sigma </math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• <math>N_{i}^{\prime }</math> is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that:

<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}</math>

or:

<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}</math>

where:
• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points

• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group

• <math>\mu </math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• <math>k</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data

• <math>S</math> is the number of groups of suspension data points

• .. is the number of suspensions in <math>{{i}^{th}}</math> group of suspension data points

• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group

• <math>FI</math> is the number of interval failure data groups

• <math>N_{i}^{\prime \prime }</math> is the number of intervals in <math>{{i}^{th}}</math> group of data intervals

• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval

• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}</math>

<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}</math>

Appendix: Maximum Likelihood Estimation Example

2011-06-27T23:36:23Z

Steve Sharp: Created page with 'If <math>x</math> is a continuous random variable with <math>pdf\ \ :</math> <math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}),</math> where <math>{{\theta }_{1}},…'

If <math>x</math> is a continuous random variable with <math>pdf\ \ :</math>

<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}),</math>

where <math>{{\theta }_{1}},</math> <math>{{\theta }_{2}},</math><math>...,</math> <math>{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,</math> <math>{{x}_{N}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:

<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

<math>i=1,2,...,N</math>

The logarithmic likelihood function is:

<math>\Lambda =\ln L=\underset{i=1}{\overset{N}{\mathop \sum }}\,\ln f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L</math> or <math>\Lambda .</math>
By maximizing <math>\Lambda ,</math> which is much easier to work with than <math>L</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are the simultaneous solutions of <math>k</math> equations such that:

<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>

Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other, and in no way suggests that the solution is wrong.

====Illustrating the MLE Method Using the Exponential Distribution====
• To estimate <math>\widehat{\lambda }</math> for a sample of <math>n</math> units (all tested to failure), first obtain the likelihood function:

<math>\begin{align}
L(\lambda |{{t}_{1}},{{t}_{2}},...,{{t}_{n}})= & \underset{i=1}{\overset{n}{\mathop \prod }}\,f({{t}_{i}}) \\
= & \underset{i=1}{\overset{n}{\mathop \prod }}\,\lambda {{e}^{-\lambda {{t}_{i}}}} \\
= & {{\lambda }^{n}}\cdot {{e}^{-\lambda \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{t}_{i}}}}
\end{align}</math>

• Take the natural log of both sides:

<math>\Lambda =\ln (L)=n\ln (\lambda )-\lambda \underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}.</math>

• Obtain <math>\tfrac{\partial \Lambda }{\partial \lambda }</math>, and set it equal to zero:

<math>\frac{\partial \Lambda }{\partial \lambda }=\frac{n}{\lambda }-\underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}=0</math>

• Solve for <math>\widehat{\lambda }</math> or:

<math>\hat{\lambda }=\frac{n}{\underset{i=1}{\overset{n}{\mathop{\sum }}}\,{{t}_{i}}}</math>

====Notes About <math>\widehat{\lambda }</math>====
Note that the value of <math>\widehat{\lambda }</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda </math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda </math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in Chapter 5.)

====Illustrating the MLE Method Using Normal Distribution====
To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},</math> for the normal distribution, start with the <math>pdf</math> of the normal distribution which is given by:

<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

If <math>{{T}_{1}},{{T}_{2}},...,{{T}_{N}}</math> are known times-to-failure (and with no suspensions), then the likelihood function is given by:

<math>L({{T}_{1}},{{T}_{2}},...,{{T}_{N}}|\bar{T},{{\sigma }_{T}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left[ \frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} \right]</math>

<math>L=\frac{1}{{{({{\sigma }_{T}}\sqrt{2\pi })}^{N}}}{{e}^{-\tfrac{1}{2}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

then:

<math>\Lambda =\ln L=-\frac{N}{2}\ln (2\pi )-N\ln {{\sigma }_{T}}-\frac{1}{2}\underset{i=1}{\overset{N}{\mathop \sum }}\,\left( \frac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)_{}^{2}</math>

Then taking the partial derivatives of <math>\Lambda </math> with respect to each one of the parameters and setting them equal to zero yields:

<math>\frac{\partial (\Lambda )}{\partial \bar{T}}=\frac{1}{\sigma _{T}^{2}}\underset{i=1}{\overset{N}{\mathop \sum }}\,({{T}_{i}}-\bar{T})=0</math>

and:

<math>\frac{\partial (\Lambda )}{\partial {{\sigma }_{T}}}=-\frac{N}{{{\sigma }_{T}}}+\frac{1}{\sigma _{T}^{3}}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}=0</math>

Solving Eqns. (dldt) and (dlds) simultaneously yields:

<math>\bar{T}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}</math>

and:

<math>\begin{align}
& \hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
& & \\
& {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}</math>

It should be noted that these solutions are only valid for data with no suspensions, i.e. all units are tested to failure. In the case where suspensions are present or all units are not tested to failure, the methodology changes and the problem becomes much more complicated.

====Illustrating with an Example of the Normal Distribution====
If we had five units that failed at 10, 20, 30, 40 and 50 hours, the mean would be:

<math>\begin{align}
\bar{T}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}} \\
= & \frac{10+20+30+40+50}{5} \\
= & 30
\end{align}</math>

The standard deviation estimate then would be:

<math>\begin{align}
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \\
= & \sqrt{\frac{{{(10-30)}^{2}}+{{(20-30)}^{2}}+{{(30-30)}^{2}}+{{(40-30)}^{2}}+{{(50-30)}^{2}}}{5}}, \\
= & 14.1421
\end{align}</math>

A look at the likelihood function surface plot in Figure A-1 reveals that both of these values are the maximum values of the function.

This three-dimensional plot represents the likelihood function. As can be seen from the plot, the maximum likelihood estimates for the two parameters correspond with the peak or maximum of the likelihood function surface.

Appendix A: Generating Random Numbers from a Distribution

2011-06-27T22:41:59Z

Steve Sharp:

Appendix A: Generating Random Numbers from a Distribution

2011-06-27T22:41:30Z

Steve Sharp: Created page with '= Sections = #Generating Random Times from a Weibull Distribution #Conditional #Regarding BlockSim's the Random Number Generator (RNG)'

= Sections =
#[[Generating Random Times from a Weibull Distribution]]
#[[Conditional]]
#[[Regarding BlockSim's the Random Number Generator (RNG)]]

Appendix B: References

2011-06-27T22:38:47Z

Steve Sharp: Created page with '1) Aitchison, J. and J.A.C. Brown, The Lognormal Distribution, Cambridge University Press, New York, 1957. 2) Barlow, R. and L. Hunter, Optimum Preventive Maintenance P…'

1) Aitchison, J. and J.A.C. Brown, The Lognormal Distribution, Cambridge University Press, New York, 1957.
 
 
2) Barlow, R. and L. Hunter, Optimum Preventive Maintenance Policies, Operations Research, Vol. 8, pp. 90-100, 1960.
 
 
3) Cramer, H., Mathematical Methods of Statistics, Princeton University Press, Princeton, NJ, 1946.
 
 
4) Davis, D.J., An Analysis of Some Failure Data, J. Am. Stat. Assoc., Vol. 47, 1952.
 
 
5) Dietrich, D., SIE 530 Engineering Statistics Lecture Notes, The University of Arizona, Tucson, Arizona.
 
 
6) Dudewicz, E.J. and S.N. Mishra, Modern Mathematical Statistics, John Wiley & Sons, Inc., New York, 1988.
 
 
7) Elsayed, E., Reliability Engineering, Addison Wesley, Reading, MA, 1996.
 
 
8) Hahn, G.J. and S.S. Shapiro, Statistical Models in Engineering, John Wiley & Sons, Inc., New York, 1967.
 
 
9) Kapur, K.C. and L.R. Lamberson, Reliability in Engineering Design, John Wiley & Sons, Inc., New York, 1977.
 
 
10) Kececioglu, D., Reliability Engineering Handbook, Prentice Hall, Inc., New Jersey, 1991.
 
 
11) Kececioglu, D., Maintainability, Availability, & Operational Readiness Engineering, Volume 1, Prentice Hall PTR, New Jersey, 1995.
 
 
12) Kijima, M. and Sumita, N., A useful generalization of renewal theory: counting process governed by nonnegative Markovian increments, Journal of Applied Probability, 23, 71--88, 1986.
 
 
13) Kijima, M., Some results for repairable systems with general repair, Journal of Applied Probability, 20, 851--859, 1989.
 
 
14) Knuth, D.E., The Art of Computer Programming: Volume 2 - Seminumerical Algorithms, Third Edition, Addison-Wesley, 1998.
 
 
15) L'Ecuyer, P., Communications of the ACM, Vol. 31, pp.724-774, 1988.
 
 
16) L'Ecuyer, P., Proceedings of the 2001 Winter Simulation Conference, pp.95-105, 2001.
 
 
17) Leemis, L.M., Reliability - Probabilistic Models and Statistical Methods, Prentice Hall, Inc., Englewood Cliffs, New Jersey, 1995.
 
 
18) Lloyd, D.K. and M. Lipow, Reliability: Management, Methods, and Mathematics, Prentice Hall, Englewood Cliffs, New Jersey, 1962.
 
 
19) Mann, N.R., R.E. Schafer and N.D. Singpurwalla, Methods for Statistical Analysis of Reliability and Life Data, John Wiley & Sons, Inc., New York, 1974.
 
 
20) Meeker, W.Q. and L.A. Escobar, Statistical Methods for Reliability Data, John Wiley & Sons, Inc., New York, 1998.
 
 
21) Mettas, A., Reliability Allocation and Optimization for Complex Systems, Proceedings of the Annual Reliability & Maintainability Symposium, 2000.
 
 
22) Nelson, W., Applied Life Data Analysis, John Wiley & Sons, Inc., New York, 1982.
 
 
23) Peters, E.E., Fractal Market Analysis: Applying Chaos Theory to Investment & Economics, John Wiley & Sons, 1994.
 
 
24) Press, W.H., S.A. Teukolsky, W.T. Vetterling and B.R. Flannery, Numerical Recipes in C: The Art of Scientific Computing, Second Edition, Cambridge University Press, 1988.
 
 
25) ReliaSoft Corporation, Life Data Analysis Reference, ReliaSoft Publishing, Tucson, Arizona, 2005.
 
 
26) Tillman, F.A., C.L. Hwang and W. Kuo, Optimization of Systems Reliability, Marcel Dekker, Inc., 1980.
 
 
27) Weibull, W., A Statistical Representation of Fatigue Failure in Solids, Transactions on the Royal Institute of Technology, No. 27, Stockholm, 1949.
 
 
28) Weibull, W., A Statistical Distribution Function of Wide Applicability, Journal of Applied Mechanics, Vol. 18, pp. 293-297, 1951.

Reliability Phase Diagrams (RPDs)

2011-06-27T22:11:54Z

Steve Sharp:

= Sections =
#[[Introduction to Reliability Phase Diagrams]]
#[[Types of Phases]]
#[[Cycles and Phase Diagram Execution]]
#[[Working with Phase Diagrams]]
#[[Operational Phase Properties]]
#[[Step-by-Step]]
#[[Rules & Assumptions]]
#[[Maintenance Phase Properties]]
#[[Understanding RPD Simulation Results]]
#[[Phase Throughput]]

Reliability Phase Diagrams (RPDs)

2011-06-27T22:09:58Z

Steve Sharp: Created page with '= Sections = #Introduction #Types of Phases #Cycles and Phase Diagram Execution #Working with Phase Diagrams #Operational Phase Properties #Step-by-Step #…'

= Sections =
#[[Introduction]]
#[[Types of Phases]]
#[[Cycles and Phase Diagram Execution]]
#[[Working with Phase Diagrams]]
#[[Operational Phase Properties]]
#[[Step-by-Step]]
#[[Rules & Assumptions]]
#[[Maintenance Phase Properties]]
#[[Understanding RPD Simulation Results]]
#[[Phase Throughput]]