Appendix: Log-Likelihood Equations

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in the chapter for each distribution.

The Two-Parameter Weibull
This log-likelihood function is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\ & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right] \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\beta \,\!$$ is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)
 * $$\eta \,\!$$ is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval failure data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

For the purposes of MLE, left censored data will be considered to be intervals with $$T_{Li}^{\prime \prime }=0.\,\!$$

The solution will be found by solving for a pair of parameters $$\left( \widehat{\beta },\widehat{\eta } \right)\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \beta }=0\,\!$$ and $$\tfrac{\partial \Lambda }{\partial \eta }=0.\,\!$$ It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{\eta } \right) \\ & -\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\ & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}} \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\ & +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\ & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}} \end{align}\,\!$$

The Three-Parameter Weibull
This log-likelihood function is again composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\ & \\    & +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right] \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\beta \,\!$$ is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)
 * $$\eta \,\!$$ is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$\gamma \,\!$$ is the Weibull location parameter (unknown a priori, the third of three parameters to be found)
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in the $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * and $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

The solution is found by solving for $$\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \beta }=0,\,\!$$ $$\tfrac{\partial \Lambda }{\partial \eta }=0,\,\!$$ and $$\tfrac{\partial \Lambda }{\partial \gamma }=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\ & -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\ & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\ & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\ & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\ & -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{\mathop{\sum }}}\,\left( \frac{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\ & +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\ & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \end{align}\,\!$$

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if $$\beta \sim 1.\,\!$$ In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem, as discussed in Hirose [14].

Non-regularity occurs when $$\beta \le 2.\,\!$$ In general, there are no MLE solutions in the region of $$0<\beta <1.\,\!$$ When $$1<\beta <2,\,\!$$ MLE solutions exist but are not asymptotically normal, as discussed in Hirose [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ Application Setup), where $$\gamma \,\!$$ is estimated using non-linear regression. Once $$\gamma \,\!$$ is obtained, the MLE estimates of $$\widehat{\beta }\,\!$$ and $$\widehat{\eta }\,\!$$ are computed using the transformation $$T_{i}^{\prime }=({{T}_{i}}-\gamma ).\,\!$$

The One-Parameter Exponential
This log-likelihood function is composed of three summation portions:


 * $$\ln (L)=\Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\lambda \,\!$$ is the failure rate parameter (unknown a priori, the only parameter to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in the $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in the $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

The solution will be found by solving for a parameter $$\widehat{\lambda }\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \lambda }=0.\,\!$$ Note that for $$FI=0\,\!$$ there exists a closed form solution.


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\ & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right] \end{align}\,\!$$

The Two-Parameter Exponential
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:


 * $$\begin{align}

& \ln (L)= & \Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma  \right) \\ & & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma  \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma  \right)}} \right], \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\lambda \,\!$$ is the failure rate parameter (unknown a priori, the first of two parameters to be found)
 * $$\gamma \,\!$$ is the location parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in the $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in the $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

The two-parameter solution will be found by solving for a pair of parameters ($$\widehat{\lambda },\widehat{\gamma }),\,\!$$ such that $$\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.\,\!$$ For the one-parameter case, solve for $$\tfrac{\partial \Lambda }{\partial \lambda }=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\ & -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\ & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma  \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma  \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma  \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma  \right)}}} \right] \end{align}\,\!$$

and:


 * $$\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda \,\!$$

Examination the derivative for $$\gamma\,\!$$ will reveal that:


 * $$\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0\,\!$$

The above equation will be equal to zero only if either:


 * $$\lambda =0\,\!$$

or:


 * $$\left( \underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0\,\!$$

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both $$\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.\,\!$$ It can be shown that the best solution for $$\gamma ,\,\!$$ satisfying the constraint that $$\gamma \le {{T}_{1}}\,\!$$ is $$\gamma ={{T}_{1}}.\,\!$$ To then solve for the two-parameter exponential distribution via MLE, one can set  equal to the first time-to-failure, and then find a $$\lambda \,\!$$ such that $$\tfrac{\partial \Lambda }{\partial \lambda }=0.\,\!$$

Using this methodology, a maximum can be achieved along the $$\lambda \,\!$$-axis, and a local maximum along the $$\gamma \,\!$$-axis at $$\gamma ={{T}_{1}}\,\!$$, constrained by the fact that $$\gamma \le {{T}_{1}}\,\!$$. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:



Normal Log-Likelihood Functions and their Partials
The complete normal likelihood function (without the constant) is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\ & +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\ & \text{ }+\underset{i=1}{\overset{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_^{^{\prime \prime }}-\mu }{\sigma } \right) \right] \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\mu \,\!$$ is the mean parameter (unknown a priori, the first of two parameters to be found)
 * $$\sigma \,\!$$ is the standard deviation parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in the $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $${{F}_{i}}\,\!$$ is the number of interval data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in the $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

The solution will be found by solving for a pair of parameters $$\left( {{\mu }_{0}},{{\sigma }_{0}} \right)\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \mu }=0\,\!$$ and $$\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \mu }= & \frac{1}\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\ & +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\ & -\frac{1}{\sigma }\underset{i=1}{\overset{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)} \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\left( \frac-\frac{1}{\sigma } \right) \\ & +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\ & -\frac{1}{\sigma }\underset{i=1}{\overset{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)} \end{align}\,\!$$

where:


 * $$\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{2}}}\,\!$$

and:


 * $$\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{2}}}dt\,\!$$

Complete Data
Note that for the normal distribution, and in the case of complete data only (as was shown in Basic Statistical Background), there exists a closed-form solution for both of the parameters or:


 * $$\widehat{\mu }=\widehat=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}\,\!$$

and:


 * $$\begin{align}

\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\ {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \end{align}\,\!$$

Lognormal Log-Likelihood Functions and their Partials
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{T}_{i}} {{\sigma }_}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'} \right) \right] \\ & \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'} \right) \right] \\ & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'} \right) \right] \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $${\mu }'\,\!$$ is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)
 * $${{\sigma }_}\,\!$$ is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in the $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in the $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

The solution will be found by solving for a pair of parameters $$\left( {\mu }',{{\sigma }_} \right)\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial {\mu }'}=0\,\!$$ and $$\tfrac{\partial \Lambda }{\partial {{\sigma }_}}=0\,\!$$:


 * $$\begin{align}

\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _^{2}}\underset{i=1}{\overset{\mathop \sum }}\,N_{i}(\ln ({{T}_{i}})-{\mu }') \\ & +\frac{1}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'} \right)} \\ & \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'} \right)} \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial {{\sigma }_}}= & \underset{i=1}{\overset{\mathop \sum }}\,N_{i}\left( \frac{\sigma _^{3}}-\frac{1} \right) \\ & +\frac{1}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'} \right)} \\ & -\frac{1}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'} \right)} \end{align}\,\!$$

where:


 * $$\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{2}}}\,\!$$

and:


 * $$\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{2}}}dt\,\!$$

Mixed Weibull Log-Likelihood Functions and their Partials
The log-likelihood function (without the constant) is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\overset{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{\left( \frac{{{T}_{i}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}} \right)}^}}} \right] \\ & \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }} \right)}^}}} \right] \\ & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^}}} \right] \end{align}\,\!$$ where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$Q\,\!$$ is the number of subpopulations
 * $${{\rho }_{k}}\,\!$$ is the proportionality of the $${{k}^{th}}\,\!$$ subpopulation (unknown a priori, the first set of three sets of parameters to be found)
 * $${{\beta }_{k}}\,\!$$ is the Weibull shape parameter of the $${{k}^{th}}\,\!$$ subpopulation (unknown a priori, the second set of three sets of parameters to be found)
 * $${{\eta }_{k}}\,\!$$ is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of groups of interval data points
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

The solution will be found by solving for a group of parameters:


 * $$\left( \widehat\widehat,\widehat,\widehat\widehat,\widehat,...,\widehat\widehat,\widehat \right)\,\!$$

so that:


 * $$\begin{align}

\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\ \frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\ \vdots \\ \frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\ \frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0 \end{align}\,\!$$

Logistic Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\ & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}} \right) \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\mu \,\!$$ is the logistic shape parameter (unknown a priori, the first of two parameters to be found)
 * $$\eta \,\!$$ is the logistic scale parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval failure data group
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

For the purposes of MLE, left censored data will be considered to be intervals with $$T_{Li}^{\prime \prime }=0.\,\!$$

The solution of the maximum log-likelihood function is found by solving for ($$\widehat{\mu },\widehat{\sigma })\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\ & -\frac{\underset{i=1}{\mathop{\overset{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{1+{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{1+{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}} \right) \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\ & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}} \\  & -\frac{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}) \end{align}\,\!$$

The Loglogistic Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\ & -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\ & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}} \right) \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\mu \,\!$$ is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)
 * $$\sigma \,\!$$ is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval failure data groups,
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

For the purposes of MLE, left censored data will be considered to be intervals with $$T_{Li}^{\prime \prime }=0.\,\!$$

The solution of the maximum log-likelihood function is found by solving for ($$\widehat{\mu },\widehat{\sigma })\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{\sigma } \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{1+{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{1+{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}} \right) \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\ & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}} \\   & -\frac{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_^{^{\prime \prime }})-\mu }{\sigma }}}}) \end{align}\,\!$$

The Gumbel Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{\sigma } \right) \\ & -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\ & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}} \right) \end{align}\,\!$$

or:


 * $$\begin{align}

\Lambda = & \underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\ & +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\ & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}} \right) \end{align}\,\!$$

where:


 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\mu \,\!$$ is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)
 * $$\sigma \,\!$$ is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval failure data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

For the purposes of MLE, left censored data will be considered to be intervals with $$T_{Li}^{\prime \prime }=0.\,\!$$

The solution of the maximum log-likelihood function is found by solving for ($$\widehat{\mu },\widehat{\sigma })\,\!$$ so that:


 * $$\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right) \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\ & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\ & \left( \frac{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right) \end{align}\,\!$$

The Gamma Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:


 * $$\begin{align}

\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}\Gamma (k)} \right) \\ & +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\   & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right) \right)  \end{align}\,\!$$

or:


 * $$\begin{align}

\Lambda = & \underset{i=1}{\mathop{\overset{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\ & \underset{i=1}{\mathop{\overset{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\ & +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\ & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu )}} \right) \right) \end{align}\,\!$$

where:
 * $${{F}_{e}}\,\!$$ is the number of groups of times-to-failure data points
 * $${{N}_{i}}\,\!$$ is the number of times-to-failure in the $${{i}^{th}}\,\!$$ time-to-failure data group
 * $$\mu \,\!$$ is the gamma shape parameter (unknown a priori, the first of two parameters to be found)
 * $$k\,\!$$ is the gamma scale parameter (unknown a priori, the second of two parameters to be found)
 * $${{T}_{i}}\,\!$$ is the time of the $${{i}^{th}}\,\!$$ group of time-to-failure data
 * $$S\,\!$$ is the number of groups of suspension data points
 * $$N_{i}^{\prime }\,\!$$ is the number of suspensions in $${{i}^{th}}\,\!$$ group of suspension data points
 * $$T_{i}^{\prime }\,\!$$ is the time of the $${{i}^{th}}\,\!$$ suspension data group
 * $$FI\,\!$$ is the number of interval failure data groups
 * $$N_{i}^{\prime \prime }\,\!$$ is the number of intervals in $${{i}^{th}}\,\!$$ group of data intervals
 * $$T_{Li}^{\prime \prime }\,\!$$ is the beginning of the $${{i}^{th}}\,\!$$ interval
 * and $$T_{Ri}^{\prime \prime }\,\!$$ is the ending of the $${{i}^{th}}\,\!$$ interval

For the purposes of MLE, left censored data will be considered to be intervals with $$T_{Li}^{\prime \prime }=0.\,\!$$

The solution of the maximum log-likelihood function is found by solving for ($$\widehat{\mu },\widehat{\sigma })\,\!$$ so that $$\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\ & +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\ & +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)} \\ & -\frac{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)}\} \end{align}\,\!$$


 * $$\begin{align}

\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\ & -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\ & +\underset{i=1}{\mathop{\overset{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_^{^{\prime \prime }})-\mu }}) \right)} \right) \end{align}\,\!$$