Appendix: Maximum Likelihood Estimation Example - Revision history

Lisa Hacker at 19:13, 15 September 2023

2023-09-15T19:13:05Z

← Older revision		Revision as of 19:13, 15 September 2023
Line 1:		Line 1:
	{{Template:~~LDABOOK_SUB~~\|Appendix B\|Maximum Likelihood Estimation Example}}		{{Template:LDABOOK\|Appendix B\|Maximum Likelihood Estimation Example}}

	==MLE Statistical Background==		==MLE Statistical Background==

Lisa Hacker: /* Illustrating the MLE Method Using Normal Distribution */

2023-03-10T18:22:34Z

Illustrating the MLE Method Using Normal Distribution

← Older revision		Revision as of 18:22, 10 March 2023
Line 55:		Line 55:
	Note that the value of <math>\widehat{\lambda }\,\!</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda \,\!</math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }\,\!</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda \,\!</math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in [[Confidence Bounds]].)		Note that the value of <math>\widehat{\lambda }\,\!</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda \,\!</math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }\,\!</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda \,\!</math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in [[Confidence Bounds]].)

	==Illustrating ~~the~~ MLE ~~Method~~ Using Normal Distribution==		==Illustrating MLE Using the Normal Distribution==
	To obtain the MLE estimates for the mean, <math>\bar{T},\,\!</math> and standard deviation, <math>{{\sigma }_{T}},\,\!</math> for the normal distribution, start with the ''pdf'' of the normal distribution which is given by:		To obtain the MLE estimates for the mean, <math>\bar{T},\,\!</math> and standard deviation, <math>{{\sigma }_{T}},\,\!</math> for the normal distribution, start with the ''pdf'' of the normal distribution which is given by:

Lisa Hacker: /* Illustrating the MLE Method Using the Exponential Distribution */

2023-03-10T18:21:49Z

Illustrating the MLE Method Using the Exponential Distribution

← Older revision		Revision as of 18:21, 10 March 2023
Line 28:		Line 28:
	Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other, and in no way suggests that the solution is wrong.		Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other, and in no way suggests that the solution is wrong.

	==Illustrating ~~the~~ MLE ~~Method~~ Using the Exponential Distribution==		==Illustrating MLE Using the Exponential Distribution==
	*To estimate <math>\widehat{\lambda }\,\!</math> for a sample of <math>n\,\!</math> units (all tested to failure), first obtain the likelihood function:		*To estimate <math>\widehat{\lambda }\,\!</math> for a sample of <math>n\,\!</math> units (all tested to failure), first obtain the likelihood function:

Richard House at 18:24, 26 September 2012

2012-09-26T18:24:43Z

Show changes

Richard House at 03:29, 29 August 2012

2012-08-29T03:29:48Z

← Older revision		Revision as of 03:29, 29 August 2012
Line 9:		Line 9:
	\end{align}</math>		\end{align}</math>

	where <math>{{\theta }_{1}},\,\!</math> <math>{{\theta }_{2}},\,\!</math><math>...,\,\!</math> <math>{{\theta }_{k}}\,\!</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N</math> independent observations, <math>{{x}_{1}},\,\!</math> <math>{{x}_{2}},\,\!</math><math>...,\,\!</math> <math>{{x}_{N}}\,\!</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:		where <math>{{\theta }_{1}},\,\!</math> <math>{{\theta }_{2}},\,\!</math><math>...,\,\!</math> <math>{{\theta }_{k}}\,\!</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N\,\!</math> independent observations, <math>{{x}_{1}},\,\!</math> <math>{{x}_{2}},\,\!</math><math>...,\,\!</math> <math>{{x}_{N}}\,\!</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:

	::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}\|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>		::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}\|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>
Line 56:		Line 56:

	==Illustrating the MLE Method Using Normal Distribution==		==Illustrating the MLE Method Using Normal Distribution==
	To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},\,\!</math> for the normal distribution, start with the ~~<math>~~pdf~~</math>~~ of the normal distribution which is given by:		To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},\,\!</math> for the normal distribution, start with the ''pdf'' of the normal distribution which is given by:

	::<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>		::<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

Richard House: /* MLE Statistical Background */

2012-08-20T22:54:35Z

MLE Statistical Background

← Older revision		Revision as of 22:54, 20 August 2012
Line 22:		Line 22:

	The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},\,\!</math> are obtained by maximizing <math>L\,\!</math> or <math>\Lambda .\,\!</math>		The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},\,\!</math> are obtained by maximizing <math>L\,\!</math> or <math>\Lambda .\,\!</math>
	By maximizing <math>\Lambda ,\,\!</math> which is much easier to work with than <math>L</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}\,\!</math> are the simultaneous solutions of <math>k</math> equations such that:		By maximizing <math>\Lambda ,\,\!</math> which is much easier to work with than <math>L\,\!</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}\,\!</math> are the simultaneous solutions of <math>k</math> equations such that:

	::<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>		::<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>

Richard House: /* Illustrating the MLE Method Using the Exponential Distribution */

2012-08-20T22:53:57Z

Illustrating the MLE Method Using the Exponential Distribution

← Older revision		Revision as of 22:53, 20 August 2012
Line 41:		Line 41:
	::<math>\Lambda =\ln (L)=n\ln (\lambda )-\lambda \underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}.</math>		::<math>\Lambda =\ln (L)=n\ln (\lambda )-\lambda \underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}.</math>

	*Obtain <math>\tfrac{\partial \Lambda }{\partial \lambda }</math>, and set it equal to zero:		*Obtain <math>\tfrac{\partial \Lambda }{\partial \lambda }\,\!</math>, and set it equal to zero:

	::<math>\frac{\partial \Lambda }{\partial \lambda }=\frac{n}{\lambda }-\underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}=0</math>		::<math>\frac{\partial \Lambda }{\partial \lambda }=\frac{n}{\lambda }-\underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}=0</math>

Richard House: /* Illustrating the MLE Method Using Normal Distribution */

2012-08-20T22:53:18Z

Illustrating the MLE Method Using Normal Distribution

← Older revision		Revision as of 22:53, 20 August 2012
Line 56:		Line 56:

	==Illustrating the MLE Method Using Normal Distribution==		==Illustrating the MLE Method Using Normal Distribution==
	To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},</math> for the normal distribution, start with the <math>pdf</math> of the normal distribution which is given by:		To obtain the MLE estimates for the mean, <math>\bar{T},</math> and standard deviation, <math>{{\sigma }_{T}},\,\!</math> for the normal distribution, start with the <math>pdf</math> of the normal distribution which is given by:

	::<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>		::<math>f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}}</math>

	If <math>{{T}_{1}},{{T}_{2}},...,{{T}_{N}}</math> are known times-to-failure (and with no suspensions), then the likelihood function is given by:		If <math>{{T}_{1}},{{T}_{2}},...,{{T}_{N}}\,\!</math> are known times-to-failure (and with no suspensions), then the likelihood function is given by:

	::<math>L({{T}_{1}},{{T}_{2}},...,{{T}_{N}}\|\bar{T},{{\sigma }_{T}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left[ \frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} \right]</math>		::<math>L({{T}_{1}},{{T}_{2}},...,{{T}_{N}}\|\bar{T},{{\sigma }_{T}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left[ \frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} \right]</math>
Line 71:		Line 71:
	::<math>\Lambda =\ln L=-\frac{N}{2}\ln (2\pi )-N\ln {{\sigma }_{T}}-\frac{1}{2}\underset{i=1}{\overset{N}{\mathop \sum }}\,\left( \frac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)_{}^{2}</math>		::<math>\Lambda =\ln L=-\frac{N}{2}\ln (2\pi )-N\ln {{\sigma }_{T}}-\frac{1}{2}\underset{i=1}{\overset{N}{\mathop \sum }}\,\left( \frac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)_{}^{2}</math>

	Then taking the partial derivatives of <math>\Lambda </math> with respect to each one of the parameters and setting them equal to zero yields:		Then taking the partial derivatives of <math>\Lambda \,\!</math> with respect to each one of the parameters and setting them equal to zero yields:

	::<math>\frac{\partial (\Lambda )}{\partial \bar{T}}=\frac{1}{\sigma _{T}^{2}}\underset{i=1}{\overset{N}{\mathop \sum }}\,({{T}_{i}}-\bar{T})=0</math>		::<math>\frac{\partial (\Lambda )}{\partial \bar{T}}=\frac{1}{\sigma _{T}^{2}}\underset{i=1}{\overset{N}{\mathop \sum }}\,({{T}_{i}}-\bar{T})=0</math>

Richard House: /* Illustrating the MLE Method Using the Exponential Distribution */

2012-08-20T22:52:35Z

Illustrating the MLE Method Using the Exponential Distribution

← Older revision		Revision as of 22:52, 20 August 2012
Line 29:		Line 29:

	==Illustrating the MLE Method Using the Exponential Distribution==		==Illustrating the MLE Method Using the Exponential Distribution==
	*To estimate <math>\widehat{\lambda }</math> for a sample of <math>n</math> units (all tested to failure), first obtain the likelihood function:		*To estimate <math>\widehat{\lambda }</math> for a sample of <math>n\,\!</math> units (all tested to failure), first obtain the likelihood function:

	::<math>\begin{align}		::<math>\begin{align}
Line 53:		Line 53:
	'''Notes About <math>\widehat{\lambda }</math>'''		'''Notes About <math>\widehat{\lambda }</math>'''

	Note that the value of <math>\widehat{\lambda }</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda </math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda </math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in [[Confidence Bounds]].)		Note that the value of <math>\widehat{\lambda }</math> is an estimate because if we obtain another sample from the same population and re-estimate <math>\lambda \,\!</math>, the new value would differ from the one previously calculated. In plain language, <math>\hat{\lambda }\,\!</math> is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case <math>\lambda \,\!</math>. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in [[Confidence Bounds]].)

	==Illustrating the MLE Method Using Normal Distribution==		==Illustrating the MLE Method Using Normal Distribution==

Richard House: /* MLE Statistical Background */

2012-08-20T22:51:43Z

MLE Statistical Background

← Older revision		Revision as of 22:51, 20 August 2012
Line 5:		Line 5:
	If <math>x</math> is a continuous random variable with ''pdf'':		If <math>x</math> is a continuous random variable with ''pdf'':

	::<math>f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}),</math>		::<math>\begin{align}
			f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}),
			\end{align}</math>

	where <math>{{\theta }_{1}},</math> <math>{{\theta }_{2}},</math><math>...,</math> <math>{{\theta }_{k}}</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N</math> independent observations, <math>{{x}_{1}},</math> <math>{{x}_{2}},</math><math>...,</math> <math>{{x}_{N}}</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:		where <math>{{\theta }_{1}},\,\!</math> <math>{{\theta }_{2}},\,\!</math><math>...,\,\!</math> <math>{{\theta }_{k}}\,\!</math> are <math>k</math> unknown constant parameters that need to be estimated, conduct an experiment and obtain <math>N</math> independent observations, <math>{{x}_{1}},\,\!</math> <math>{{x}_{2}},\,\!</math><math>...,\,\!</math> <math>{{x}_{N}}\,\!</math>, which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:

	::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}\|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>		::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}\|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>
Line 19:		Line 21:
	::<math>\Lambda =\ln L=\underset{i=1}{\overset{N}{\mathop \sum }}\,\ln f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>		::<math>\Lambda =\ln L=\underset{i=1}{\overset{N}{\mathop \sum }}\,\ln f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})</math>

	The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},</math> are obtained by maximizing <math>L</math> or <math>\Lambda .</math>		The maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}},\,\!</math> are obtained by maximizing <math>L\,\!</math> or <math>\Lambda .\,\!</math>
	By maximizing <math>\Lambda ,</math> which is much easier to work with than <math>L</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}</math> are the simultaneous solutions of <math>k</math> equations such that:		By maximizing <math>\Lambda ,\,\!</math> which is much easier to work with than <math>L</math>, the maximum likelihood estimators (MLE) of <math>{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}\,\!</math> are the simultaneous solutions of <math>k</math> equations such that:

	::<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>		::<math>\frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k</math>