Template:Normal distribution maximum likelihood estimation - Revision history

Lisa Hacker: Redirected page to The Normal Distribution

2012-08-03T09:06:44Z

Redirected page to The Normal Distribution

← Older revision		Revision as of 09:06, 3 August 2012
Line 1:		Line 1:
	~~===Maximum Likelihood Estimation===~~		#REDIRECT [[The_Normal_Distribution]]
	~~As it was outlined in chapter~~ [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in [[Appendix: Distribution Log-Likelihood Equations]].


	~~'''Special Note About Bias'''~~

	Estimators (i.e. parameter estimates) have properties such as unbiasedness, minimum variance, sufficiency, consistency, squared error constancy, efficiency and completeness [[Appendix: Weibull References\|[7][5]]]. Numerous books and papers deal with these properties and this coverage is beyond the scope of this reference.

	However, we would like to briefly address one of these properties, unbiasedness. An estimator is said to be unbiased if the estimator <math>\widehat{\theta }=d({{X}_{1,}}{{X}_{2,}}...,{{X}_{n)}}</math> satisfies the condition <math>E\left[ \widehat{\theta } \right]</math> <math>=\theta </math> for all <math>\theta \in \Omega .</math>

	~~Note that <math>E\left[ X \right]</math> denotes the expected value of X and is defined (for continuous distributions) by:~~

	~~::<math>\begin{align}~~
	~~E\left[ X \right]= \int_{\varpi }x\cdot f(x)dx \\~~
	~~X\in & \varpi .~~
	~~\end{align}</math>~~


	It can be shown [[Appendix: Weibull References\|[7][5]]] that the MLE estimator for the mean of the normal (and lognormal) distribution does satisfy the unbiasedness criteria, or <math>E\left[ \widehat{\mu } \right]</math> <math>=\mu .</math> The same is not true for the estimate of the variance <math>\hat{\sigma }^{2}</math> . The maximum likelihood estimate for the variance for the normal distribution is given by:

	~~::<math>\hat{\sigma }^{2}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}}</math>~~

	~~with a standard deviation of:~~

	~~::<math>{{\hat{\sigma }}}=\sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}}}</math>~~


	These estimates, however, have been shown to be biased. It can be shown [[Appendix: Weibull References\|[7][5]]] that the unbiased estimate of the variance and standard deviation for complete data is given by:


	~~::<math>\begin{align}~~
	\hat{\sigma }^{2}= & \left[ \frac{N}{N-1} \right]\cdot \left[ \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}} \right]=\frac{1}{N-1}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}} \\
	~~{{{\hat{\sigma }}}}= & \sqrt{\left[ \frac{N}{N-1} \right]\cdot \left[ \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}} \right]} \\~~
	~~= & \sqrt{\frac{1}{N-1}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}}}~~
	~~\end{align}</math>~~


	~~Note that for larger values of <math>N</math> , <math>\sqrt{\left[ N/(N-1) \right]}</math> tends to 1.~~

	~~The Use Unbiased Std on Normal Data option in the User Setup under the Calculations tab allows biasing to be considered when estimating the parameters.~~

	When this option is selected, Weibull++ returns the unbiased standard deviation as defined. This is only true for complete data sets. For all other data types, Weibull++ by default returns the biased standard deviation as defined above regardless of the selection status of this option. The next figure shows this setting in Weibull++.


	~~<math></math>~~
	~~[[Image:Weibull Calculation User Setting.png\|thumb\|center\|250px\|~~ ]]

Nicolette Young: /* Maximum Likelihood Estimation */

2012-04-25T19:00:25Z

Maximum Likelihood Estimation

← Older revision		Revision as of 19:00, 25 April 2012
Line 44:		Line 44:

	<math></math>		<math></math>
	[[Image:Weibull Calculation User Setting.png\|thumb\|center\|~~400px~~\| ]]		[[Image:Weibull Calculation User Setting.png\|thumb\|center\|250px\| ]]

Nicolette Young: /* Maximum Likelihood Estimation */

2012-03-14T19:23:36Z

Maximum Likelihood Estimation

← Older revision		Revision as of 19:23, 14 March 2012
Line 44:		Line 44:

	<math></math>		<math></math>
	[[Image:Weibull Calculation User Setting.png\|thumb\|center\|~~300px~~\| ]]		[[Image:Weibull Calculation User Setting.png\|thumb\|center\|400px\| ]]

Harry Guo: /* Maximum Likelihood Estimation */

2012-03-02T22:57:08Z

Maximum Likelihood Estimation

← Older revision		Revision as of 22:57, 2 March 2012
Line 44:		Line 44:

	<math></math>		<math></math>
	[[Image:~~ldachp8fig3~~.~~gif~~\|thumb\|center\|300px\| ]]		[[Image:Weibull Calculation User Setting.png\|thumb\|center\|300px\| ]]

Harry Guo: /* Maximum Likelihood Estimation */

2012-02-13T18:24:46Z

Maximum Likelihood Estimation

← Older revision		Revision as of 18:24, 13 February 2012
Line 1:		Line 1:
	===Maximum Likelihood Estimation===		===Maximum Likelihood Estimation===
	As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in [[Appendix: Distribution Log-Likelihood Equations]].		As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in [[Appendix: Distribution Log-Likelihood Equations]].


			'''Special Note About Bias'''

			Estimators (i.e. parameter estimates) have properties such as unbiasedness, minimum variance, sufficiency, consistency, squared error constancy, efficiency and completeness [[Appendix: Weibull References\|[7][5]]]. Numerous books and papers deal with these properties and this coverage is beyond the scope of this reference.

			However, we would like to briefly address one of these properties, unbiasedness. An estimator is said to be unbiased if the estimator <math>\widehat{\theta }=d({{X}_{1,}}{{X}_{2,}}...,{{X}_{n)}}</math> satisfies the condition <math>E\left[ \widehat{\theta } \right]</math> <math>=\theta </math> for all <math>\theta \in \Omega .</math>

			Note that <math>E\left[ X \right]</math> denotes the expected value of X and is defined (for continuous distributions) by:

			::<math>\begin{align}
			E\left[ X \right]= \int_{\varpi }x\cdot f(x)dx \\
			X\in & \varpi .
			\end{align}</math>


			It can be shown [[Appendix: Weibull References\|[7][5]]] that the MLE estimator for the mean of the normal (and lognormal) distribution does satisfy the unbiasedness criteria, or <math>E\left[ \widehat{\mu } \right]</math> <math>=\mu .</math> The same is not true for the estimate of the variance <math>\hat{\sigma }^{2}</math> . The maximum likelihood estimate for the variance for the normal distribution is given by:

			::<math>\hat{\sigma }^{2}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}}</math>

			with a standard deviation of:

			::<math>{{\hat{\sigma }}}=\sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}}}</math>


			These estimates, however, have been shown to be biased. It can be shown [[Appendix: Weibull References\|[7][5]]] that the unbiased estimate of the variance and standard deviation for complete data is given by:


			::<math>\begin{align}
			\hat{\sigma }^{2}= & \left[ \frac{N}{N-1} \right]\cdot \left[ \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}} \right]=\frac{1}{N-1}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}} \\
			{{{\hat{\sigma }}}}= & \sqrt{\left[ \frac{N}{N-1} \right]\cdot \left[ \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}} \right]} \\
			= & \sqrt{\frac{1}{N-1}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{t}_{i}}-\bar{T})}^{2}}}
			\end{align}</math>


			Note that for larger values of <math>N</math> , <math>\sqrt{\left[ N/(N-1) \right]}</math> tends to 1.

			The Use Unbiased Std on Normal Data option in the User Setup under the Calculations tab allows biasing to be considered when estimating the parameters.

			When this option is selected, Weibull++ returns the unbiased standard deviation as defined. This is only true for complete data sets. For all other data types, Weibull++ by default returns the biased standard deviation as defined above regardless of the selection status of this option. The next figure shows this setting in Weibull++.


			<math></math>
			[[Image:ldachp8fig3.gif\|thumb\|center\|300px\| ]]

Harry Guo at 18:21, 13 February 2012

2012-02-13T18:21:50Z

← Older revision		Revision as of 18:21, 13 February 2012
Line 1:		Line 1:
	===Maximum Likelihood Estimation===		===Maximum Likelihood Estimation===
	As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in [[Appendix: Distribution Log-Likelihood Equations]].		As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in [[Appendix: Distribution Log-Likelihood Equations]].

	~~'''Example 4:'''~~
	~~{{Example: Normal Distribution MLE}}~~

Harry Guo: /* Maximum Likelihood Estimation */

2012-02-10T23:36:19Z

Maximum Likelihood Estimation

← Older revision		Revision as of 23:36, 10 February 2012
Line 1:		Line 1:
	===Maximum Likelihood Estimation===		===Maximum Likelihood Estimation===
	As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in Appendix C.		As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in [[Appendix: Distribution Log-Likelihood Equations]].

	'''Example 4:'''		'''Example 4:'''
	{{Example: Normal Distribution MLE}}		{{Example: Normal Distribution MLE}}

Harry Guo: /* Maximum Likelihood Estimation */

2012-02-10T21:05:34Z

Maximum Likelihood Estimation

← Older revision		Revision as of 21:05, 10 February 2012
Line 1:		Line 1:
	===Maximum Likelihood Estimation===		===Maximum Likelihood Estimation===
	As it was outlined in chapter [[Parameter ~~Estimations~~]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in Appendix C.		As it was outlined in chapter [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in Appendix C.

	'''Example 4:'''		'''Example 4:'''
	{{Example: Normal Distribution MLE}}		{{Example: Normal Distribution MLE}}

Harry Guo: /* Maximum Likelihood Estimation */

2012-02-10T21:05:17Z

Maximum Likelihood Estimation

← Older revision		Revision as of 21:05, 10 February 2012
Line 1:		Line 1:
	===Maximum Likelihood Estimation===		===Maximum Likelihood Estimation===
	As it was outlined in ~~Chapter 3~~, maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in Appendix C.		As it was outlined in chapter [[Parameter Estimations]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in Appendix C.

	'''Example 4:'''		'''Example 4:'''
	{{Example: Normal Distribution MLE}}		{{Example: Normal Distribution MLE}}

Harry Guo: /* Maximum Likelihood Estimation */

2012-02-10T19:03:02Z

Maximum Likelihood Estimation

@@ Line 2: / Line 2: @@
 As it was outlined in Chapter 3, maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution.  Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the normal distribution are covered in Appendix C.
-'''Special Note About Bias'''
+'''Example 4:'''
+{{Example: Normal Distribution MLE}}