ANOVA Calculations in Multiple Linear Regression: Difference between revisions

From ReliaWiki
Jump to navigation Jump to search
(Created page with 'ANOVA Calculations in Multiple Linear Regression The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in s…')
 
No edit summary
Line 1: Line 1:
ANOVA Calculations in Multiple Linear Regression  
=Reference Appendix A: ANOVA Calculations in Multiple Linear Regression=
 
The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in simple linear regression, except that the matrix notation is preferred in the case of multiple linear regression. In the case of both the simple and multiple linear regression models, once the observed and fitted values are available, the sum of squares are calculated in an identical manner. The difference between the two models lies in the way the fitted values are obtained. In a simple linear regression model, the fitted values are obtained from a model having only one predictor variable. In multiple linear regression analysis, the model used to obtained the fitted values contains more than one predictor variable.
The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in simple linear regression, except that the matrix notation is preferred in the case of multiple linear regression. In the case of both the simple and multiple linear regression models, once the observed and fitted values are available, the sum of squares are calculated in an identical manner. The difference between the two models lies in the way the fitted values are obtained. In a simple linear regression model, the fitted values are obtained from a model having only one predictor variable. In multiple linear regression analysis, the model used to obtained the fitted values contains more than one predictor variable.


   
   
==Total Sum of Squares==


Total Sum of Squares
Recall from Chapter 4 on simple linear regression that the total sum of squares, , is obtained using the following equation:
Recall from Chapter 4 on simple linear regression that the total sum of squares, , is obtained using the following equation:


Line 20: Line 21:


    
    
Model Sum of Squares  
==Model Sum of Squares==
 
Similarly, the model sum of squares or the regression sum of squares, , can be obtained in matrix notation as:
Similarly, the model sum of squares or the regression sum of squares, , can be obtained in matrix notation as:


Line 26: Line 28:
where  is the hat matrix and is calculated using .
where  is the hat matrix and is calculated using .


Error Sum of Squares  
==Error Sum of Squares==
 
The error sum of squares or the residual sum of squares, , is obtained in the matrix notation from the vector of residuals, , as:
The error sum of squares or the residual sum of squares, , is obtained in the matrix notation from the vector of residuals, , as:




    
    
Mean Squares  
==Mean Squares==
 
Mean squares are obtained by dividing the sum of squares with their associated degrees of freedom. The number of degrees of freedom associated with the total sum of squares, , is () since there are  observations in all, but one degree of freedom is lost in the calculation of the sample mean, . The total mean square is:
Mean squares are obtained by dividing the sum of squares with their associated degrees of freedom. The number of degrees of freedom associated with the total sum of squares, , is () since there are  observations in all, but one degree of freedom is lost in the calculation of the sample mean, . The total mean square is:


Line 45: Line 49:


    
    
Calculation of the Statistic
==Calculation of the Statistic==
 
Once the mean squares  and  are known, the statistic to test the significance of regression can be calculated as follows:
Once the mean squares  and  are known, the statistic to test the significance of regression can be calculated as follows:

Revision as of 00:12, 2 July 2011

Reference Appendix A: ANOVA Calculations in Multiple Linear Regression

The sum of squares for the analysis of variance in multiple linear regression is obtained using the same relations as those in simple linear regression, except that the matrix notation is preferred in the case of multiple linear regression. In the case of both the simple and multiple linear regression models, once the observed and fitted values are available, the sum of squares are calculated in an identical manner. The difference between the two models lies in the way the fitted values are obtained. In a simple linear regression model, the fitted values are obtained from a model having only one predictor variable. In multiple linear regression analysis, the model used to obtained the fitted values contains more than one predictor variable.


Total Sum of Squares

Recall from Chapter 4 on simple linear regression that the total sum of squares, , is obtained using the following equation:


The first term, , can be expressed in matrix notation using the vector of observed values, , as:


If represents an square matrix of ones, then the second term, , can be expressed in matrix notation as:


Therefore, the total sum of squares in matrix notation is: (31)

where is the identity matrix of order .


Model Sum of Squares

Similarly, the model sum of squares or the regression sum of squares, , can be obtained in matrix notation as:


where is the hat matrix and is calculated using .

Error Sum of Squares

The error sum of squares or the residual sum of squares, , is obtained in the matrix notation from the vector of residuals, , as:


Mean Squares

Mean squares are obtained by dividing the sum of squares with their associated degrees of freedom. The number of degrees of freedom associated with the total sum of squares, , is () since there are observations in all, but one degree of freedom is lost in the calculation of the sample mean, . The total mean square is:


The number of degrees of freedom associated with the regression sum of squares, , is . There are () degrees of freedom associated with a regression model with () coefficients, , , .... However, one degree of freedom is lost because the deviations, , are subjected to the constraints that they must sum to zero (). The regression mean square is:


The number of degrees of freedom associated with the error sum of squares is (), as there are observations in all, but () degrees of freedom are lost in obtaining the estimates of , , ... to calculate the predicted values, . The error mean square is:


The error mean square, , is an estimate of the variance, , of the random error terms, .


Calculation of the Statistic

Once the mean squares and are known, the statistic to test the significance of regression can be calculated as follows: