Learn moreLearn moreApplied Statistics Handbook

Table of Contents

 


 

Multiple Regression 

Multiple regression is an extension of the simple regression model that allows us into incorporate more than one independent variable into our explanation of Y.  It helps clarify the relationship between each independent variable and Y by holding constant the effects of the other independent variables.  Its assumptions and interpretation are very similar to the simple regression model.

 

 

General equation:    

 

 

Interpretation:  The value of Y is determined by a linear combination of the independent variables (Xk) plus an error term (e).

 

 

We continue to use the least squares approach for fitting the data based on the formula for the straight line, You should recall this as the least sum of the squared differences between observed Y and predicted Y.

 

 

However, you cannot visualize the fit graphically on a two dimensional scattergram.  The line is best visualized on a three or more dimensional plane.

 

 

Partial slope equation (Three variable example)

 

 

Process:

 

1. Assume there is some correlation between the independent variables.

 

 

2. Measure portion of X1 not explained by X2, which is a simple application of the bivariate model through the following process:

 

Use equation for a straight line to predict X1 given X2

 

 

 

 

(this notation is analogous to )

 

Compute the prediction error where u represents the portion of X1 which X2 cannot explain.

 

 

2. Repeat steps to measure portion of Y which is not explained by X2:

 

Use equation for a straight line to predict Y given X2

 

 

 

(this notation is analogous to )

 

Compute the prediction error where v represents the portion of Y which X2 cannot explain.

 

3. The above is incorporated into the formula for b1

 

                        or        

 

This equation results in a measure of the slope for X1 that is independent of the linear effect of X2 on X1 and Y.   Put in more applied terms, a multiple regression model allows us to evaluate the spuriousness of relationships.  As an example.  We found in our bivariate model that education has a significant causal relationship with income.   Given the simplicity of this model, we don't know if the relationship might disappear if we controlled for employee experience.  In adding experience to our model, we face three possible outcomes:

 

1.     Education is significant but experience is not (evidence bivariate relationship between education and income is not spurious for experience)

2.     Experience is significant but education is not (evidence bivariate relationship between education and income is spurious)

3.     Both education and experience are significant (evidence bivariate relationship between education and income is not spurious)

4.     Neither are significant (evidence of high correlation between education and experience)

 

Partial Slope Example


Google

 


Copyright 2015, AcaStat Software. All Rights Reserved.