Multiple
Regression
Multiple
regression is an extension of the simple regression model that allows us into incorporate
more than one independent variable into our explanation of Y. It helps clarify the relationship between each
independent variable and Y by holding constant the effects of the other independent
variables. Its assumptions and interpretation
are very similar to the simple regression model.
General
equation: _{}
Interpretation: The value of Y is determined by a linear combination
of the independent variables (X_{k}) plus an error term (e).
We continue
to use the least squares approach for fitting the data based on the formula for the
straight line, You should recall this as the least sum of the squared differences between
observed Y and predicted Y. _{}
However, you
cannot visualize the fit graphically on a two dimensional scattergram. The line is best visualized on a three or more
dimensional plane.
Partial
slope equation (Three
variable example)
_{}
Process:
1. Assume
there is some correlation between the independent variables.
_{}
2. Measure
portion of X_{1} not explained by X_{2}, which is a simple application of
the bivariate model through the following process:
Use
equation for a straight line to predict X_{1} given X_{2}
_{}
(this
notation is analogous to _{})
Compute
the prediction error where u represents the portion of X_{1} which X_{2}
cannot explain.
_{}
2. Repeat
steps to measure portion of Y which is not explained by X_{2}:
Use
equation for a straight line to predict Y given X_{2}
_{}
(this
notation is analogous to _{})
Compute
the prediction error where v represents the portion of Y which X_{2} cannot
explain.
_{}
3. The above
is incorporated into the formula for b_{1}
_{}
or _{}
This
equation results in a measure of the slope for X_{1} that is independent of the
linear effect of X_{2} on X_{1} and Y.
Put in more applied terms, a multiple regression model allows us to evaluate
the spuriousness of relationships. As an
example. We found in our bivariate model that
education has a significant causal relationship with income.
Given the simplicity of this model, we don't know if the relationship might
disappear if we controlled for employee experience. In
adding experience to our model, we face three possible outcomes:
1.
Education is
significant but experience is not (evidence
bivariate relationship between education and income is not spurious for experience)
2.
Experience
is significant but education is not (evidence
bivariate relationship between education and income is spurious)
3.
Both
education and experience are significant (evidence
bivariate relationship between education and income is not spurious)
4.
Neither are
significant (evidence of high correlation between education and experience)
Partial Slope Example
