Multiple regression is an extension of simple linear regression. It is used when we want to predict the value of a variable based on the value of two or more other variables. The variable we want to predict is called the dependent variable (or sometimes, the outcome, target or criterion variable). The variables we are using to predict the value of the dependent variable are called the independent variables (or sometimes, the predictor, explanatory or regressor variables).
Multiple regression also allows you to determine the overall fit (variance explained) of the model and the relative contribution of each of the predictors to the total variance explained.
The Multiple Regression Model
Multiple regression analysis helps us to predict the value of Y for given values of X1, X2, …, Xk. In general, the multiple regression equation of Y on X1, X2, …, Xk is given by:
Y = b0 + b1 X1 + b2 X2 + …………………… + bk Xk
Here b0 is the intercept or constant and b1, b2, b3, …, bk are analogous to the slope in linear regression equation and are also called regression coefficients or regression weights. They can be interpreted the same way as slope. Thus if bi = 2.5, it would indicates that Y will increase by 2.5 units if Xi increased by 1 unit.
The problem is to find the values of b1 and b2 and so on in the equation shown below that give the best predictions of Y.
You must understand the following three venn diagrams in order to understand the concepts of multiple correlation and multiple regression.
Diagram 1: The following diagram presents how 2 variables are related to each other.
Diagram 2: The following diagram presents how 2 independent variables are related to a dependent variable; while the 2 independent variable are not related to each other.
Diagram 3: The following diagram presents how 2 independent variables are related to a dependent variable while the 2 independent variables are related to each other.
Source: Biddle website