Regression

Regression:
Mathematical model to predict one variable from another. One variable is considered the dependent variable that can be predicted by one or more independent variables. Based on the assumption that a change in one variable results in a change in another.

1) Linear/Sample/Bivariate regression:
Correlation between two variables – uses the formula for the ‘best fit line’ or regression line.

Analysis of Residuals:
The vertical distance from the line to a data point is called the residual. Analysis of the residuals allows determination if the assumptions of regression analysis have been met.

Assumption of simple regression analysis:
• The relationship between the variables is approximately linear
• The relationship between the variables is approximately normally distributed
• The residuals are normally distributed about the fitted line
• The residuals are independent of each other

Limitations:
• accuracy dependent on size of residuals
• only for linear relationships

2) Nonlinear regression:
eg polynomial

3) Multiple regression:
Used when one dependent variable (Y) is related to two or more independent variables (X1, X2, X3…)

Stepwise multiple regression:
The regression is done in steps so the contribution of each variable is evaluated independently and sequentially  gives the best set of independent variables to explain the dependent variable

Assumptions and regression:
• The ratio of subjects to independent variables should be at least 5:1 – ideally about 20:1.
• Outliers (which will have a large residual)  large and inappropriate leverage in equation
• The variances for each set of residuals should be approximately equal – called homoscedasticity – a lack of uniformity of variance is called heteroscdasticity
• Ideally the independent variables should be related to the dependent variables and not to each other. Variables that are related to one another do not add anything more when added to a regression equation – called multicollinearity

We have not yet got to this page to finish it yet. We will eventually. Please contact us if you have something to contribute to it or sign up for our newsletter or like us on Facebook and Instagram or follow us on Twitter.

Page last updated: @ 4:47 am