Regression Optimization and Estimation

Abstract

Linear regression is an important and widely used approach amongst the wealth of machine learning techniques.This approach is used for prediction and forecasting by using massive datasets that have high dimensionality. The results obtained are beneficial for both the organization and the consumer for better decision making. Numerous different models have been proposed for selecting the best regression model i.e. is the model having less dimensionality and high degree of accuracy.The criteria for selection of attributes in these models is based only on their significance without considering the linear relationship between the attributes involved in the model (multicollinearity) which violates the assumptions of linear regression . This thesis is devoted to developing an algorithm which builds a model for the prediction considering high accuracy and low multicollinearity for every subset formed. It is not specific to a particular domain and can be applied to any dataset. The attributes are added to the model based on the above mentioned features.The Variance Inflation factor(VIF) and the sum of the summation of Cooks statistic, square root of mean square error, absolute value of difference of fits and Press residual of each observation are taken into consideration to keep the multicollinearity low and accuracy high.