💻Try Linear Regression Modeling in Python
Since we’ve covered basic concepts like mean, median, correlation and linear regression, it is time we practiced using Python. Please refer to the pictures in this post for detailed coding and data display.
🖋️Get data set from Kaggle
I downloaded the Titanic data set from the famous data science website Kaggle. I used the file train.csv for modeling purpose.
There is explanation about the data set in the website so that we get a better understanding about our task and what we have at our disposal.
🖋️Import the packages
Numpy, pandas, sklearn, seaborn and matplotlib are used.
🖇️Numpy and pandas: data cleaning and processing
🖇️Sklearn: linear regression model building
🖇️Seaborn and matplotlib: data visualization
🖋️Data cleaning before modeling
I used pd.read_csv() to load the Titanic data set into my project and named the data tr. To see what my data looked like, tr.head() showed the very first 5 entries. Columns like PassengerId, Survived and Pclass are called variables, or features. In the Titanic case, the dependent variable is Survived and features like Pclass, Fare or Cabin are potential candidates for independent variables.
Before we move forward, we need to check if the data set has missing values. For this purpose I used tr.info() to get a summary of my data set. Age, Cabin and Embarked have missing values while other variables don’t. Since there are almost 80% missing values in Cabin, we will not use this variable in our model. Then we need to simulate some value to fill the missing ones in Age and Embarked.
Age is clearly a numeric variable and the most common practice to fill such missing values is to use its mean or median. This time I chose median.
Embarked is a categorical variable and to use mode for missing value filling is a good choice.
Plus, linear regression does not cope with categorical variables. Dummy variables shall be constructed to convert categorical variables into what linear regression can use. Hence we need to do a bit more work for variables like Sex and Embarked.
🖋️Which variables are more likely to be used in a model?
I checked the correlations between each pair of variables so that I can choose those that are more correlated with the dependent variable Survived. A heat map is a good visualization tool.
🖋️Modeling time
After I figured out which variables might be useful in my model, I used LinearRefression() in the sklearn package to build my linear regression model to predict the chance of survival of each passenger.
I then used a threshold of 0.5 for classification. If the predicted chance of survival is greater than 0.5 then the passenger survives otherwise they don’t. You may also use other threshold like 0.65 or 0.4 for your predicted results, as long as it makes your predction more accurate.
Kaggle: https://www.kaggle.com/competitions/titanic/data
Linear regression is a widely used statistical technique that helps predict outcomes based on linear relationships between variables. In this tutorial, we focus on applying linear regression using Python, particularly with the popular Titanic dataset from Kaggle. The Titanic dataset consists of historical passenger data, which includes crucial features such as PassengerId, Survived, Pclass, Name, Sex, Age, and Fare. This data allows data scientists and analysts to uncover insights related to survival rates based on various factors. The first step in this modeling process is data preparation, which includes handling missing values. The mean or median is typically used to replace missing numerical entries, while categorical variables, like 'Embarked' or 'Sex,' can be recoded into numerical format to be utilized in linear regression. To visualize relationships among variables, correlation matrices and heatmaps are essential tools that help identify which features are most relevant to the dependent variable, 'Survived.' By assessing these correlations, you can efficiently select which features to include in your model. Once the data is prepared and analyzed, the LinearRegression model from the scikit-learn library can be employed to create predictions. The predicted survival probabilities can guide further analysis, allowing you to apply varying thresholds based on your specific classification criteria. For instance, adjusting the threshold for classifying predictions can enhance the model's accuracy based on the business problem at hand. This Python implementation of linear regression not only equips you with essential programming skills but also deepens your understanding of data science concepts. By engaging with hands-on projects like modeling the Titanic survival data, you're set on a path to mastering data-driven decision-making.