⬡ Hub
Skip to content

beginner_house_price_prediction_decision_tree

Beginner - House Price Prediction with Decision Trees

Description

This project serves as a beginner-friendly introduction to regression tasks in machine learning. It demonstrates how to use a DecisionTreeRegressor from the scikit-learn library to predict house prices based on a set of features. The model learns to make predictions by creating a tree-like structure of decisions based on the input data.

To keep the focus on the model itself, the project uses a simple, synthetically generated dataset where house prices are determined by features like square footage and the number of bedrooms.

Functionality

  1. Data Generation: A synthetic dataset is created using numpy and pandas. This dataset simulates a realistic relationship between house features (size, bedrooms) and their prices, with some added noise.
  2. Data Splitting: The dataset is split into a training set (80%) and a testing set (20%). The model is trained on the training data and evaluated on the unseen testing data.
  3. Model Training: A DecisionTreeRegressor model is instantiated and trained using the fit() method on the training data.
  4. Prediction: The trained model is used to predict the house prices for the test set.
  5. Evaluation: The model's performance is evaluated using two common regression metrics:
    • Mean Squared Error (MSE): Measures the average squared difference between the actual and predicted values.
    • R-squared (R2) Score: Indicates the proportion of the variance in the dependent variable that is predictable from the independent variables (a score closer to 1 is better).
  6. Visualization:
    • The structure of the trained decision tree is plotted, providing insight into how the model makes its predictions.
    • The importance of each feature in the prediction process is calculated and visualized in a bar chart.

Architecture

  • scikit-learn: The primary library used for this project. It provides the DecisionTreeRegressor model, the train_test_split function, and the evaluation metrics (mean_squared_error, r2_score).
  • pandas: Used to create and manage the data in a structured DataFrame.
  • numpy: Used for the underlying numerical operations and data generation.
  • matplotlib: Used for all visualizations, including plotting the decision tree and the feature importance bar chart.

How to Run

Prerequisites

Make sure you have Python installed, along with the required libraries. You can install them using pip:

pip install scikit-learn pandas numpy matplotlib

Execution

To run the project, navigate to the project directory and execute the following command:

python beginner_house_price_prediction_decision_tree.py

The script will print the model's evaluation metrics to the console and then display two plots: one showing the decision tree structure and another showing the feature importances.

Concepts Covered

  • Regression: A type of supervised learning problem where the goal is to predict a continuous value.
  • Decision Trees: A non-linear model that can be used for both classification and regression.
  • Train-Test Split: The fundamental practice of splitting data to train a model and evaluate its performance on unseen data.
  • Model Fitting: The process of training a model on data.
  • Regression Metrics: Understanding how to evaluate a regression model with MSE and R2 score.
  • Feature Importance: A concept that describes which features have the most impact on the model's predictions.
  • Overfitting: The concept of a model learning the training data too well, which can be controlled in decision trees by parameters like max_depth.

Files and Subdirectories