⬡ Hub
Skip to content

Scikit-learn: Machine Learning in Python

Scikit-learn is a free software machine learning library for the Python programming language. It features various classification, regression and clustering algorithms including support vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy.

Key Features:

  • Simple and Efficient: Tools for data mining and data analysis.
  • Accessible: Accessible to everybody, and reusable in various contexts.
  • Open Source: Built on NumPy, SciPy, and Matplotlib.
  • Comprehensive: Covers a wide range of machine learning tasks, including:
    • Classification: Identifying to which category an object belongs.
    • Regression: Predicting a continuous-valued attribute associated with an object.
    • Clustering: Automatic grouping of similar objects into sets.
    • Dimensionality Reduction: Reducing the number of random variables to consider.
    • Model Selection: Comparing, validating and choosing parameters and models.
    • Preprocessing: Feature extraction and normalization.

Getting Started: Installation

You can install Scikit-learn using pip or conda.

Using pip:

pip install scikit-learn

Using conda:

conda install scikit-learn

Basic Concepts: Estimators

In Scikit-learn, the fundamental building blocks are "estimators." An estimator is any object that learns from data; it can be a classification, regression, or clustering algorithm, or even a transformer that pre-processes data.

All estimators expose a fit(X, y) method for training, and a predict(X) method for making predictions (for supervised learning).

Example: Linear Regression

Let's illustrate with a simple linear regression example.

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

# 1. Generate some synthetic data
X = np.array([[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]])
y = np.array([2, 4, 5, 4, 5, 7, 8, 9, 10, 12])

# 2. Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# 3. Create a Linear Regression model (estimator)
model = LinearRegression()

# 4. Train the model using the training data
model.fit(X_train, y_train)

# 5. Make predictions on the test data
y_pred = model.predict(X_test)

# 6. Evaluate the model
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error: {mse}")
print(f"Coefficients: {model.coef_}")
print(f"Intercept: {model.intercept_}")

# Predict for a new value
new_X = np.array([[11]])
predicted_y = model.predict(new_X)
print(f"Prediction for X=11: {predicted_y[0]}")

Key Steps in a Machine Learning Workflow with Scikit-learn:

  1. Data Loading: Load your dataset (often using Pandas for data frames).
  2. Data Preprocessing: Clean, transform, and prepare your data (e.g., handling missing values, encoding categorical features, scaling numerical features). Scikit-learn provides many transformers for this.
  3. Feature Engineering: Create new features from existing ones to improve model performance.
  4. Model Selection: Choose an appropriate algorithm for your task.
  5. Training: Fit the model to your training data using the fit() method.
  6. Prediction: Use the predict() method to make predictions on new, unseen data.
  7. Evaluation: Assess the model's performance using appropriate metrics (e.g., accuracy, precision, recall, F1-score for classification; MSE, R-squared for regression).
  8. Hyperparameter Tuning: Optimize model parameters (not learned from data) using techniques like GridSearchCV or RandomizedSearchCV.

Further Topics:

  • Classification Algorithms (e.g., SVM, Decision Trees, Random Forests)
  • Clustering Algorithms (e.g., K-Means, DBSCAN)
  • Dimensionality Reduction (e.g., PCA)
  • Model Selection and Evaluation Metrics
  • Preprocessing and Feature Engineering
  • Pipelines and ColumnTransformers

This document provides a basic introduction to Scikit-learn. More detailed topics, advanced techniques, and practical examples will be covered in subsequent files.