⬡ Hub
Skip to content

Core ML: Classification

Classification algorithms are supervised learning models used to predict categorical class labels. Examples include predicting if an email is "Spam" or "Not Spam", diagnosing a tumor as "Malignant" or "Benign", or identifying handwritten digits (0-9).

1. Logistic Regression

Despite having "regression" in its name, this is purely a classification algorithm. - How it works: It uses the same math as Linear Regression but wraps the output in a Sigmoid Function $S(x) = \frac{1}{1 + e^{-x}}$. This squashes any mathematical output into a smooth probability curve strictly between 0 and 1. - Decision Boundary: Usually, if the output probability is $> 0.5$, the model predicts Class 1. Otherwise, Class 0.

2. Decision Trees & Ensembles

  • Decision Trees: Split the dataset based on feature questions (e.g., "Is Age > 30?"). They are highly interpretable (you can print the "if-else" flowchart) but have a massive tendency to overfit (memorize noise in the training data).
  • Random Forests (Bagging): The solution to decision tree overfitting. It trains 500 shallow individual decision trees on random subsets of the data (and random subsets of the features), and lets them "vote" on the final classification.
  • Gradient Boosting / XGBoost (Boosting): The reigning champion of tabular data classification. Instead of training 500 independent trees, it trains them sequentially. Tree #2 is specifically trained to correct the errors made by Tree #1.

3. Evaluation Metrics

In classification, "Accuracy" is notoriously dangerous. If a dataset has 99 Legitimate Transactions and 1 Fraudulent Transaction, a model that simply hardcodes "Predict Legitimate" every single time will achieve 99% Accuracy—but it is a useless model.

  • Precision: Of all the transactions my model flagged as Fraud, how many were actually Fraud? (High Precision means few false alarms).
  • Recall (Sensitivity): Of all the actual Fraud in the real world, how many did my model catch? (High Recall means you miss almost nothing, but you might have false alarms).
  • F1-Score: The harmonic mean of Precision and Recall. Used when you need a balanced metric.
  • ROC-AUC: Measures the model's ability to distinguish between classes across all possible probability thresholds.

How to execute the examples:

Go to the Examples/ folder and run the scripts: python Class_LogisticRegression.py python Class_RandomForest.py python Class_XGBoost_Optuna.py