Health Dataset Classifiers — Naive Bayes + Decision Tree (Pruning)

This project implements and compares multiple classifiers on the Pima Indians Diabetes dataset, with a focus on:

• building models from scratch
• stratified k-fold cross validation for reliable evaluation
• improving a baseline decision tree using pruning / optimization to reduce overfitting

Models Included

• Naive Bayes (Gaussian NB for numeric features; see A2_naive_bayes/)
• Decision Tree (baseline) (see decision_tree/)
• Decision Tree (improved / pruned) (see decision_tree_star/)
  • adds pruning / branch reduction to improve generalization and reduce unnecessary splits

Key Concepts Demonstrated

• Probabilistic classification (Gaussian PDF)
• Tree learning + splitting criteria (as implemented in code)
• Pruning to control model complexity and reduce overfitting
• Stratified cross validation to preserve class distribution per fold
• Evaluation utilities (accuracy + prediction)

---

Repository Structure

A2_FILES/
  A2_naive_bayes/
    program.py
    pima.csv
  
  decision_tree/
    program.py

  decision_tree_star/
    program.py   # pruned/improved tree
  
  report/
  ML_Report.pdf

  stratified-folds/        # stratified CV utilities
  accuracy_predict/
    accuracy.py

  *.csv                    # processed train/test splits
  dt.py                    # extra helper / experiments
  readme.md

How to Run

1. Setup (recommended)

Use a virtual environment and install requirements if you have them:

python3 -m venv .venv source .venv/bin/activate pip install -U pip

If you don’t have external dependencies (pure Python), you can skip installs.

2. Naive Bayes cd A2_naive_bayes python3 program.py

Dataset: pima.csv (or update the script to point to your desired CSV)

3. Decision Tree (baseline) cd ../decision_tree python3 program.py

4. Decision Tree (improved with gain ration & early stopping / pruned) cd ../decision_tree_star python3 program.py

5. Accuracy / evaluation helper cd ../accuracy_predict python3 accuracy.py

If your scripts require arguments (e.g., train/test file paths, k folds, etc.), add them to the commands above. A good improvement is to print a --help usage message.

Evaluation This project uses stratified k-fold cross validation to ensure each fold maintains similar class proportions. This helps produce more reliable performance estimates than a single train/test split, especially if the dataset is imbalanced.

Artifacts:

CrossValidation.png — diagram / notes on CV

stratified-folds/ — fold generation or fold data (if applicable)

Decision Tree Star (Pruned / Improved Tree)

decision_tree_star/ contains an improved decision tree implementation that adds pruning / branch reduction.

Why pruning matters:

• reduces overfitting by limiting unnecessary splits
• improves generalization on unseen data
• can reduce tree size and improve interpretability
• (Implementation details are documented in code comments and/or report.)