Here’s a comprehensive description of the project:

Dataset

  • Train set: train_v2.csv.zip — indexed by id, contains loan features and a loss target column.
  • Test set: test_v2.csv.zip — same features, no target; used for final predictions.
  • Sample submission: A template CSV that defines the required output format.

The data is high-dimensional with many columns, and both train and test sets contain missing values across multiple features.

Pipeline & Methodology

1. Data Loading & Exploration

  • Loads both train and test datasets with pandas.
  • Inspects shape, dtypes, and missing value counts using a custom df_nulls() function that reports both the count and percentage of nulls per column.

2. Feature Selection — Numeric Only

  • Filters down to only numeric columns, dropping all categorical/object-type features entirely. This is a simplifying design choice.
  • Drops rows where the loss target is null from the training set.

3. Train/Validation Split

  • An 80/20 split using train_test_split (with random_state=42 for reproducibility).

4. Preprocessing Pipeline

  • Uses a scikit-learn Pipeline with two steps:
    • SimpleImputer (mean strategy) — fills missing values.
    • StandardScaler — normalizes numeric features.
  • Wrapped in a ColumnTransformer for clean, leak-free application.

5. Feature Importance via Permutation Importance

  • Trains a Logistic Regression model (used as a helper model, not the final one).
  • Uses eli5’s PermutationImportance on a 1,000-sample validation subset to rank features by their actual impact on predictions.
  • Applies SelectFromModel with a threshold of 0.001 to filter down to only the most predictive features.

6. Model Training — Random Forest Classifier

  • The final model is a RandomForestClassifier with:
    • n_estimators=10 (10 trees — relatively small)
    • criterion='entropy'
  • Evaluated using 5-fold cross-validation, with mean accuracy reported.
  • After validation, the model is retrained on the full training set for improved generalization before generating test predictions.

7. Submission

  • Predictions are written into the submission.csv file in the format required by the Kaggle competition.

Key Design Choices & Observations

Aspect Detail
Target variable loss — a binary/multiclass classification target
Feature engineering None; raw numeric features only
Imputation Mean imputation (could miss non-MAR patterns)
Feature selection Permutation importance via a proxy Logistic Regression
Final model Random Forest (small forest with 10 estimators)
Evaluation metric Cross-validation accuracy
Categorical features Dropped entirely

Kaggle

Updated: