c5tree 🌳

C5.0 Decision Tree Classifier for Python — a scikit-learn-compatible implementation of Ross Quinlan's C5.0 algorithm, with an optional native C++ numeric splitter.

Why C5.0?

scikit-learn's DecisionTreeClassifier uses CART (binary splits, Gini/entropy). C5.0 offers several advantages:

Installation

pip install c5tree

Quick Start

from c5tree import C5Classifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

clf = C5Classifier(pruning=True, cf=0.25)
clf.fit(X_train, y_train)

print(f"Accuracy: {clf.score(X_test, y_test):.3f}")
print(f"Tree depth: {clf.get_depth()}")
print(f"Leaves: {clf.get_n_leaves()}")

Key Features

Gain Ratio Splitting

Corrects ID3's bias toward features with many distinct values by normalising information gain by split information.

Optional Native Acceleration

Numeric split evaluation uses an optional C++ extension through pybind11 when a compiler is available. Standard installs remain supported without a compiler and automatically use the equivalent Python implementation. This keeps C5.0 gain-ratio behavior while accelerating its hottest numeric operation; it is not a LightGBM or XGBoost model.

Native Missing Value Handling

No imputation needed. Missing instances are distributed fractionally across branches, weighted by the proportion of known instances going each way.

import numpy as np

X_with_missing = X.copy().astype(float)
X_with_missing[0, 2] = np.nan   # inject a missing value

clf.fit(X_with_missing, y)      # works out of the box
clf.predict(X_with_missing)     # also works

Categorical Feature Support

Pass a pandas DataFrame and c5tree automatically detects object/category columns and applies multi-way splits:

import pandas as pd

df = pd.DataFrame({
    "outlook":  ["sunny", "overcast", "rainy", "sunny", "rainy"],
    "humidity": [85, 65, 70, 95, 80],
    "play":     [0, 1, 1, 0, 1],
})
X = df[["outlook", "humidity"]]
y = df["play"]

clf = C5Classifier(pruning=False).fit(X, y)

Pessimistic Error Pruning

After the tree is grown, subtrees are replaced by leaves if the pessimistic error estimate of the subtree is no better than a single leaf. Controlled by the cf parameter.

# cf=0.25  → default, moderate pruning
# cf=0.05  → aggressive pruning, very small tree
# cf=0.50  → light pruning, larger tree
clf = C5Classifier(pruning=True, cf=0.05)

Human-Readable Tree

print(clf.text_report())
# [feature_2 <= 1.9000]
#   left:
#     → Predict: 0  [0: 1.00, 1: 0.00, 2: 0.00]  (n=40.0)
#   right:
#     [feature_3 <= 1.7500]
#       ...

Tested sklearn Compatibility

The tested estimator workflows include Pipelines, GridSearchCV, cross_val_score, and clone:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import GridSearchCV

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("clf", C5Classifier()),
])

param_grid = {"clf__cf": [0.05, 0.25, 0.50], "clf__max_depth": [None, 5, 10]}
search = GridSearchCV(pipe, param_grid, cv=5)
search.fit(X_train, y_train)
print(search.best_params_)

Parameters

Running Tests

python -m pip install -e ".[dev]"
pytest tests/ -v --cov=c5tree

Building Packages

python -m pip install build
python -m build

The wheel includes the native C++ splitter when it can be compiled. Installing from source without a supported compiler still succeeds and uses the Python fallback. Binary wheels can be published for each supported operating system and Python version, as with LightGBM and XGBoost, so most users do not need a compiler.

Background

C5.0 is the successor to C4.5 and ID3, developed by Ross Quinlan. The algorithm was open-sourced in 2011 under the GPL licence. This package is a clean Python reimplementation that integrates C5.0-style splitting with the scikit-learn estimator API.

Reference: Quinlan, J.R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.

Contributing

Contributions are very welcome! Please open an issue before submitting a PR.

Licence

GPL-3.0-or-later License — see LICENSE.