dolphinboost 0.2.0

A lightweight machine learning library with educational implementations built from scratch using NumPy.

DolphinBoost 🐬

A lightweight Machine Learning library built from scratch using NumPy.

---

Installation

pip install dolphinboost

---

Features

• ✅ Simple Linear Regression
• ✅ Multiple Linear Regression (5 solvers!)
• ✅ Pure NumPy — no dependencies except NumPy
• ✅ sklearn-compatible API

---

Quick Start

import numpy as np
from dolphinboost import SimpleLinearRegression
from dolphinboost import MultipleLinearRegression

# Simple Linear Regression
X = np.array([1, 2, 3, 4, 5]).reshape(-1, 1)
y = np.array([2, 4, 6, 8, 10], dtype=float)

model = SimpleLinearRegression()
model.fit(X, y)
print(model.predict(np.array([[6]])))  # [12.]

# Multiple Linear Regression
X = np.random.randn(100, 3)
y = X @ np.array([1.0, 2.0, 3.0])

model = MultipleLinearRegression()
model.fit(X, y)
print(model.coef_.round(2))  # [1. 2. 3.]

---

Simple Linear Regression

Usage

from dolphinboost import SimpleLinearRegression
import numpy as np

X = np.array([1, 2, 3, 4, 5]).reshape(-1, 1)
y = np.array([2, 4, 6, 8, 10], dtype=float)

model = SimpleLinearRegression()
model.fit(X, y)

print(model.coef_)        # 2.0
print(model.intercept_)   # 0.0
print(model.beta_)        # [0. 2.]

X_new = np.array([[6], [7]])
print(model.predict(X_new))  # [12. 14.]

Parameters

Parameter	Type	Default	Description
fit_intercept	bool	True	Whether to compute intercept

Attributes

Attribute	Type	Description
coef_	float	Slope β₁
intercept_	float	Intercept β₀
beta_	ndarray	[β₀, β₁]
X_mean_	float	Mean of training X
y_mean_	float	Mean of training y

Methods

Method	Description
fit(X, y)	Fit model to training data
predict(X)	Predict target values

Input Requirements

# X must be 2D with exactly 1 feature
X = np.array([[1], [2], [3]])       # ✅ correct
X = np.array([1, 2, 3])             # ❌ use X.reshape(-1, 1)
X = np.array([[1, 2], [3, 4]])      # ❌ use MultipleLinearRegression

# y must be 1D
y = np.array([1, 2, 3])             # ✅ correct
y = np.array([[1], [2], [3]])       # ❌ must be 1D

fit_intercept=False

model = SimpleLinearRegression(fit_intercept=False)
model.fit(X, y)
print(model.intercept_)  # 0

Errors

# 1. Wrong X dimensions
model.fit(np.array([1, 2, 3]), y)
# ValueError: X must be 2-dimensional, got 1D.
#             Shape: (3,). Use X.reshape(-1, 1) if needed.

# 2. Multiple features
model.fit(np.array([[1, 2], [3, 4]]), y)
# ValueError: SimpleLinearRegression only supports 1 feature,
#             but got 2 features.
#             Use MultipleLinearRegression for multiple features.

# 3. Empty X
model.fit(np.array([]).reshape(0, 1), y)
# ValueError: X has 0 samples

# 4. Predict before fit
m = SimpleLinearRegression()
m.predict(X)
# ValueError: Model not fitted yet! Call fit() first.

# 5. All X values identical
model.fit(np.array([[1], [1], [1]]), np.array([1, 2, 3]))
# ValueError: Cannot fit when all X values are identical

# 6. NaN or infinite values
model.fit(X_with_nan, y)
# ValueError: X contains NaN or infinite values

---

Multiple Linear Regression

Usage

from dolphinboost import MultipleLinearRegression
import numpy as np

np.random.seed(42)
X = np.random.randn(100, 3)
y = X @ np.array([1.0, 2.0, 3.0]) + np.random.randn(100) * 0.1

model = MultipleLinearRegression(solver='lstsq')
model.fit(X, y)

print(model.coef_.round(2))   # [1. 2. 3.]
print(model.intercept_)        # ~0.0
print(model.n_features_in_)    # 3

X_new = np.random.randn(5, 3)
print(model.predict(X_new))

Parameters

Parameter	Type	Default	Description
fit_intercept	bool	True	Whether to compute intercept
solver	str	'lstsq'	Solver to use

Attributes

Attribute	Type	Description
coef_	ndarray	Feature coefficients [β₁, β₂, ...]
intercept_	float	Intercept β₀
beta_	ndarray	Full array [β₀, β₁, β₂, ...]
n_features_in_	int	Number of features seen during fit

Methods

Method	Description
fit(X, y)	Fit model to training data
predict(X)	Predict target values

Solvers

Solver	Speed	Stability	Best For
normal	⚡⚡⚡ Fastest	❌ Least stable	Small, clean data
cholesky	⚡⚡ Fast	⚠️ Moderate	Medium, clean data
qr	⚡ Medium	✅ Good	General purpose
svd	🐢 Slower	✅✅ Very stable	Multicollinear data
lstsq	🐢 Slower	✅✅ Best	Default — always safe!

# Choose based on your data
model = MultipleLinearRegression(solver='normal')    # fastest
model = MultipleLinearRegression(solver='cholesky')  # fast + stable
model = MultipleLinearRegression(solver='qr')        # general
model = MultipleLinearRegression(solver='svd')       # multicollinear
model = MultipleLinearRegression(solver='lstsq')     # default ✅

fit_intercept=False

model = MultipleLinearRegression(fit_intercept=False)
model.fit(X, y)
print(model.intercept_)  # 0

When solvers fail

# Perfect multicollinearity — col2 = 2 × col1
X = np.array([[1, 2], [2, 4], [3, 6], [4, 8]])
y = np.array([1, 2, 3, 4])

MultipleLinearRegression(solver='normal').fit(X, y)
# ValueError: Normal equation failed because XᵀX is
#             singular or ill-conditioned.
#             Try solver='lstsq' or solver='svd'.

MultipleLinearRegression(solver='cholesky').fit(X, y)
# ValueError: Cholesky decomposition failed because XᵀX
#             is not positive definite.
#             Try solver='lstsq', 'svd', or 'qr'.

MultipleLinearRegression(solver='qr').fit(X, y)
# ValueError: QR solver failed because the matrix
#             is rank deficient.
#             Try solver='lstsq' or solver='svd'.

MultipleLinearRegression(solver='svd').fit(X, y)    # ✅ works!
MultipleLinearRegression(solver='lstsq').fit(X, y)  # ✅ works!

Errors

# 1. Wrong X dimensions
model.fit(np.array([1, 2, 3]), y)
# ValueError: X must be 2-dimensional, got 1D.

# 2. Empty X
model.fit(np.array([]).reshape(0, 3), y)
# ValueError: X has 0 samples.

# 3. Length mismatch
model.fit(X_100, y_50)
# ValueError: X and y must have same length

# 4. NaN or infinite values
model.fit(X_with_nan, y)
# ValueError: X contains Nan or Infinite values

# 5. Unknown solver
MultipleLinearRegression(solver='unknown').fit(X, y)
# ValueError: Unknown solver 'unknown'.
#             Choose from ['normal', 'cholesky', 'qr', 'svd', 'lstsq']

# 6. Predict before fit
m = MultipleLinearRegression()
m.predict(X)
# ValueError: Model not fitted yet! Call fit() first.

# 7. Feature mismatch in predict
model.fit(X_3features, y)
model.predict(X_4features)
# ValueError: Expected 3 features, got 4.

---

Comparison with sklearn

import numpy as np
from sklearn.linear_model import LinearRegression
from dolphinboost import MultipleLinearRegression

np.random.seed(42)
X = np.random.randn(1000, 5)
y = X @ np.array([1.0, 2.0, 3.0, 4.0, 5.0]) + np.random.randn(1000) * 0.1

# DolphinBoost
db = MultipleLinearRegression()
db.fit(X, y)
print("DolphinBoost:", db.coef_.round(2))
# [1. 2. 3. 4. 5.]  ✅

# Sklearn
sk = LinearRegression()
sk.fit(X, y)
print("Sklearn:     ", sk.coef_.round(2))
# [1. 2. 3. 4. 5.]  ✅

---

Mathematical Background

Simple Linear Regression

Model:

$$y = \beta_0 + \beta_1 x$$

where

$$\beta_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}$$

$$\beta_0 = \bar{y} - \beta_1 \bar{x}$$

Multiple Linear Regression

Model:

$$y = X\beta$$

where $X$ is the design matrix, $y$ is the target vector, $\beta$ is the coefficient vector.

Normal Equation

$$\beta = (X^TX)^{-1}X^Ty$$

Cholesky Decomposition

$$X^TX = LL^T$$

Solve $Lz = X^Ty$, then $L^T\beta = z$

QR Decomposition

$$X = QR$$

Solve $R\beta = Q^Ty$

Singular Value Decomposition (SVD)

$$X = U\Sigma V^T$$

$$\beta = V\Sigma^{-1}U^Ty$$

Least Squares

Find $\beta$ that minimizes:

$$|X\beta - y|^2$$

---

Requirements

python >= 3.8
numpy >= 1.20.0

---

Links

• 📦 PyPI: https://pypi.org/project/dolphinboost

---

License

MIT License

---

Author

Peddakotla Karthikeya Built with ❤️ from scratch using NumPy!

---

Contributing

Found a bug or have an idea? Open an issue on PyPI or contact via PyPI page! 🐬