🛡️ Phishing URL Detector

> 🚨 A Machine Learning based system for detecting whether a URL is **Legitimate** or **Phishing**.

📌 Table of Contents

• About the Project
• Problem Statement
• Project Objective
• How It Works
• Machine Learning Pipeline
• Dataset
• Data Preprocessing
• Feature Engineering
• Features Used
• Target Encoding
• Train-Test Split
• Machine Learning Model
• Prediction
• Current Performance
• Project Structure
• Python Package Architecture
• Installation
• Usage
• Roadmap
• Limitations
• Security Disclaimer
• Learning Outcomes

---

🛡️ About the Project

Phishing URL Detector is a Machine Learning project designed to classify URLs into two categories:

• ✅ Legitimate
• 🚨 Phishing

The project analyzes the structural and lexical characteristics of URLs rather than directly opening or visiting websites.

The current version uses manually extracted URL features and a Logistic Regression classifier.

The project is also being developed with a reusable Python package structure so that feature extraction functionality can be imported into other Python applications.

---

🎯 Problem Statement

Phishing attacks commonly use malicious URLs to trick users into visiting fraudulent websites.

Attackers may create URLs that:

• contain unusually long paths
• use many special characters
• contain multiple subdomains
• imitate legitimate websites
• contain suspicious URL structures
• redirect users toward malicious destinations

The objective of this project is to investigate whether URL-level characteristics can be used by a Machine Learning model to distinguish between legitimate and phishing URLs.

---

🚀 Project Objective

The main objectives are:

1. Load a real-world URL dataset.
2. Clean and preprocess the dataset.
3. Convert categorical labels into numerical values.
4. Extract meaningful numerical features from raw URLs.
5. Split the dataset into training and testing subsets.
6. Train a Machine Learning classification model.
7. Generate predictions for unseen URLs.
8. Evaluate the model.
9. Improve feature engineering.
10. Develop reusable Python package components.

---

🔄 How It Works

                RAW DATASET
                     │
                     ▼
              Data Loading
                     │
                     ▼
            Data Preprocessing
                     │
                     ▼
            Feature Extraction
                     │
                     ▼
          Numerical Feature Matrix
                     │
                     ▼
              Train / Test Split
                     │
                     ▼
           Logistic Regression
                     │
              ┌──────┴──────┐
              ▼             ▼
          Training       Testing
              │             │
              └──────┬──────┘
                     ▼
                 Prediction
                     │
                     ▼
          Legitimate / Phishing

---

🧠 Machine Learning Pipeline

1. Data Loading

The dataset is loaded using Pandas:

df = pd.read_csv(r"data\raw_data.csv")

2. Data Cleaning

Only the expected binary classes are retained:

df = df[df["label"].isin([
    "legitimate",
    "phishing"
])].copy()

The current dataset contains one anomalous P label, which is excluded rather than assigning it an unverified meaning.

3. Feature Extraction

Raw URLs are converted into numerical features:

Raw URL
   ↓
Feature Extraction
   ↓
[URL Length, Dots, Hyphens, @, Slashes]

4. Target Encoding

legitimate → 0
phishing   → 1

5. Train-Test Split

80% → Training
20% → Testing

6. Model Training

The current baseline model is Logistic Regression.

7. Prediction

y_pred = model.predict(X_test)

---

📊 Dataset

The current dataset contains:

253,098 original rows
2 columns

Columns:

Column	Description
url	Raw URL
label	Classification label

Original Label Distribution

Label	Count
legitimate	129,420
phishing	123,677
P	1

After filtering:

253,097 usable samples

The two primary classes are approximately balanced.

---

🧹 Data Preprocessing

The dataset is filtered to retain only legitimate and phishing samples:

df = df[df["label"].isin([
    "legitimate",
    "phishing"
])].copy()

This prepares the dataset for binary classification.

---

⚙️ Feature Engineering

Machine Learning models cannot directly use raw URL strings in the current implementation.

For example:

https://example.com/login/account

is transformed into numerical information:

URL
 │
 ├── Length
 ├── Number of dots
 ├── Number of hyphens
 ├── Number of @ symbols
 └── Number of slashes

These values form the feature matrix X.

---

🔢 Features Used

The current implementation extracts 5 URL features.

1. URL Length

data["url_length"] = df["url"].apply(len)

Measures the total number of characters in the URL.

2. Number of Dots

data["num_dots"] = df["url"].apply(
    lambda x: x.count(".")
)

Counts . characters.

3. Number of Hyphens

data["num_hyphens"] = df["url"].apply(
    lambda x: x.count("-")
)

Counts - characters.

4. Number of @ Symbols

data["num_at"] = df["url"].apply(
    lambda x: x.count("@")
)

Counts @ symbols.

5. Number of Slashes

data["num_slashes"] = df["url"].apply(
    lambda x: x.count("/")
)

Counts / characters.

---

🧮 Feature Matrix

Each URL becomes a numerical feature vector:

url_length | num_dots | num_hyphens | num_at | num_slashes

Current dimensions:

253,097 rows × 5 features

---

🎯 Target Encoding

The labels are converted from text to numbers:

y = df["label"].map({
    "legitimate": 0,
    "phishing": 1
})

Therefore:

0 → Legitimate
1 → Phishing

---

✂️ Train-Test Split

The project uses train_test_split():

X_train, X_test, y_train, y_test = tt(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y
)

Parameter	Value
Training data	80%
Testing data	20%
Random state	42
Stratification	Enabled

Current shapes:

Training:
(202477, 5)

Testing:
(50620, 5)

---

🤖 Machine Learning Model

Logistic Regression

The first baseline classifier is Logistic Regression:

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X_train, y_train)

Logistic Regression is a useful starting point because it is:

• simple
• fast
• interpretable
• suitable for binary classification
• useful for establishing a baseline

The project starts with a simple model before moving to more complex algorithms.

---

🔮 Prediction

After training:

y_pred = model.predict(X_test)

Example:

Actual:    1 1 1 0 0
Predicted: 1 1 0 0 0

Where:

0 → Legitimate
1 → Phishing

---

📈 Current Performance

Current baseline accuracy:

59.68%

Approximately:

Accuracy = 0.5968

This is a baseline result, not the final expected performance.

The current model uses only five basic structural URL features, so improving feature engineering is the next major step.

Note: Advanced evaluation techniques such as the confusion matrix will be introduced and integrated after understanding the underlying concepts.

---

📁 Project Structure

phising-website-predictor/
│
├── .env/
├── .vscode/
│
├── data/
│   ├── data_load.py
│   ├── feature_engineering.py
│   └── raw_data.csv
│
├── src/
│   └── phishing_url_dector/
│       ├── __init__.py
│       ├── detector.py
│       ├── features.py
│       └── banner.py
│
├── tests/
│
├── train.py
├── LICENSE
├── README.md
├── pyproject.toml
└── requirements.txt

---

📦 Python Package Architecture

The project is structured as a reusable Python package:

phishing_url_dector/
│
├── __init__.py
├── features.py
├── detector.py
└── banner.py

features.py

Contains reusable feature extraction logic:

extract_url_features()

detector.py

Reserved for URL detection and model prediction functionality.

banner.py

Contains the terminal ASCII banner.

__init__.py

Exposes package functionality, for example:

from .features import extract_url_features

This allows package-level imports.

---

🖥️ Terminal Banner

The project includes a custom ASCII banner for the command-line interface.

Example:

from phishing_url_dector.banner import BANNER

print(BANNER)

---

🛠️ Technology Stack

Programming

• 🐍 Python

Data Processing

• 🐼 Pandas
• 🔢 NumPy

Machine Learning

• 🤖 Scikit-learn

Package Development

• pyproject.toml
• pip
• Python package structure

Development

• Visual Studio Code
• Python virtual environment

---

📥 Installation

Clone the repository:

git clone <repository-url>

Enter the project directory:

cd phising-website-predictor

Create a virtual environment:

python -m venv .env

Activate it on Windows:

.env\Scripts\activate

Install dependencies:

pip install -r requirements.txt

Install the package in editable mode:

pip install -e .

---

▶️ Usage

Run the training program:

python train.py

The program currently performs:

1. Display banner
2. Load dataset
3. Filter invalid labels
4. Extract URL features
5. Encode target labels
6. Split data
7. Train Logistic Regression
8. Generate predictions

---

🚧 Development Status

[✓] Dataset loading
[✓] Dataset cleaning
[✓] Label encoding
[✓] Feature extraction
[✓] Train-test split
[✓] Logistic Regression
[✓] Model prediction
[✓] Python package structure
[✓] Editable package installation
[✓] CLI banner

[ ] Advanced feature engineering
[ ] Additional ML models
[ ] Model comparison
[ ] Advanced evaluation
[ ] Model serialization
[ ] Prediction API
[ ] Web interface

---

🚀 Roadmap

Phase 1 — Baseline

• Load dataset
• Clean dataset
• Encode labels
• Extract basic URL features
• Train Logistic Regression
• Generate predictions

Phase 2 — Feature Engineering

Planned features:

HTTPS detection
Domain length
Hostname length
Path length
Query length
Subdomain count
Digit count
Special-character count
Double-slash detection
IP-address detection
Suspicious keywords
URL entropy
Domain-related features

Phase 3 — Model Comparison

Planned models:

Logistic Regression
        ↓
K-Nearest Neighbors
        ↓
Decision Tree
        ↓
Random Forest
        ↓
Additional classification algorithms

Phase 4 — Model Persistence

Save the trained model using a serialization solution such as joblib, allowing the model to be reused without retraining.

Phase 5 — Prediction Interface

User enters URL
       ↓
Feature Extraction
       ↓
Trained Model
       ↓
Prediction
       ↓
Legitimate / Phishing

Phase 6 — API

A future REST API may use:

Frontend
   │
   ▼
FastAPI
   │
   ▼
Feature Extraction
   │
   ▼
ML Model
   │
   ▼
Prediction

---

🔬 Future Feature Engineering

Future versions may analyze additional URL characteristics.

URL Structure

URL length
Hostname length
Path length
Query length
Fragment length

Character Analysis

Digits
Dots
Hyphens
Underscores
@ symbols
Percent encoding
Special characters

Domain Analysis

IP address
Subdomain count
Domain length
Suspicious TLD

Security Signals

HTTPS usage
Suspicious keywords
Redirect patterns
Encoded characters

These additional signals may provide more discriminative information to the classifier.

---

⚠️ Limitations

The current version has several limitations:

Limited Feature Set

Only five basic URL features are currently used.

No Website Content Analysis

The current system does not inspect:

• HTML
• JavaScript
• screenshots
• DNS information
• certificates
• website content

The current implementation focuses on URL-level characteristics.

Baseline Model

Only Logistic Regression is currently being used as the initial classifier.

Baseline Accuracy

The current accuracy of approximately 59.68% shows that the present feature set is not yet sufficient for a strong production-grade phishing detector.

---

🔐 Security Disclaimer

This project is intended for:

• educational purposes
• Machine Learning experimentation
• cybersecurity research
• URL classification research

It should not currently be treated as a production-grade security solution.

A URL classified as legitimate should not automatically be considered safe.

Likewise, a URL classified as phishing should be investigated using appropriate security tools and procedures.

---

🎓 Learning Outcomes

This project is being developed as a practical Machine Learning learning exercise.

The development process covers:

Python
   ↓
Pandas
   ↓
Data Cleaning
   ↓
Feature Engineering
   ↓
Numerical Data
   ↓
Train-Test Split
   ↓
Binary Classification
   ↓
Logistic Regression
   ↓
Prediction
   ↓
Model Evaluation
   ↓
Python Packaging

The project combines:

Machine Learning + Python Development + Software Engineering

rather than focusing only on model training.

---

📌 Current Baseline Summary

Component	Current Status
Dataset	253,098 original rows
Usable samples	253,097
Classes	2
Features	5
Training samples	202,477
Testing samples	50,620
Model	Logistic Regression
Accuracy	~59.68%
Package	In development
API	Planned
Web UI	Planned

---

🌱 Development Philosophy

The project follows an incremental Machine Learning workflow:

Start Simple
     ↓
Understand the Data
     ↓
Build Baseline
     ↓
Measure Performance
     ↓
Improve Features
     ↓
Compare Models
     ↓
Optimize
     ↓
Deploy

The goal is to understand each stage before increasing system complexity.

---

⭐ Future Vision

The long-term goal is to evolve this project from a basic Machine Learning experiment into a reusable phishing URL detection system.

                 USER
                  │
                  ▼
             URL INPUT
                  │
                  ▼
        ┌──────────────────┐
        │ Feature Extraction│
        └────────┬─────────┘
                 │
                 ▼
          Trained ML Model
                 │
                 ▼
        ┌──────────────────┐
        │   Classification │
        └────────┬─────────┘
                 │
          ┌──────┴──────┐
          ▼             ▼
     LEGITIMATE       PHISHING

---

Developed By Founder Of CodeUdaan

Devidutta Das

⚡ Project Status

🚧 Active Development

The current version represents the baseline stage of the project.

Next Major Focus

Improve feature engineering and understand why the current model achieves only ~59.68% accuracy.

---

📜 License

This project is distributed under the license included in this repository.

---

⭐ If you find this project useful

Consider giving the repository a ⭐ and following the development journey.