jsonl-archive-splitter 0.1.0

CLI tool to split JSONL archive files by Jalali boundary date.

JSONL Archive Splitter

A production-ready Python CLI tool for splitting large JSONL archive files based on a Jalali (Persian) boundary date and configurable maximum file size.

Designed for memory efficiency, clean architecture, and extensibility.

---

✨ Features

• Split JSONL files based on publish_ts
• Supports Jalali (Shamsi) boundary dates
• Automatic file rotation by maximum size
• Line-by-line processing (memory efficient)
• Modular architecture (CLI / Core / Utils separation)
• Logging-based execution (no noisy prints)
• Ready for packaging and future PyPI publishing

---

📁 Project Structure

jsonl-archive-splitter/
│
├── src/
│   └── jsonl_archive_splitter/
│       ├── __init__.py
│       ├── __main__.py
│       ├── cli.py
│       ├── splitter.py
│       └── utils.py
│
├── requirements.txt
├── README.md
└── .gitignore

---

📦 Installation

Clone the repository:

git clone https://github.com/yourusername/jsonl-archive-splitter.git
cd jsonl-archive-splitter

Install dependencies:

pip install -r requirements.txt

requirements.txt:

jdatetime

---

🚀 Usage

Run from the project root directory:

python -m src.jsonl_archive_splitter   --src-dir input   --before-dir out_before   --after-dir out_after   --boundary-date 1404-07-30   --start 1   --end 10   --max-size 400

---

🧾 CLI Arguments

Argument Description

---

--src-dir Directory containing input JSONL files --before-dir Output directory for messages up to boundary date --after-dir Output directory for messages after boundary date --boundary-date Jalali date in format YYYY-MM-DD --start Start file index --end End file index --max-size Max size per output file in MB (default: 400)

---

🧠 How It Works

1. Converts Jalali boundary date to Gregorian.
2. Computes timestamp for 23:59:59 of boundary date.
3. Iterates over JSONL files sequentially.
4. Routes messages based on publish_ts.
5. Rotates output files automatically when size limit is reached.

---

⚙️ Technical Design

• No global state
• Clear separation of concerns
• Logging instead of print statements
• Package-ready structure
• Easily extensible (e.g., Gregorian support, dry-run mode, progress bar)

---

🔮 Possible Future Improvements

• Add unit tests
• Add tqdm progress bar
• Add dry-run mode
• Add PyPI packaging (pyproject.toml)
• Add GitHub Actions CI

---

📄 License

MIT License

---

👤 Author

Saeed -- Software Engineer
Focused on clean architecture, scalable systems, and production-quality tooling.