chatpack-py 🚀

High-performance Python bindings for chatpack - parse chat exports from Telegram, WhatsApp, Instagram, and Discord with Rust-powered speed.

Features

• ⚡ Blazing Fast: Rust implementation for maximum performance
• 🔄 Multiple Platforms: Telegram, WhatsApp, Instagram, Discord
• 💾 Memory Efficient: Streaming API for large files
• 🐍 Pythonic API: Easy to use, well-documented
• 🎯 Type Hints: Full IDE support with .pyi stubs
• 🔧 Flexible: Filter, merge, and transform messages

Installation

pip install chatpack

Or build from source:

pip install maturin
maturin develop --release

Quick Start

Simple Parsing

import chatpack

# Parse Telegram export
messages = chatpack.parse_telegram("result.json", merge=True, min_length=5)

# Parse WhatsApp export
messages = chatpack.parse_whatsapp("chat.txt", merge=True)

# Parse Instagram export
messages = chatpack.parse_instagram("messages.json")

# Parse Discord export
messages = chatpack.parse_discord("export.json")

Object-Oriented API

# Create parser instance
parser = chatpack.TelegramParser()

# Parse with filters
messages = parser.parse(
    "result.json",
    merge=True,
    min_length=10,
    date_from="2024-01-01",
    date_to="2024-12-31"
)

# Access message properties
for msg in messages:
    print(f"{msg.sender}: {msg.content}")
    print(f"Timestamp: {msg.timestamp}")

Streaming Large Files

For files that don't fit in memory:

# Stream messages one by one
parser = chatpack.TelegramStreamParser("huge_export.json")

for msg in parser:
    process_message(msg)  # O(1) memory usage

Integration with Pandas

import chatpack
import pandas as pd

# Parse messages
messages = chatpack.parse_telegram("result.json", merge=True)

# Convert to DataFrame
df = pd.DataFrame([m.to_dict() for m in messages])

# Analyze
print(df.groupby('sender')['content'].count())

Filtering Messages

# Create filter configuration
config = chatpack.FilterConfig(
    min_length=10,
    max_length=1000,
    sender="Alice",
    date_from="2024-01-01",
    date_to="2024-12-31"
)

# Apply filters
filtered = chatpack.apply_filters(messages, config)

Merging Consecutive Messages

# Merge messages from same sender within 5 minutes
merged = chatpack.merge_consecutive(messages, time_threshold=300)

API Reference

Parsers

Eager Loading

• parse_telegram(path, merge=False, min_length=None, date_from=None, date_to=None)
• parse_whatsapp(path, merge=False, min_length=None, date_from=None, date_to=None)
• parse_instagram(path, merge=False, min_length=None, date_from=None, date_to=None)
• parse_discord(path, merge=False, min_length=None, date_from=None, date_to=None)

Streaming (for large files)

• TelegramStreamParser(path) - Returns iterator
• WhatsAppStreamParser(path) - Returns iterator
• InstagramStreamParser(path) - Returns iterator
• DiscordStreamParser(path) - Returns iterator

Classes

Message

msg = chatpack.Message(
    sender="Alice",
    content="Hello, world!",
    timestamp="2024-01-15T10:30:00Z",
    platform="telegram"
)

# Properties
msg.sender      # str
msg.content     # str
msg.timestamp   # Optional[str] (ISO 8601)
msg.platform    # Optional[str]

# Methods
msg.to_dict()   # Convert to dictionary

FilterConfig

config = chatpack.FilterConfig(
    min_length=5,
    max_length=1000,
    sender="Alice",
    date_from="2024-01-01",
    date_to="2024-12-31"
)

# Builder pattern
config.with_min_length(10)
config.with_sender("Bob")

OutputConfig

config = chatpack.OutputConfig(
    include_timestamps=True,
    include_platform=True
)

Utility Functions

• merge_consecutive(messages, time_threshold=300) - Merge messages from same sender
• apply_filters(messages, config) - Apply filter configuration

Platform Support

Platform	Format	Special Features
Telegram	JSON	Service messages, forwarded messages
WhatsApp	TXT	Auto-detects 4 locale date formats
Instagram	JSON	Fixes Mojibake encoding (Meta bug)
Discord	JSON/CSV/TXT	Attachments, stickers, replies

Performance

chatpack-py leverages Rust for parsing, making it significantly faster than pure Python implementations:

• 10-100x faster than regex-based parsers
• Memory efficient streaming for multi-GB files
• Zero-copy where possible with PyO3

Development

Setup

# Clone repository
git clone https://github.com/berektassuly/chatpack-py
cd chatpack-py

# Install development dependencies
pip install maturin pytest

# Build in development mode
maturin develop

# Run tests
pytest

Project Structure

chatpack-py/
├── Cargo.toml          # Rust dependencies
├── pyproject.toml      # Python package metadata
├── src/
│   ├── lib.rs          # PyO3 module entry point
│   ├── types.rs        # Python type wrappers
│   ├── parsers.rs      # Parser implementations
│   ├── streaming.rs    # Streaming iterators
│   └── conversion.rs   # Rust ↔ Python conversion
├── python/
│   └── chatpack/
│       ├── __init__.py
│       └── chatpack.pyi  # Type stubs
└── tests/
    ├── test_basic.py
    └── test_parsers.py

Building Wheels

# Build for current platform
maturin build --release

# Build for multiple platforms (requires Docker)
maturin build --release --manylinux 2014

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

License

MIT License - see LICENSE for details.

Credits

Built on top of the excellent chatpack Rust library by Berektassuly.

Links

• Documentation
• Rust chatpack
• Issue Tracker