Skip to main content

The Dark Knight of Data Structures - High-Performance C++ Sparse Bitset for Python.

Project description

🦇 Bat-Bit: The Dark Knight of Data Structures

"It's not who I am underneath, but what I do that defines me." High-Performance C++ Sparse Bitset Engine for Python.

PyPI version License: MIT Buy Me A Coffee


🌃 The Problem

Python's built-in set() and list are powerful, but they are memory-hungry. Trying to store 10 Million integers in a Python set can consume over 600 MB of RAM. If you work with Big Data, High-Frequency Trading, or massive IoT logs, you will hit MemoryError fast.

🦇 The Solution

Bat-Bit is a blazing fast C++ extension wrapped for Python. It uses Sparse Paging Bitset architecture to store data.

  • It allocates memory only when needed (Sparse).
  • It performs operations at the Bit-level (Processor friendly).
  • It releases the GIL (Global Interpreter Lock) for True Multi-Core Parallelism.

⚡ Benchmarks (The Shocking Truth)

Tested on 10 Million Records (Integers):

Metric Standard Python set() Bat-Bit 🦇 Improvement
Memory Usage ~600 MB ~120 MB 80% RAM Savings 💾
Insertion Time ~12.0s ~1.2s 10x Faster 🚀
Ops / Second ~800k 7.7 Million Hypersonic Speed 🔥

🛠 Installation

Get it directly from PyPI:

pip install bat-bit

Requires: Python 3.7+

💻 Usage

1. Basic Usage (Like a Set)

import bat_bit

# Initialize the BatCave
bruce = bat_bit.BatCave()

# Deploy single gadget (Insert number)
bruce.deploy(500)
bruce.deploy(1000000000) # Handles massive gaps effortlessly!

# Signal check (Search / Lookup) - O(1) Time Complexity
if bruce.signal(500):
    print("Target Found!") # Prints: Target Found!

if not bruce.signal(999):
    print("Target Not Found!")

2. High-Speed Batch Processing

Avoid Python overhead by sending data in bulk.

data = [x for x in range(1000000)] # 1 Million items
bruce.deploy_batch(data) # Super Fast

3. 🚀 Hypersonic Parallel Mode (Multi-Core)

Unleash the full power of your CPU. This method releases the GIL and uses all available CPU cores to insert data in parallel.

import bat_bit
import random

# Generate 10 Million numbers
data_stream = [random.randint(0, 1000000000) for _ in range(10000000)]

bruce = bat_bit.BatCave()

# Fires on ALL Cores (e.g., 8 Threads)
bruce.deploy_batch_parallel(data_stream)

print(f"Memory Used: {bruce.memory_usage() / 1024 / 1024:.2f} MB")

🧠 How it Works

Unlike a Hash Map that stores the number itself (32/64 bits) + pointers + overhead, Bat-Bit treats memory as a Bit-Map.

The number 5 is just the 5th bit in a page set to 1.

Sparse Paging: If you store numbers 1 and 1,000,000,000, Bat-Bit only creates 2 tiny pages of memory. The empty space in between takes 0 RAM.

🤝 Contributing

We welcome fellow vigilantes!

  1. Fork the repo.
  2. Create your feature branch (git checkout -b feature/NewGadget).
  3. Commit your changes.
  4. Push to the branch.
  5. Open a Pull Request.

📜 License

Distributed under the MIT License. See LICENSE for more information.

Built with 🖤 using C++ & Pybind11

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bat_bit-1.0.0.tar.gz (4.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

bat_bit-1.0.0-cp313-cp313-win_amd64.whl (81.4 kB view details)

Uploaded CPython 3.13Windows x86-64

File details

Details for the file bat_bit-1.0.0.tar.gz.

File metadata

  • Download URL: bat_bit-1.0.0.tar.gz
  • Upload date:
  • Size: 4.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.6

File hashes

Hashes for bat_bit-1.0.0.tar.gz
Algorithm Hash digest
SHA256 20bd706574d0bda2a4adaedadda3701d25bb9354021b9e4791f0f573138c6c94
MD5 4f69ceb6ba1d7cf42385919c60c79f11
BLAKE2b-256 3b64aecd880d2ca7272bf3ae8503d1b548384691fc4fcfd7bc1df908dd5610df

See more details on using hashes here.

File details

Details for the file bat_bit-1.0.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: bat_bit-1.0.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 81.4 kB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.6

File hashes

Hashes for bat_bit-1.0.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 cf797add03cf343ad751c210a676b411097f917f378ebdeaa1f125cf64b177d3
MD5 d257673e52954496188b797ae84d0fa3
BLAKE2b-256 fb5b7f20ecba7ac3e3a19a6e56742a2b97f7ceb1845a3f0468182788bdf908da

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page