Skip to main content

The Dark Knight of Data Structures - High-Performance C++ Sparse Bitset for Python.

Project description

🦇 Bat-Bit: The Dark Knight of Data Structures

"It's not who I am underneath, but what I do that defines me." High-Performance C++ Sparse Bitset Engine for Python.

PyPI version License: MIT Buy Me A Coffee


🌃 The Problem

Python's built-in set() and list are powerful, but they are memory-hungry. Trying to store 10 Million integers in a Python set can consume over 600 MB of RAM. If you work with Big Data, High-Frequency Trading, or massive IoT logs, you will hit MemoryError fast.

🦇 The Solution

Bat-Bit is a blazing fast C++ extension wrapped for Python. It uses Sparse Paging Bitset architecture to store data.

  • It allocates memory only when needed (Sparse).
  • It performs operations at the Bit-level (Processor friendly).
  • It releases the GIL (Global Interpreter Lock) for True Multi-Core Parallelism.

⚡ Benchmarks (The Shocking Truth)

Tested on 10 Million Records (Integers):

Metric Standard Python set() Bat-Bit 🦇 Improvement
Memory Usage ~600 MB ~120 MB 80% RAM Savings 💾
Insertion Time ~12.0s ~1.2s 10x Faster 🚀
Ops / Second ~800k 7.7 Million Hypersonic Speed 🔥

🛠 Installation

Get it directly from PyPI:

pip install bat-bit

Requires: Python 3.7+

💻 Usage

1. Basic Usage (Like a Set)

import bat_bit

# Initialize the BatCave
bruce = bat_bit.BatCave()

# Deploy single gadget (Insert number)
bruce.deploy(500)
bruce.deploy(1000000000) # Handles massive gaps effortlessly!

# Signal check (Search / Lookup) - O(1) Time Complexity
if bruce.signal(500):
    print("Target Found!") # Prints: Target Found!

if not bruce.signal(999):
    print("Target Not Found!")

2. 🆕 New in v1.2.0: Arrays & Maps

Bat-Bit now replaces Python Lists and Dictionaries with high-performance C++ alternatives.

📏 BatVector (The Fast Array)

A contiguous memory array for high-speed numeric processing.

import bat_bit

# Create Vector
vec = bat_bit.BatVector()

# Batch Push (Hypersonic Speed)
data = list(range(1000000))
vec.push_batch(data)

# Access (O(1))
print(vec[500]) 

🗺️ BatMap (The Efficient Dictionary)

A typed hash map that avoids Python object overhead.

m = bat_bit.BatMap()

# Batch Insert
keys = [1, 2, 3]
values = [10.5, 20.5, 30.5]
m.put_batch(keys, values)

print(m[1]) # Output: 10.5

3. High-Speed Batch Processing

Avoid Python overhead by sending data in bulk.

data = [x for x in range(1000000)] # 1 Million items
bruce.deploy_batch(data) # Super Fast

4. 🚀 Hypersonic Parallel Mode (Multi-Core)

Unleash the full power of your CPU. This method releases the GIL and uses all available CPU cores to insert data in parallel.

import bat_bit
import random

# Generate 10 Million numbers
data_stream = [random.randint(0, 1000000000) for _ in range(10000000)]

bruce = bat_bit.BatCave()

# Fires on ALL Cores (e.g., 8 Threads)
bruce.deploy_batch_parallel(data_stream)

print(f"Memory Used: {bruce.memory_usage() / 1024 / 1024:.2f} MB")

🧠 How it Works

Unlike a Hash Map that stores the number itself (32/64 bits) + pointers + overhead, Bat-Bit treats memory as a Bit-Map.

The number 5 is just the 5th bit in a page set to 1.

Sparse Paging: If you store numbers 1 and 1,000,000,000, Bat-Bit only creates 2 tiny pages of memory. The empty space in between takes 0 RAM.

🤝 Contributing

We welcome fellow vigilantes!

  1. Fork the repo.
  2. Create your feature branch (git checkout -b feature/NewGadget).
  3. Commit your changes.
  4. Push to the branch.
  5. Open a Pull Request.

📜 License

Distributed under the MIT License. See LICENSE for more information.

Built with 🖤 using C++ & Pybind11

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bat_bit-1.2.0.tar.gz (6.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

bat_bit-1.2.0-cp313-cp313-win_amd64.whl (87.7 kB view details)

Uploaded CPython 3.13Windows x86-64

File details

Details for the file bat_bit-1.2.0.tar.gz.

File metadata

  • Download URL: bat_bit-1.2.0.tar.gz
  • Upload date:
  • Size: 6.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.6

File hashes

Hashes for bat_bit-1.2.0.tar.gz
Algorithm Hash digest
SHA256 29ec86e5f1b09d365e8f954dd3a0a2cd80bda2e2d7acccc57b9a9cd9c83b73a5
MD5 c4a865d37aad82fbf692b700899fc026
BLAKE2b-256 f9a35c727a59cd9846ba00644f40614eb3474a134abb3b3a483e73c35aa2ed6a

See more details on using hashes here.

File details

Details for the file bat_bit-1.2.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: bat_bit-1.2.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 87.7 kB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.6

File hashes

Hashes for bat_bit-1.2.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 ca5795e76acef254f2e98996d1821f7b4d48f816cc8b3c2afd6de7f19dfd0401
MD5 cf574b47d761d2a6973ac6f7fa87c954
BLAKE2b-256 2fdcd6e35bc5534f0e6f6a0dc80d981215e58f09002d6fb18c62d60f3cacbc8b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page