Wirehead

Caching system for scaling of synthetic data generators using MongoDB.

Features

• Cache and efficiently serve synthetic data from generators
• Scalable architecture using MongoDB for storage
• Support for numpy and torch tensors
• Configurable caching behavior

Quick Start

# Generator example
import numpy as np
from wirehead import WireheadGenerator 

def create_generator():
    while True: 
        img = np.random.rand(256,256,256)
        lab = np.random.rand(256,256,256)
        yield (img, lab)

brain_generator = create_generator()
wirehead_runtime = WireheadGenerator(
    generator = brain_generator,
    config_path = "config.yaml" 
)
wirehead_runtime.run_generator()

# Dataset example
from wirehead import MongoheadDataset
dataset = MongoheadDataset(config_path = "config.yaml")
data = dataset[[0]]

MongoDB Setup Required

Requires a running MongoDB instance.

Documentation

For full documentation and examples, visit: https://github.com/neuroneural/wirehead