M2M Gaussian Splatting

Hierarchical Retrieval and Memory Management for 3D Gaussian Splatting

M2M Gaussian Splatting provides fast similarity search and efficient memory management for large-scale 3D Gaussian splat datasets. Optimized for CPU with Numba JIT compilation.

✨ Features

• Hierarchical Indexing (HRM2) - Two-level clustering for fast retrieval
• 640D Embeddings - Position, color, and attribute encodings
• Numba JIT Acceleration - 5-10x speedup over pure Python
• Memory Management - Three-tier memory (VRAM/RAM/Disk)
• High Recall - >90% recall at 50x+ speedup
• CPU Optimized - No GPU required

📦 Installation

# Clone repository
git clone https://github.com/schwabauerbriantomas-gif/m2m-gaussian-splatting.git
cd m2m-gaussian-splatting

# Install dependencies
pip install -r requirements.txt

# Or with pip
pip install numpy numba scipy scikit-learn

🚀 Quick Start

from src.core import GaussianSplat, HRM2Engine, generate_test_splats

# Generate test data
splats = generate_test_splats(n_splats=10000)

# Create and index
engine = HRM2Engine(n_coarse=50, n_fine=200)
engine.add_splats(splats)
engine.index()

# Query for similar splats (use the engine's embeddings)
query = engine.embeddings[0]  # Use first splat's embedding
results = engine.query(query, k=10)

for splat, distance in results:
    print(f"Splat {splat.id}: distance={distance:.4f}")

📊 Performance

Benchmarks on ryzen 5 3400g (640D embeddings, k=10):

🔧 Architecture

Embedding (640D)

┌────────────────────────────────────────────────────────┐
│                    640D Embedding                      │
├────────────────────────────────────────────────────────┤
│  Position (64D)  │  Color (512D)  │  Attributes (64D)  │
│  Sinusoidal PE   │  Histogram     │  Opacity/Scale     │
└────────────────────────────────────────────────────────┘

HRM2 Index

Query Vector
     │
     ▼
┌──────────────────┐
│ Level 1: Coarse  │  ← K-Means (100 clusters)
│ Find nearest     │
└──────────────────┘
     │
     ▼
┌──────────────────┐
│ Level 2: Fine    │  ← K-Means (1000 clusters)
│ Search within    │
└──────────────────┘
     │
     ▼
  Top-K Results

📁 Project Structure

m2m-gaussian-splatting/
├── src/
│   ├── core/
│   │   ├── splat_types.py     # Data structures
│   │   ├── encoding.py        # Numba JIT encoders
│   │   ├── clustering.py      # K-Means implementation
│   │   └── hrm2_engine.py     # Hierarchical retrieval
│   └── memory/
│       └── manager.py         # Memory tiers
├── scripts/
│   ├── quick_demo.py          # Getting started
│   └── run_benchmarks.py      # Performance tests
├── tests/
│   └── test_*.py
├── docs/
│   └── ARCHITECTURE.md
├── requirements.txt
└── README.md

🎯 Use Cases

3D Scene Retrieval

Find similar regions in large 3D scans.

# Find similar scene regions
results = engine.query(region_embedding, k=20)

Asset Search

Search through millions of 3D assets.

# Find assets with similar appearance
similar = engine.query(asset_embedding, k=10)

Point Cloud Processing

Efficient queries on LiDAR data.

# Find points matching a pattern
matches = engine.query(pattern_embedding, k=100)

🧪 Running Tests

# Run all tests
python -m pytest tests/ -v

# Run with coverage
python -m pytest tests/ --cov=src

# Run benchmarks
python scripts/run_benchmarks.py

📖 API Reference

GaussianSplat

splat = GaussianSplat(
    id=0,
    position=[x, y, z],        # 3D position
    color=[r, g, b],           # RGB color
    opacity=0.9,               # Transparency
    scale=[sx, sy, sz],        # Scale factors
    rotation=[w, x, y, z]      # Quaternion
)

HRM2Engine

engine = HRM2Engine(
    n_coarse=100,    # Coarse clusters
    n_fine=1000,     # Fine clusters
    n_probe=5,       # Clusters to search
)

engine.add_splats(splats)
engine.index()

results = engine.query(query_vector, k=10)

SplatMemoryManager

memory = SplatMemoryManager(
    vram_limit=100000,   # Hot tier
    ram_limit=1000000,   # Warm tier
)

memory.add_splats(splats)
splat = memory.get_splat(splat_id)

🔬 Technical Details

Position Encoding (64D)

NeRF-style multi-frequency sinusoidal encoding:

PE(x, 2i)   = sin(x_norm * 2^i)
PE(x, 2i+1) = cos(x_norm * 2^i)

where x_norm is the coordinate normalized to [0, 1].

Color Encoding (512D)

Histogram-based with Gaussian smoothing:

• 8 bins per channel
• 8³ = 512 total dimensions
• Smoothed with Gaussian kernel

K-Means Implementation

• K-Means++ initialization
• Numba JIT for speed
• Parallel distance computation

🤝 Contributing

Contributions welcome! Please:

1. Fork the repository

2. Create feature branch (git checkout -b feature/amazing)

3. Commit changes (git commit -m 'Add amazing feature')

4. Push to branch (git push origin feature/amazing)

5. Open a Pull Request

📄 License

MIT License - see LICENSE for details.

🙏 Acknowledgments

• 3D Gaussian Splatting - Kerbl et al.
• Numba - JIT compilation
• scikit-learn - Reference implementations

📧 Contact

Brian Schwabauer - schwabauerbriantomas@gmail.com

Project: https://github.com/schwabauerbriantomas-gif/m2m-gaussian-splatting

🔖 Keywords

gaussian-splatting 3dgs hierarchical-retrieval vector-search numba kmeans similarity-search point-cloud 3d-reconstruction nerf embedding cpu-optimized memory-management hrm2 clustering jit-compilation python