rememory 0.1.0

A lightweight shared memory Dict and List type for safe multiprocessing access

Rememory

Rememory provides shared-memory data structures (RememoryDict and RememoryList) that work safely across multiple processes – even across completely independent Python interpreters – using OS-level named locks.

This is designed for scenarios where you need a simple, Python-native way to share structured state between processes without using a database, message broker, or serialization.

Features

• Shared-memory backend using multiprocessing.shared_memory
• Cross-process synchronization with OS-level locks (Windows mutex / POSIX semaphore)
• Works across separate scripts and interpreters, not just multiprocessing.Process
• Drop-in replacements for dict and list
• Type-safe generics for editors and type checkers (e.g. RememoryDict[str, int])

---

Installation

pip install rememory

Dependencies

On Linux/macOS posix_ipc, on Windows pywin32:

---

Basic Usage

RememoryDict

from rememory import Dict # RMDict, or RememoryDict also work

# Create or attach to a shared dict
shared: Dict[str, dict[str, int]] = Dict("game_state")

# Write to it (any process using the same name sees these changes)
shared["player1"] = {"score": 10, "level": 2}

# Read from it
print(shared["player1"])

# Iterate
for key, value in shared.items():
    print(key, value)

Any process that does Dict("game_state") will connect to the same shared memory block.

RememoryList

from rememory import List # RMList, or RememoryList also work

shared_list: List[str] = List("chat")

# Append items
shared_list.append("hello")
shared_list.append("world")

# Read
print(shared_list[0])  # "hello"
for item in shared_list:
    print(item)

---

Multiprocessing Example

from multiprocessing import Process
from rememory import Dict, List

SHARED_DICT = "game"
SHARED_LIST = "events"

def worker():
    d = Dict[str, dict[str, int]](SHARED_DICT)
    pid = str(os.getpid())
    d[pid] = {"score": 1, "level": 1}
    d[pid]["score"] += 1

    l = List[str](SHARED_LIST)
    l.append(f"{pid}-joined")

if __name__ == "__main__":
    d = Dict[str, dict[str, int]](SHARED_DICT)
    l = List[str](SHARED_LIST)

    # Clear previous state
    d._write_data({})
    l._write_data([])

    procs = [Process(target=worker) for _ in range(4)]
    for p in procs: p.start()
    for p in procs: p.join()

    print("Shared dict:", dict(d.items()))
    print("Shared list:", list(l))

---

Locking

Rememory uses OS-named synchronization primitives:

• Windows: Named mutex via pywin32
• Linux/macOS: POSIX named semaphore via posix_ipc

This ensures that even separate Python scripts respect locking when using the same shared memory name.

---

When to Use

• Game engines or simulations needing shared state between processes
• Multi-process pipelines without a heavy DB or broker
• Debugging tools that need live shared state

When Not to Use

• When your data doesn’t fit in memory
• When you need persistence beyond process lifetime (shared memory is volatile)

---

License

This project is released under the MIT License. See the LICENSE file for details.

---

Author

Dr Nichola J Walch [e-mail]