hotcore 1.2.1

A Redis-based hierarchical entity model for application data management

hotcore

A Redis-backed entity model for managing hierarchical structured data with relationship support and attribute-based search capabilities.

Overview

Hotcore provides a clean, Redis-backed data model for applications that need hierarchical data storage with powerful querying capabilities. It organizes entities in parent-child relationships and indexes all attributes automatically for efficient searching.

Key Features

• Hierarchical Structure: Organize entities in a tree-like parent-child structure
• Attribute-Based Search: Find entities by exact values or patterns (wildcards)
• Geospatial Search: Automatic geospatial indexing and bounding box search for entities with coordinates
• Optimistic Locking: Safely modify entities in concurrent environments
• Automatic Indexing: All entity attributes are automatically indexed for fast retrieval
• Type Hinting: Comprehensive type annotations for improved IDE support
• Error Handling: Robust error handling with informative messages
• TLS-Ready Connections: Flexible SSL/TLS configuration with custom contexts and connection parameters

Installation

Prerequisites

• Python 3.10+
• redis-py 5.0.4 or newer
• Redis server (local or remote)

Setup

1. Install the package:

# From PyPI (minimal installation)
pip install hotcore

# For development with all tools
pip install -r requirements-dev.txt

# From source (minimal installation)
git clone https://github.com/your-org/hotcore.git
cd hotcore
pip install -e .

# Development installation with all tools
pip install -e ".[dev]"

# Enable geospatial extras (includes Uber's h3 library)
pip install -e ".[h3]"

2. Make sure Redis is running:

# Local Redis server (default)
redis-server

# Or connect to a remote Redis instance in your code

Quick Start

from hotcore import Model

# Initialize the model (connects to Redis)
model = Model(host='localhost')  # Default port is 6379

# Create a root entity
root = model.init({})  # Generates a UUID automatically
root['name'] = 'Root Entity'
root['type'] = 'container'
model.create('root', root)  # 'root' is a special parent ID for root entities

# Create a child entity
child = model.init({})
child['name'] = 'Child Entity'
child['type'] = 'item'
child['status'] = 'active'
model.create(root['uuid'], child)  # Root UUID as parent

# Retrieve an entity
entity = model.get(child['uuid'])
print(entity)  # {'uuid': '...', 'name': 'Child Entity', 'type': 'item', 'status': 'active'}

# Update an entity
changes = {'uuid': child['uuid'], 'status': 'inactive', 'priority': 'high'}
model.apply(changes)

# Delete an entity
model.delete(child)

# Find entities by attribute values
for entity in model.find(type='item', status='active'):
    print(entity)

# Find entities with pattern matching (wildcards)
for entity in model.find(name='Child*'):
    print(entity)

# Geospatial search for entities within a bounding box
# (entities with 'lat' and 'long' attributes are automatically indexed)
entities_in_nyc = model.search_bounding_box(40.0, 41.0, -74.0, -73.0)
for entity in entities_in_nyc:
    print(f"{entity['name']} at ({entity['lat']}, {entity['long']})")

# Get all children of a parent
for child in model.get_children(root['uuid']):
    print(child)

# Get parent of an entity
parent = model.get_parent(child['uuid'])
print(parent)

TLS / SSL configuration

RedisConnectionManager and the Model facade now accept custom TLS settings. Supply an ssl.SSLContext or explicit connection_kwargs to match your Redis deployment:

import ssl
from hotcore import Model

tls_context = ssl.create_default_context()
tls_context.load_verify_locations("/etc/ssl/certs/redis-ca.pem")

secure_model = Model(
    host="redis.example.com",
    port=6380,
    ssl_context=tls_context,
    connection_kwargs={"ssl_check_hostname": True},
)

You can also provide write_connection_kwargs (and write_ssl_context) if your write path needs a different configuration than reads.

Project Structure

• hotcore/model.py – Facade that wires storage, relationships, search, geospatial, and optional H3 indexing.
• hotcore/connection.py – Redis connection pooling and key helpers.
• hotcore/storage.py – CRUD operations with attribute indexing.
• hotcore/relationships.py – Parent/child traversal utilities.
• hotcore/search.py – Attribute-based and wildcard search helpers.
• hotcore/geospatial.py – Redis GEO index management.
• hotcore/h3_index.py – Optional Uber H3 integration (requires hotcore[h3]).
• hotcore/hotcore.py – Compatibility exports for the legacy import surface.

Unit and integration tests live under tests/; see tests/README.md for an overview.

Data Model Concepts

Entities

An entity is a dictionary-like object with attributes:

• Must have a uuid attribute (automatically generated by model.init())
• Can contain any number of key-value pairs (attributes)
• Values should be strings for optimal indexing and searching
• Geospatial Support: Entities with lat and long attributes are automatically added to a Redis geospatial index

Relationships

• Each entity can have one parent (except root entities)
• Each entity can have multiple children
• The parent-child relationship forms a tree structure

Indexing

• All entity attributes are automatically indexed
• Indexes enable fast lookup by attribute value
• Pattern-based searches are supported (using wildcards)

Advanced Usage

Optimistic Locking

The model uses optimistic locking to handle concurrent modifications:

# If two processes try to modify the same entity simultaneously,
# one will succeed and the other will automatically retry (up to 3 times by default)

Pattern Matching

You can use Redis pattern matching in searches:

# Find entities with names starting with "A"
for entity in model.find(name='A*'):
    print(entity)

# Find entities with specific pattern in type
for entity in model.find(type='user_?????'):
    print(entity)

# Find entities with names containing specific characters
for entity in model.find(name='*[Aa]dmin*'):
    print(entity)

Combining Search Criteria

When multiple criteria are provided, all must match (logical AND):

# Find active users with admin role
for entity in model.find(type='user', status='active', role='admin'):
    print(entity)

Geospatial Search

Hotcore automatically manages geospatial indexing for entities with lat and long attributes:

# Create entities with coordinates (automatically indexed)
user1 = model.init({
    'name': 'Alice Johnson',
    'type': 'user',
    'lat': 40.7128,    # NYC latitude
    'long': -74.0060   # NYC longitude
})
model.create('root', user1)

user2 = model.init({
    'name': 'Bob Smith',
    'type': 'user',
    'lat': 34.0522,    # LA latitude
    'long': -118.2437  # LA longitude
})
model.create('root', user2)

# Search for entities within a bounding box by type
nyc_users = model.search_bounding_box(40.0, 41.0, -74.0, -73.0, "user")
for user in nyc_users:
    print(f"{user['name']} in NYC area")

# Update coordinates (automatically updates geospatial index)
changes = {
    'uuid': user1['uuid'],
    'lat': 42.3601,    # Boston latitude
    'long': -71.0589   # Boston longitude
}
model.apply(changes)

# Search again to see updated locations
boston_users = model.search_bounding_box(42.0, 43.0, -72.0, -71.0)

Key Features:

• Automatic Indexing: Entities with lat and long are automatically added to the geospatial index
• Transparent Updates: Coordinate changes automatically update the geospatial index
• Bounding Box Search: Find entities within rectangular geographic areas
• Performance: O(log(N)) for adding/updating, O(N+log(M)) for searching
• Validation: Coordinates are validated to ensure they're within valid ranges (-90 to 90° lat, -180 to 180° lon)

API Reference

Model Class

Initialization

• Model(host='localhost', port=6379, db=0): Initialize the model with Redis connection parameters

Entity Management

• init(entity: dict) -> dict: Add a UUID to an entity dictionary
• create(parent_uuid: str, entity: dict) -> dict: Create and save a new entity
• get(entity_uuid: str) -> dict: Retrieve an entity by its UUID
• apply(change: dict) -> None: Apply changes to an existing entity
• delete(entity: dict) -> None: Delete an entity

Relationships

• get_children(parent_uuid: str) -> Generator[dict]: Get all children of a parent entity
• get_parent(child_uuid: str) -> dict: Get the parent of an entity

Search

• find(**kwargs) -> Generator[dict]: Find entities matching attribute criteria
• get_entity_from_index(index_hit: str) -> Generator[dict]: Get entities from a specific index
• search_bounding_box(min_lat, max_lat, min_lon, max_lon, entity_type="default") -> List[dict]: Search for entities of a specific type within a geographic bounding box

Best Practices

1. Use UUIDs for entity identification: Generate UUIDs using model.init({})
2. Keep attributes as strings: For optimal indexing and searching
3. Manage entity lifecycle: Create, update, and delete entities using the model API
4. Use pattern matching judiciously: Wildcard searches can be powerful but resource-intensive
5. Handle errors: Implement proper error handling for potential Redis errors
6. Geospatial coordinates: Use lat and long attributes for automatic geospatial indexing and search

Performance Considerations

1. Connection Pooling: The model uses Redis connection pooling for optimal performance
2. Pipelines: Operations use Redis pipelines to minimize network roundtrips
3. Optimistic Locking: Helps maintain data consistency with minimal performance impact
4. Temporary Sets: Complex searches using wildcards create temporary sets that expire after 60 seconds

Testing

Hotcore's test suite is designed to work without requiring a Redis server for most tests. We use fakeredis to simulate Redis functionality for unit and integration tests.

Running Tests

# Run tests using the provided script
./run_tests.sh

# Run only unit tests
./run_tests.sh --unit

# Run only integration tests
./run_tests.sh --integration

# Run tests with coverage report
./run_tests.sh --cov

# Run with a real Redis server
./run_tests.sh --real-redis

With Virtual Environment (venv)

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate

# Install development dependencies
pip install -r requirements-dev.txt

# Run tests that don't require Redis
python -m pytest tests/unit/ tests/integration/

# Run all tests including those requiring Redis
USE_REAL_REDIS=true python -m pytest

With System Python

# Install development dependencies
pip install -r requirements-dev.txt

# Run tests that don't require Redis
python -m pytest tests/unit/ tests/integration/

# Run all tests including those requiring Redis
USE_REAL_REDIS=true python -m pytest

Test Organization

The tests are organized to minimize Redis server dependencies:

• tests/unit/ - Unit tests (no Redis server needed, uses fakeredis)
• tests/integration/ - Integration tests (no Redis server needed, uses fakeredis)
• tests/real_redis/ - Tests that require a real Redis server (skipped by default)

For more details, see the tests README.

Development

• Read the contribution guide for environment setup, coding standards, and release process.
• All contributions are covered by our Code of Conduct.
• Security issues should be reported privately (see SECURITY.md).

Testing Strategy

• Default test runs use fakeredis and cover both unit and integration suites (pytest or ./run_tests.sh).
• Tests that require a real Redis instance live in tests/real_redis/ and are marked with @pytest.mark.redis_required.
• Optional geospatial features can be exercised by installing hotcore[h3]; related tests skip automatically when H3 is unavailable.
• CI (planned) will enforce formatting (black, isort), linting (flake8), type checks (mypy), and the full pytest suite across supported Python versions.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Optional Dependencies

• hotcore[h3]: Enables H3-based geospatial indexing via Uber's h3 library (Apache License 2.0).

Third-Party Notices

This project distributes under the MIT License. Optional geospatial functionality depends on Uber's h3 library, which is available under the Apache License 2.0.

License

MIT License