django-gyro 0.2.0

Django Gyro

Django Gyro

A declarative system for importing and exporting CSV data with Django models. Django Gyro provides a clean, validation-rich way to map CSV columns to Django model fields with automatic foreign key resolution and intelligent data slicing capabilities.

Features

• Declarative Import System: Define how CSV data maps to your Django models using simple class-based importers
• Automatic Foreign Key Resolution: Intelligently resolves relationships between models during import
• Circular Dependency Handling: Automatically resolves circular relationships (e.g., Customer ↔ CustomerReferral)
• ID Remapping Strategies: Multiple strategies for handling ID conflicts during imports
• Data Slicing & Export: Export subsets of your data with complex relationships intact
• Multi-tenant Support: Built-in support for multi-tenant architectures with tenant-aware remapping
• PostgreSQL Bulk Loading: High-performance imports using PostgreSQL COPY operations
• Validation-Rich: Comprehensive validation during import/export operations
• Progress Tracking: Built-in progress bars for long-running operations

Quick Start

Installation

pip install django-gyro

Basic Usage

1. Define Your Importers

# myapp/importers.py
from django_gyro import Importer
from myapp.models import Tenant, Shop, Customer, Product, Order

class TenantImporter(Importer):
    model = Tenant

    class Columns:
        pass

class ShopImporter(Importer):
    model = Shop

    class Columns:
        tenant = Tenant

class CustomerImporter(Importer):
    model = Customer

    class Columns:
        shop = Shop
        tenant = Tenant

2. Export Data

from django_gyro import DataSlicer, ImportJob

# Define what data to export
tenant = Tenant.objects.filter(id=1)
shops = Shop.objects.filter(tenant=tenant)
customers = Customer.objects.filter(shop__in=shops)

# Export to CSV files
DataSlicer.run(
    source=DataSlicer.Postgres(database_url),
    target=DataSlicer.File('/path/to/export/'),
    jobs=[
      ImportJob(model=Tenant, query=tenant),
      ImportJob(model=Shop, query=shops),
      ImportJob(model=Customer, query=customers),
   ],
)

3. Import Data

# Import from CSV files
DataSlicer.run(
    source=DataSlicer.File('/path/to/import/'),
    target=DataSlicer.Postgres(database_url),
    jobs=[
      ImportJob(model=Tenant),
      ImportJob(model=Shop),
      ImportJob(model=Customer),
    ],
)

4. Management Command for CSV Import

Django Gyro includes a management command for importing CSV data with ID remapping:

# Import CSV data with automatic ID remapping
python manage.py import_csv_data /path/to/csv/files/ --strategy sequential

# Import with tenant-aware remapping (for multi-tenant setups)
python manage.py import_csv_data /path/to/csv/files/ --strategy tenant --tenant-id 1

ID Remapping Strategies:

• sequential: Assigns new sequential IDs starting from MAX(existing_id) + 1
• hash: Uses deterministic hashing based on business keys
• tenant: Automatically handles tenant-specific ID remapping
• none: Preserves original IDs (use with caution)

Use Cases

• Data Migration: Move data between environments while preserving relationships
• Selective Exports: Export specific subsets of data for development or testing
• Multi-tenant Data Management: Handle complex tenant-based data relationships
• CSV Import/Export: Robust CSV handling with validation and error reporting

Documentation

For detailed documentation, examples, and advanced usage, see TECHNICAL_DESIGN.md.

Requirements

• Python 3.8+
• Django 3.2+
• PostgreSQL (for DataSlicer operations)

Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• 📖 Documentation
• 🐛 Issue Tracker
• 💬 Discussions