sb-sync 1.6.1

Django package for data synchronization with PUSH/PULL APIs

SB Sync - Django Data Synchronization Package

A robust Django package for data synchronization with PUSH/PULL APIs, featuring JWT authentication, comprehensive logging, performance optimizations, and multi-tenant, role-based access control for any Django application.

🚀 Features

• PUSH/PULL API Endpoints: Bidirectional data synchronization
• 🌐 Web-Based Configuration Interface: Visual management of permissions and settings
• 📋 Audit Trails: Complete change history tracking with Django Simple History
• JWT Authentication: Secure token-based authentication
• Multi-Tenant Support: Organization-based data isolation
• Role-Based Access Control: Granular permissions per user role
• Comprehensive Logging: Detailed sync operation tracking
• Performance Optimizations: Bulk operations and caching
• Health Monitoring: Built-in health check endpoints
• Background Tasks: Celery integration for maintenance tasks
• Data Validation: Automatic model structure validation
• Error Handling: Robust error handling and reporting

📋 Requirements

• Python >= 3.8
• Django >= 3.2, < 5.3 (supports Django 3.2, 4.0, 4.1, 4.2, 5.0, 5.1, 5.2)
• Django REST Framework >= 3.14.0
• PyJWT >= 2.6.0
• Django Simple History >= 3.4.0 (for audit trails)
• Celery (for background tasks)

🔧 Django Compatibility

This package is designed to work with a wide range of Django versions:

Django Version	Status	Support Level
3.2.x	✅ Supported	LTS (Long Term Support)
4.0.x	✅ Supported	Standard
4.1.x	✅ Supported	Standard
4.2.x	✅ Supported	LTS (Long Term Support)
5.0.x	✅ Supported	Standard
5.1.x	✅ Supported	Standard
5.2.x	✅ Supported	Standard
5.3.x	❌ Not yet	Future versions

Note: The package is tested against Django LTS versions and the latest stable releases. For production use, we recommend using Django LTS versions (3.2.x, 4.2.x) for maximum stability.

🛠️ Installation

1. Install the package:

pip install sb-sync

2. Add to your Django settings:

INSTALLED_APPS = [
    # ... other apps
    'sb_sync',
]

# Optional settings
SB_SYNC_LOG_DIR = 'logs'  # Directory for sync logs

3. Run migrations:

python manage.py migrate

4. Access the web configuration interface:

# Navigate to: http://your-domain/api/sync/config/
# Login with admin credentials (staff members only)

5. Setup audit trails (optional but recommended):

# Setup Django Simple History for audit trails
python manage.py setup_audit_trails --action setup

# Check audit trails status
python manage.py setup_audit_trails --action check

# Cleanup old audit trail records
python manage.py setup_audit_trails --action cleanup

🔧 Configuration

Required Settings

Add to your Django settings:

# JWT Settings
SECRET_KEY = 'your-secret-key'

# Celery Settings (for background tasks)
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'

# Optional: Custom log directory
SB_SYNC_LOG_DIR = 'logs'

URL Configuration

Include the sync URLs in your main urls.py:

from django.urls import path, include

urlpatterns = [
    # ... other URLs
    path('api/sync/', include('sb_sync.urls')),
]

🏢 Multi-Tenant Setup

1. Create Organizations

# Create organizations
python manage.py setup_organizations --action create_org --org-name "Acme Corporation" --org-slug acme-corp
python manage.py setup_organizations --action create_org --org-name "Global Retail" --org-slug global-retail
python manage.py setup_organizations --action create_org --org-name "City University" --org-slug city-university

2. Create Django Groups

# Create common groups for the application
python manage.py setup_organizations --action create_groups

3. Add Users to Organizations

# Add users with specific groups
python manage.py setup_organizations --action add_user --username john_manager --org-slug acme-corp --group-name Managers
python manage.py setup_organizations --action add_user --username mary_sales --org-slug global-retail --group-name Sales
python manage.py setup_organizations --action add_user --username bob_analyst --org-slug city-university --group-name Analysts

4. Setup Complete Example System

# Setup all example organizations with groups and permissions
python manage.py setup_organizations --action setup_example

5. Dynamic Permission Configuration

Using Command Line

# Discover all models in your project
python manage.py dynamic_permissions --action discover

# Discover models from specific app
python manage.py dynamic_permissions --action discover --app-label myapp

# Generate permission configuration
python manage.py dynamic_permissions --action generate --org-slug acme-corp --permission-template read_write --output-file permissions.json

# Apply permission configuration
python manage.py dynamic_permissions --action apply --org-slug acme-corp --config-file permissions.json

# Export current permissions
python manage.py dynamic_permissions --action export --org-slug acme-corp --output-file current_permissions.json

# Validate configuration file
python manage.py dynamic_permissions --action validate --config-file permissions.json

# Show available templates
python manage.py dynamic_permissions --action template

Using Web Interface (Recommended)

For easier management, use the web-based configuration interface:

1. Access the permission matrix: http://your-domain/api/sync/config/permissions/
2. Select your organization from the dropdown
3. Use checkboxes to grant/revoke permissions for each model and group
4. Changes are saved automatically via AJAX
5. Use bulk operations (Select All/Deselect All) for efficiency

The web interface provides a visual matrix showing all models vs permissions, making it much easier to manage complex permission scenarios.

👥 User Groups and Permissions

Using Django Groups

The system uses Django's built-in auth.Group system for role-based access control. This makes it completely generic and reusable across any Django project.

Available Groups

The system comes with common groups that can be customized for any domain:

1. Administrators - Full access to all data (create, read, update, delete)
2. Managers - Can push/pull data, create, update (no delete)
3. Users - Can push/pull data, create, update (no delete)
4. Analysts - Can only pull data (read-only access)
5. Sales - Can push/pull sales-related data
6. Support - Can push/pull support-related data
7. Read Only - Can only pull data, no push access

Permission Matrix

Group	User	Group	ContentType	Session	Delete
Administrators	✅ Full	✅ Full	✅ Full	✅ Full	✅ Yes
Managers	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	❌ No
Users	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	❌ No
Analysts	❌ Read	❌ Read	❌ Read	❌ Read	❌ No
Sales	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	❌ No
Support	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	✅ Push/Pull	❌ No
Read Only	✅ Pull	✅ Pull	✅ Pull	✅ Pull	❌ No

🌐 Web-Based Configuration Interface

The SB Sync package includes a comprehensive web-based configuration interface that allows you to manage model permissions, monitor sync operations, and configure model discovery through an intuitive web interface.

🚀 Interface Features

• 📊 Dashboard: Overview statistics and quick actions
• 🔐 Permission Matrix: Visual matrix interface for managing model permissions
• 🔍 Model Discovery: Configure which models are available for sync
• 📋 Sync Logs: View operation history and performance data
• 📈 Performance Metrics: Interactive charts and detailed metrics

🛠️ Accessing the Interface

1. Navigate to the configuration dashboard:

   http://your-domain/api/sync/config/
   

2. Login with admin credentials (staff members only)

3. Use the sidebar navigation to access different sections

🔐 Permission Matrix

The permission matrix provides a visual interface for managing model permissions:

• Visual Matrix: See all models vs permissions in a matrix format
• Checkbox Controls: Grant or revoke permissions with simple checkboxes
• Organization Selection: Switch between different organizations
• Real-time Updates: Changes are saved immediately via AJAX
• Bulk Operations: Select all/deselect all functionality

URL: /api/sync/config/permissions/

🔍 Model Discovery Configuration

Configure which models are discovered and available for sync:

• Auto Discovery Settings: Enable/disable automatic model discovery
• App Filtering: Include specific apps or all apps
• Model Exclusion: Exclude specific models from sync operations
• Live Preview: See which models are discovered in real-time

URL: /api/sync/config/model-discovery/

📋 Sync Logs

Monitor sync operations and performance:

• Operation History: View all sync operations with timestamps
• Performance Data: See processing times and record counts
• Export Functionality: Download logs as CSV
• Auto-refresh: Logs update automatically every 30 seconds

URL: /api/sync/config/logs/

📈 Performance Metrics

Track system performance and optimization:

• Performance Charts: Visual charts showing processing time trends
• Detailed Metrics: View batch sizes, memory usage, and query counts
• Performance Analysis: Automatic suggestions for optimization
• Export Data: Download performance data as CSV

URL: /api/sync/config/metrics/

📋 Audit Trails

Track all changes to sync system with comprehensive audit trails:

• Complete History: Track all changes to sync models (create, update, delete)
• User Attribution: See who made each change and when
• Field-Level Changes: View exactly what fields were modified
• Filtering & Search: Filter by model type, user, date range
• Export Capabilities: Download audit trail data as CSV
• Real-time Updates: Auto-refreshing audit trail display

URL: /api/sync/config/audit-trails/

🎨 Interface Features

• Modern Design: Bootstrap 5 with gradient backgrounds and smooth animations
• Responsive Design: Works on all devices (mobile, tablet, desktop)
• Real-time Updates: AJAX-powered with immediate feedback
• Security: Staff-only access with CSRF protection
• Performance: Cached data and optimized queries

For detailed documentation on the web interface, see WEB_CONFIG_README.md.

📡 API Endpoints

Authentication

POST /api/sync/auth/token/

Get JWT token for authentication:

{
    "username": "your_username",
    "password": "your_password"
}

Response (now includes organization info):

{
    "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9...",
    "user": {
        "id": 1,
        "username": "dr_smith",
        "email": "[e-mail]"
    },
    "organizations": [
        {
            "id": 1,
            "name": "Acme Corporation",
            "slug": "acme-corp",
            "group": "Managers"
        }
    ]
}

PUSH API

POST /api/sync/push/

Push data to Django models (with multi-tenant permissions):

{
    "data": [
        {
            "_model": "myapp.Customer",
            "name": "John Doe",
            "age": 45,
            "department": "SALES",
            "assigned_manager_id": 1
        }
    ]
}

Headers:

Authorization: Bearer <your_jwt_token>

Response:

{
    "status": "success",
    "success_count": 1,
    "error_count": 0,
    "processed_models": {
        "myapp.Customer": {
            "created": 1,
            "updated": 0
        }
    },
    "processing_time": 0.045
}

PULL API

POST /api/sync/pull/

Pull data from Django models (with role-based filtering):

{
    "models": {
        "myapp.Customer": "2024-01-14T10:00:00Z",
        "myapp.Order": "2024-01-14T10:00:00Z"
    },
    "batch_size": 100
}

Headers:

Authorization: Bearer <your_jwt_token>

Response:

{
    "data": [
        {
            "_model": "myapp.Customer",
            "id": 1,
            "name": "John Doe",
            "age": 45,
            "department": "SALES",
            "assigned_manager_id": 1,
            "organization": 1
        }
    ],
    "metadata": {
        "myapp.Customer": {
            "count": 1,
            "last_sync": "2024-01-15T10:30:00Z",
            "user_last_sync": "2024-01-14T10:00:00Z"
        }
    },
    "batch_info": {
        "batch_size": 100,
        "total_records": 1
    }
}

Performance Monitoring

GET /api/sync/performance/

Get performance statistics:

{
    "performance_stats": {
        "total_operations": 150,
        "average_processing_time": 0.045,
        "cache_hit_rate": 0.85
    },
    "current_memory_usage": 245.6,
    "cache_stats": {
        "cache_hits": 1200,
        "cache_misses": 200
    },
    "optimization_suggestions": [
        "High memory usage detected. Consider reducing batch sizes."
    ]
}

Health Check

GET /api/sync/health/

Check system health:

{
    "status": "healthy",
    "timestamp": "2024-01-01T12:00:00Z",
    "checks": {
        "database": "healthy",
        "cache": "healthy",
        "logging": "healthy"
    }
}

🗄️ Database Models

Core Models

SyncLog

Tracks all synchronization operations:

class SyncLog(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE, db_index=True)
    operation = models.CharField(max_length=10, choices=[('PUSH', 'Push'), ('PULL', 'Pull')], db_index=True)
    status = models.CharField(max_length=10, choices=[('SUCCESS', 'Success'), ('ERROR', 'Error'), ('WARNING', 'Warning')], db_index=True)
    model_name = models.CharField(max_length=100, blank=True, db_index=True)
    object_count = models.IntegerField(default=0)
    error_message = models.TextField(blank=True)
    request_data = models.JSONField(blank=True, null=True)
    timestamp = models.DateTimeField(default=timezone.now, db_index=True)
    processing_time = models.FloatField(default=0.0)

SyncMetadata

Tracks last sync timestamps for models:

class SyncMetadata(models.Model):
    model_name = models.CharField(max_length=100, unique=True, db_index=True)
    last_sync = models.DateTimeField(default=timezone.now, db_index=True)
    total_synced = models.BigIntegerField(default=0)

PerformanceMetrics

Tracks performance metrics for optimization:

class PerformanceMetrics(models.Model):
    operation_type = models.CharField(max_length=20, db_index=True)
    model_name = models.CharField(max_length=100, db_index=True)
    batch_size = models.IntegerField()
    processing_time = models.FloatField()
    memory_usage = models.FloatField(null=True, blank=True)
    query_count = models.IntegerField()
    timestamp = models.DateTimeField(default=timezone.now, db_index=True)

Multi-Tenant Models

Organization

Represents organizations, companies, or institutions:

class Organization(models.Model):
    name = models.CharField(max_length=200, unique=True)
    slug = models.CharField(max_length=50, unique=True, db_index=True)
    description = models.TextField(blank=True)
    is_active = models.BooleanField(default=True)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

UserOrganization

Links users to organizations with roles:

class UserOrganization(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE, db_index=True)
    organization = models.ForeignKey(Organization, on_delete=models.CASCADE, db_index=True)
    role = models.CharField(max_length=50, choices=[
        ('ADMIN', 'Administrator'),
        ('MANAGER', 'Manager'),
        ('USER', 'User'),
        ('ANALYST', 'Analyst'),
        ('SALES', 'Sales'),
        ('SUPPORT', 'Support'),
        ('READ_ONLY', 'Read Only'),
    ], db_index=True)
    is_active = models.BooleanField(default=True)
    created_at = models.DateTimeField(auto_now_add=True)

ModelPermission

Defines which models each role can access:

class ModelPermission(models.Model):
    organization = models.ForeignKey(Organization, on_delete=models.CASCADE, db_index=True)
    role = models.CharField(max_length=50, db_index=True)
    model_name = models.CharField(max_length=100, db_index=True)
    can_push = models.BooleanField(default=False)
    can_pull = models.BooleanField(default=False)
    can_create = models.BooleanField(default=False)
    can_update = models.BooleanField(default=False)
    can_delete = models.BooleanField(default=False)
    can_read = models.BooleanField(default=True)
    filters = models.JSONField(blank=True, null=True)

UserSyncMetadata

Tracks last sync timestamps for models per user/organization:

class UserSyncMetadata(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE, db_index=True)
    organization = models.ForeignKey(Organization, on_delete=models.CASCADE, db_index=True)
    model_name = models.CharField(max_length=100, db_index=True)
    last_sync = models.DateTimeField(default=timezone.now, db_index=True)
    total_synced = models.BigIntegerField(default=0)

DataFilter

Custom data filters for role-based access:

class DataFilter(models.Model):
    organization = models.ForeignKey(Organization, on_delete=models.CASCADE, db_index=True)
    role = models.CharField(max_length=50, db_index=True)
    model_name = models.CharField(max_length=100, db_index=True)
    filter_name = models.CharField(max_length=100)
    filter_condition = models.JSONField()  # e.g., {"field": "department", "operator": "exact", "value": "CARDIOLOGY"}
    is_active = models.BooleanField(default=True)
    created_at = models.DateTimeField(auto_now_add=True)

🔄 Background Tasks

Celery Configuration

Add to your project's __init__.py:

from .celery import app as celery_app
__all__ = ('celery_app',)

Available Tasks

1. cleanup_old_sync_logs: Removes sync logs older than 90 days
2. generate_sync_report: Generates daily sync statistics
3. optimize_database_tables: Analyzes and optimizes database tables
4. bulk_sync_operation: Processes bulk sync operations asynchronously
5. cache_warmup: Warms up cache with frequently accessed data
6. memory_optimization: Performs garbage collection and cache cleanup
7. performance_analysis: Analyzes performance and provides recommendations

Running Celery

# Start Celery worker
celery -A your_project worker -l info

# Start Celery beat (for scheduled tasks)
celery -A your_project beat -l info

🛡️ Security

• JWT Authentication: All API endpoints require valid JWT tokens
• Token Expiration: Tokens expire after 7 days by default
• Multi-Tenant Isolation: Complete data isolation between organizations
• Role-Based Access Control: Granular permissions per user role
• Data Filtering: Role-specific data access filters
• User Validation: All operations are tied to authenticated users
• Input Validation: Comprehensive data validation against Django models

📊 Monitoring & Logging

Log Files

Sync operations are logged to logs/sb_sync.log with daily rotation and 30-day retention.

Log Format

2024-01-01 12:00:00 - sb_sync - INFO - PUSH request from user john_manager in Acme Corporation: {"data": [...]}

Health Monitoring

Use the health check endpoint to monitor:

• Database connectivity
• Cache functionality
• Log directory accessibility
• Memory usage
• Performance metrics

⚡ Performance Optimizations

Bulk Operations

The package includes optimized bulk operations for high-volume data processing:

from sb_sync.optimizations import BulkOperations

# Bulk create/update with batching
results = BulkOperations.bulk_create_or_update(
    model_class=YourModel,
    data_list=large_dataset,
    batch_size=1000
)

Model Metadata Caching

Model field information is cached for improved performance:

from sb_sync.optimizations import ModelMetadataCache

# Get cached model fields
fields = ModelMetadataCache.get_model_fields('app.ModelName')

Query Optimization

from sb_sync.optimizations import QueryOptimizer

# Get optimized sync logs
logs = QueryOptimizer.get_optimized_sync_logs(user, organization)

# Count queries with decorator
@QueryOptimizer.count_queries
def my_function():
    # Your code here
    pass

Memory Optimization

from sb_sync.optimizations import MemoryOptimizer

# Monitor memory usage
memory_usage = MemoryOptimizer.get_memory_usage()

# Memory monitoring decorator
@MemoryOptimizer.monitor_memory
def my_function():
    # Your code here
    pass

Cache Optimization

from sb_sync.optimizations import CacheOptimizer

# Get or set cache
data = CacheOptimizer.get_or_set_cache('key', lambda: expensive_operation())

# Cache model data
CacheOptimizer.cache_model_data('key', data, timeout=300)

🔧 Management Commands

Model Discovery and Default Models

The system automatically discovers all Django models in your application and makes them available for push/pull operations by default. Only specific apps and models are excluded via configuration.

# Show model discovery summary
python manage.py show_models --action summary

# Show all discovered models
python manage.py show_models --action all

# Show enabled models only
python manage.py show_models --action enabled

# Show default models for push/pull operations
python manage.py show_models --action default

# Show detailed model information
python manage.py show_models --action details --verbose

# Show models from specific app
python manage.py show_models --action enabled --app-label myapp

Default Behavior:

• ✅ Auto-discovers all Django models in your application
• ✅ INCLUDE_APPS: List of apps whose models will be synced (empty = all apps)
• ✅ EXCLUDE_MODELS: Models within those included apps that will be excluded
• ✅ Makes all discovered models available for push/pull operations
• ✅ No configuration required - works out of the box

Configuration Options:

• AUTO_DISCOVER_MODELS: Enable/disable automatic model discovery
• INCLUDE_APPS: List of apps whose models will be synced (empty = all apps)
• EXCLUDE_MODELS: Models within included apps that will be excluded from sync
• INCLUDE_CUSTOM_MODELS: Include custom models from your apps

Cleanup Sync Logs

python manage.py cleanup_sync_logs

Performance Optimization

# Analyze performance
python manage.py optimize_performance --action analyze --days 7

# Optimize performance
python manage.py optimize_performance --action optimize --force

# Cleanup old data
python manage.py optimize_performance --action cleanup

# Monitor resources
python manage.py optimize_performance --action monitor

Configuration Management

# Show current configuration
python manage.py manage_config --action show

# Export configuration
python manage.py manage_config --action export --file config.json --format json

# Import configuration
python manage.py manage_config --action import --file config.json --format json

# Validate configuration
python manage.py manage_config --action validate

# Get configuration summary
python manage.py manage_config --action summary

# Reset to defaults
python manage.py manage_config --action reset --force

Organization Setup

# Create organization
python manage.py setup_organizations --action create_org --org-name "Acme Corporation" --org-slug acme-corp

# Add user to organization
python manage.py setup_organizations --action add_user --username john_manager --org-slug acme-corp --group-name Managers

# Set permissions from config file
python manage.py setup_organizations --action set_permissions --org-slug acme-corp --config-file permissions.json

# Setup complete example system
python manage.py setup_organizations --action setup_example

Audit Trails Management

# Setup audit trails for all sync models
python manage.py setup_audit_trails --action setup

# Check audit trails status
python manage.py setup_audit_trails --action check

# Cleanup old audit trail records
python manage.py setup_audit_trails --action cleanup

# Setup audit trails for specific model
python manage.py setup_audit_trails --action setup --model sb_sync.Organization

# Check specific model audit trails
python manage.py setup_audit_trails --action check --model sb_sync.ModelPermission

Web-Based Configuration Interface

For easier management, use the web interface instead of command-line tools:

# Access the web interface
http://your-domain/api/sync/config/

Available Web Interfaces:

• Dashboard: /api/sync/config/ - Overview and quick actions
• Permission Matrix: /api/sync/config/permissions/ - Visual permission management
• Model Discovery: /api/sync/config/model-discovery/ - Configure model discovery
• Sync Logs: /api/sync/config/logs/ - View operation history
• Performance Metrics: /api/sync/config/metrics/ - Monitor performance
• Audit Trails: /api/sync/config/audit-trails/ - Track all changes

Benefits of Web Interface:

• ✅ Visual Management: Checkbox-based permissions instead of JSON files
• ✅ Real-time Updates: Changes saved immediately via AJAX
• ✅ Bulk Operations: Select all/deselect all functionality
• ✅ Live Monitoring: Auto-refreshing logs and metrics
• ✅ Export Capabilities: Download data as CSV
• ✅ No Configuration Files: Everything managed through web interface
• ✅ Audit Trails: Complete change history tracking

Dynamic Permission Configuration

# Discover all models in your project
python manage.py dynamic_permissions --action discover

# Generate permission configuration
python manage.py dynamic_permissions --action generate --org-slug acme-corp --permission-template read_write

# Apply permission configuration
python manage.py dynamic_permissions --action apply --org-slug acme-corp --config-file permissions.json

# Export current permissions
python manage.py dynamic_permissions --action export --org-slug acme-corp --output-file current_permissions.json

# Validate configuration file
python manage.py dynamic_permissions --action validate --config-file permissions.json

# Show available templates
python manage.py dynamic_permissions --action template

Model Discovery and Default Models

# Show model discovery summary
python manage.py show_models --action summary

# Show all discovered models
python manage.py show_models --action all

# Show enabled models only
python manage.py show_models --action enabled

# Show default models for push/pull
python manage.py show_models --action default

# Show detailed model information
python manage.py show_models --action details --verbose

# Show models from specific app
python manage.py show_models --action enabled --app-label myapp

🧪 Testing

Example Usage

import requests
import json

# Get authentication token
auth_response = requests.post('http://localhost:8000/api/sync/auth/token/', {
    'username': 'dr_smith',
    'password': 'password'
})
token = auth_response.json()['token']

# Push data (with multi-tenant permissions)
headers = {'Authorization': f'Bearer {token}'}
push_data = {
    'data': [
        {
            '_model': 'myapp.Customer',
            'name': 'John Doe',
            'department': 'SALES',
            'assigned_manager_id': 1
        }
    ]
}
response = requests.post('http://localhost:8000/api/sync/push/', 
                        json=push_data, headers=headers)
print(response.json())

# Pull data (with role-based filtering)
pull_data = {
    'models': {
        'myapp.Customer': '2024-01-14T10:00:00Z',
        'myapp.Order': '2024-01-14T10:00:00Z'
    },
    'batch_size': 100
}
response = requests.post('http://localhost:8000/api/sync/pull/', 
                        json=pull_data, headers=headers)
print(response.json())

📝 Error Handling

The package provides comprehensive error handling:

• Validation Errors: Detailed field validation messages
• Model Errors: Clear error messages for missing or invalid models
• Authentication Errors: Proper JWT token validation
• Permission Errors: Multi-tenant and role-based access control errors
• Database Errors: Transaction rollback on errors
• Partial Success Handling: Graceful handling of partial failures

🏢 Multi-Tenant Use Case

Scenario: Multiple Organizations

The system supports complex multi-tenant scenarios with multiple organizations:

# Organization A (Acme Corp) - Manager Smith
POST /api/sync/push/
{
    "data": [
        {
            "_model": "myapp.Customer",
            "name": "John Doe",
            "department": "SALES",
            "assigned_manager_id": 1
        }
    ]
}
# ✅ Success - Manager Smith has permission

# Organization B (Global Retail) - Sales Wilson  
POST /api/sync/pull/
{
    "models": {"myapp.Customer": "2024-01-14T10:00:00Z"}
}
# ✅ Success - Sales Wilson gets only their assigned customers

# Organization C (City University) - Analyst Garcia
POST /api/sync/push/
{
    "data": [
        {
            "_model": "myapp.Report",
            "customer_id": 5,
            "report_type": "ANALYSIS",
            "results": "Positive"
        }
    ]
}
# ✅ Success - Analyst has permission for reports

Key Benefits for Multi-Tenant Applications:

1. 🔒 Data Isolation: Each organization only sees their own data
2. 👥 Role-Based Access: Different roles have appropriate permissions
3. 📊 Granular Control: Filter by department, assigned staff, etc.
4. 🔄 Per-User Sync Tracking: Each user has their own sync history
5. ⚡ Performance: Optimized for high-volume data processing
6. 🛡️ Security: Meets enterprise data privacy requirements

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Submit a pull request

📄 License

This project is licensed under the MIT License.

🆘 Support

For issues and questions:

• Check the health endpoint: /api/sync/health/
• Review sync logs in logs/sb_sync.log
• Monitor Celery task logs for background operations
• Check performance metrics: /api/sync/performance/

📋 Changelog

v1.6.1 (Latest) - 2024-01-XX

🔧 Template Tag Fix & Bug Resolution

• Fixed: AttributeError in permission matrix template ('bool' object has no attribute 'get')
• Enhanced: lookup filter in sb_sync_extras.py to handle multiple data types
• Added: Support for boolean values, dictionary-like objects, lists, and object attributes
• Improved: Error handling for template tag operations
• Tested: Permission matrix endpoint accessibility and functionality

v1.6.0 - 2024-01-XX

🔧 Model Discovery Logic Enhancement

• Fixed: Model discovery logic to exclude Django built-in apps when INCLUDE_APPS is empty
• Fixed: Excluded sb-sync app itself and its dependencies from model discovery
• Enhanced: get_all_models() and is_model_enabled() methods with proper exclusion logic
• Added: Comprehensive list of excluded apps (Django built-ins, sb-sync, dependencies)
• Fixed: Missing dependencies (psutil, django-simple-history) installation issues
• Updated: URL configuration to properly expose model discovery endpoint
• Tested: Model discovery logic with custom test apps
• Improved: Server startup reliability and dependency management

v1.5.3 - 2024-01-XX

🔧 Package Installation Fixes

• Fixed: Resolved pip installation issues with proper package structure
• Fixed: Moved all files to sb_sync/ directory for correct Python package layout
• Fixed: Cleaned up excessive dependencies in setup.py
• Fixed: Removed invalid entry points that caused installation errors
• Added: MANIFEST.in file for proper package data inclusion
• Updated: Project URLs to point to correct GitHub repository
• Tested: Package installation and import functionality

v1.5.2 - 2024-01-XX

🔧 Django 5.2.x Compatibility & Installation Fixes

• Added: Support for Django 5.2.x
• Updated: Django version constraint from <5.1 to <5.3
• Added: Django 5.1 and 5.2 framework classifiers
• Fixed: Package structure for proper pip installation
• Added: Comprehensive Django compatibility documentation

v1.5.1 - 2024-01-XX

🔧 Django Compatibility Enhancement

• Added: Support for Django 5.1.x and 5.2.x
• Updated: Django version requirements to support broader range
• Added: Django compatibility table in documentation
• Fixed: Version constraints in setup.py

v1.5.0 - 2024-01-XX

🌐 Web-Based Configuration Interface & Audit Trails

• Added: Complete web-based configuration interface with dashboard
• Added: Visual permission matrix for model permissions management
• Added: Model discovery configuration UI
• Added: Sync logs viewer with real-time updates
• Added: Performance metrics visualization with charts
• Added: Django Simple History integration for comprehensive audit trails
• Added: Audit trails management command (setup_audit_trails)
• Added: Custom template tags for enhanced UI functionality
• Enhanced: Admin interface with historical records support
• Updated: All sync models with audit trail capabilities
• Added: Bootstrap 5, Font Awesome, and Chart.js for modern UI

v1.4.0 - 2024-01-XX

🧹 Package Streamlining & Code Organization

• Removed: Dynamic data sources functionality for focused scope
• Improved: Code organization and structure
• Enhanced: Documentation clarity and completeness
• Streamlined: Package for Django model synchronization focus
• Cleaned: Codebase organization and removed unused components

v1.3.0 - 2024-01-XX

🔍 Model Discovery & Configuration Enhancement

• Added: Include-based model discovery system
• Enhanced: Configuration system with better flexibility
• Improved: Test coverage across all components
• Updated: Comprehensive documentation cleanup
• Added: Model introspection capabilities
• Enhanced: Error handling and validation

v1.2.0 - 2024-01-XX

🚀 Performance & Multi-Tenant Features

• Added: Multi-tenant support with organization-based isolation
• Added: Role-based access control (RBAC)
• Enhanced: Performance optimizations with bulk operations
• Added: Comprehensive health monitoring
• Improved: Background task processing with Celery
• Added: Advanced caching mechanisms

v1.1.0 - 2024-01-XX

⚡ Performance & Error Handling

• Enhanced: Configuration system flexibility
• Added: Performance optimizations for large datasets
• Improved: Error handling and reporting mechanisms
• Added: Advanced logging capabilities
• Enhanced: Data validation processes

v1.0.0 - 2024-01-XX

🎉 Initial Release

• Added: PUSH/PULL API endpoints for bidirectional sync
• Added: JWT authentication system
• Added: Comprehensive logging infrastructure
• Added: Basic model synchronization capabilities
• Added: Health check endpoints
• Added: Initial documentation and setup guides

---

🔄 Migration Guide

Upgrading to v1.5.x

If you're upgrading from an earlier version:

1. Update your installation:

   pip install --upgrade sb-sync
   

2. Run new migrations:

   python manage.py migrate
   

3. Setup audit trails (recommended):

   python manage.py setup_audit_trails --action setup
   

4. Access new web interface:

   • Navigate to: http://your-domain/api/sync/config/
   • Login with admin/staff credentials

Breaking Changes

• v1.4.0: Removed dynamic data sources - migrate to model-based sync
• v1.5.0: Added new dependencies (django-simple-history) - run pip install --upgrade sb-sync

New Features Guide

• Web Interface: Access at /api/sync/config/ for visual management
• Audit Trails: Use management command to setup and manage historical records
• Django 5.2.x: Full compatibility with latest Django versions