Skip to main content

half_orm development Framework.

Project description

half-orm-dev

WARNING! half-orm-dev is in alpha development phase!

Git-centric patch management and database versioning for half-orm projects

Python 3.8+ License: GPLv3 half-orm PyPI Downloads

Modern development workflow for PostgreSQL databases with automatic code generation, semantic versioning, and production-ready deployment system.


๐Ÿ“– Description

half-orm-dev provides a complete development lifecycle for database-driven applications:

  • Git-centric workflow: Patches stored in Git branches and release files
  • Semantic versioning: Automatic version calculation (patch/minor/major)
  • Code generation: Python classes auto-generated from schema changes
  • Safe deployments: Automatic backups, rollback support, validation
  • Team collaboration: Distributed locks, branch notifications, conflict prevention
  • Test-driven development: Systematic validation before any integration

Perfect for teams managing evolving PostgreSQL schemas with Python applications.

โœจ Features

๐Ÿ”ง Development

  • Patch-based development: Isolated branches for each database change
  • Automatic code generation: half-orm Python classes created from schema
  • Complete testing: Apply patches with full release context
  • Conflict detection: Distributed locks prevent concurrent modifications

๐Ÿงช Test-Driven Development & Validation

Systematic Testing Before Integration

half-orm-dev enforces a test-first approach that guarantees code quality:

1. Validation on Temporary Branches

# When closing a patch, tests run FIRST
# You must be on the patch branch
git checkout ho-patch/456-user-auth
half_orm dev patch merge

# What happens behind the scenes:
# 1. Creates temp-valid-1.3.6 branch
# 2. Merges ALL release patches + new patch
# 3. Runs pytest tests/
# 4. If merge and tests PASS โ†’ merges patch branch into release branch; changes patch status to "staged" in TOML and commits
# 5. If anything FAILS โ†’ nothing committed (temp branch is deleted)

2. No Integration Without Tests

  • โŒ BLOCKED: Patches cannot be added to releases if anything fails
  • โœ… SAFE: Only validated code reaches stage/rc/production
  • ๐Ÿ”’ GUARANTEED: Every release is testable before deployment

3. Business Logic Testing (TDD Best Practice)

# Your business logic is fully testable
# Example: tests/test_user_authentication.py

def test_user_creation():
    """Test user creation through half-orm models."""
    user = User(
        username='john',
        email='john@example.com'
    ).ho_insert()

    assert user.id is not None
    assert user.username == 'john'

def test_invalid_email_rejected():
    """Test validation prevents invalid emails."""
    with pytest.raises(ValidationError):
        User(username='john', email='invalid').ho_insert()

4. Full Release Context Testing

# Test your patch with ALL previous patches
half_orm dev patch apply

# What happens:
# 1. Restores DB to production state
# 2. Applies all RC patches (if any)
# 3. Applies all stage patches
# 4. Applies YOUR patch in correct order
# 5. Generates code
# โ†’ Your tests run in realistic production-like environment

5. Workflow Integration

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Development Cycle with Test Validation                            โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 1. Create patch                                                   โ”‚
โ”‚ 2. Write tests FIRST (TDD)                                        โ”‚
โ”‚ 3. Implement feature                                              โ”‚
โ”‚ 4. Run tests locally: pytest                                      โ”‚
โ”‚ 5. Add to release โ†’ AUTOMATIC VALIDATION                          โ”‚
โ”‚    โ”œโ”€ temp-valid branch created                                   โ”‚
โ”‚    | โ”œโ”€ All patches merged                                        โ”‚
โ”‚    | โ””โ”€ pytest runs automatically                                 โ”‚
โ”‚    โ””โ”€ Only commits if everything is OK                            โ”‚
โ”‚ 6. Promote to RC โ†’ Tests validated again, code merged on ho-prod  โ”‚
โ”‚ 7. Deploy to prod โ†’ Tested code only                              โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Benefits:

  • โœ… Catch Integration Issues Early: Test interactions between patches
  • โœ… Prevent Regressions: Existing tests protect against breaking changes
  • โœ… Document Behavior: Tests serve as executable specifications
  • โœ… Safe Refactoring: Change implementation with confidence
  • โœ… Team Collaboration: Clear expectations for code quality

๐Ÿ“ฆ Release Management

  • Semantic versioning: patch/minor/major increments
  • Release candidates: RC validation before production
  • Sequential promotion: stage โ†’ rc โ†’ production workflow
  • Branch cleanup: Automatic deletion after RC promotion
  • Test validation: Automated testing at every promotion step

๐Ÿš€ Production

  • Safe upgrades: Automatic database backups before changes
  • Incremental deployment: Apply releases sequentially
  • Dry-run mode: Preview changes before applying
  • Version tracking: Complete release history in database
  • Rollback support: Automatic rollback on failures

๐Ÿ‘ฅ Team Collaboration

  • Distributed locks: Prevent concurrent ho-prod modifications
  • Branch notifications: Alert developers when rebase needed
  • Multiple stages: Parallel development of different releases
  • Git-based coordination: No external tools required

๐Ÿš€ Installation

Prerequisites

  • Python 3.8+
  • PostgreSQL 12+
  • Git
  • half-orm (pip install half-orm)

Install

pip install half-orm-dev

Verify Installation

half_orm dev --help

๐Ÿ“– Quick Start

Initialize New Project

# Create project with database
half_orm dev init myproject --database mydb

# Navigate to project
cd myproject

Clone Existing Project

# Clone from Git
half_orm dev clone https://github.com/user/project.git

# Navigate to project
cd project

First Patch (TDD Development)

# First, create a release integration branch
half_orm dev release new minor  # Creates ho-release/0.1.0

# Now create patch (automatically added to candidates)
half_orm dev patch create 1-users
# โ†’ Auto-added to 0.1.0-patches.toml as "candidate"

# Add schema changes
echo "CREATE TABLE users (id uuid PRIMARY KEY DEFAULT gen_random_uuid(), username TEXT NOT NULL);" > Patches/1-users/01_users.sql

# Apply patch - this generates Python code AND test directory structure
half_orm dev patch apply
# โ†’ Restores database
# โ†’ Applies SQL patches
# โ†’ Generates Python classes (mydb/public/user.py)
# โ†’ Creates test directory structure (tests/public/user/)

# Now write business logic tests (TDD approach)
# IMPORTANT: Test business logic, NOT ORM operations like ho_insert()
cat > tests/public/user/test_user_business_logic.py << 'EOF'
from mydb.public.user import User

def test_user_creation_with_valid_data():
    """Test creating a user with valid business logic."""
    # Assuming you implement a create() business method
    user = User.create(username='alice')
    assert user['username'] == 'alice'
    assert user['id'] is not None

def test_user_creation_rejects_empty_username():
    """Test business validation."""
    # Should raise an error if you implement validation
    with pytest.raises(ValueError):
        User.create(username='')
EOF

# Implement business logic in your User class
cat >> mydb/public/user.py << 'EOF'

@classmethod
def create(cls, username: str):
    """Business logic for creating a user."""
    if not username or not username.strip():
        raise ValueError("Username cannot be empty")
    return cls(username=username).ho_insert()
EOF

# Run tests
pytest

# Commit your work
git add .
git commit -m "Add users table with business logic and tests"

# Close patch - integrate to release (automatic validation runs here!)
# You must be on the patch branch
git checkout ho-patch/1-users
half_orm dev patch merge
# โ†’ Status changed to "staged" in TOML
# โ†’ Tests validated automatically

๐Ÿ’ป Development Workflow

Vision: Git-Flow Release Management

The workflow follows a Git-Flow approach with dedicated integration branches (ho-release/X.Y.Z) that serve as test sandboxes before production.

Motivation:

  • Patches are visible and testable on ho-release/X.Y.Z before production
  • ho-prod remains stable and contains only validated versions (RC or production)
  • Workflow compatible with GitLab/GitHub (milestones, merge requests, issues)
  • No need to create RC just to make a patch accessible

Complete Cycle: Patch โ†’ Release โ†’ Deploy

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ DEVELOPMENT (ho-release/X.Y.Z branch)                           โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 1. release new <level>     Create ho-release/X.Y.Z              โ”‚
โ”‚ 2. patch create <id>          Create patch (auto in candidates)    โ”‚
โ”‚ 3. patch apply             Apply & test changes                 โ”‚
โ”‚ 4. patch merge             Merge into ho-release (TESTS!)       โ”‚
โ”‚                            (run from ho-patch/<id> branch)      โ”‚
โ”‚                                                                 โ”‚
โ”‚ RELEASE PROMOTION                                               โ”‚
โ”‚ 5. release promote rc      Create RC (tags ho-release branch)   โ”‚
โ”‚ 6. release promote prod    Merge to ho-prod + deploy            โ”‚
โ”‚                                                                 โ”‚
โ”‚ PRODUCTION DEPLOYMENT                                           โ”‚
โ”‚ 7. update                  Check available releases             โ”‚
โ”‚ 8. upgrade                 Apply on production servers          โ”‚
โ”‚                                                                 โ”‚
โ”‚ HOTFIX WORKFLOW (urgent fixes)                                  โ”‚
โ”‚ 9. release hotfix          Reopen production version            โ”‚
โ”‚ 10. patch create/close        Same workflow on hotfix branch       โ”‚
โ”‚ 11. release promote hotfix Deploy as vX.Y.Z-hotfixN             โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Workflow Details

Concepts: Release Files and Patch States

Release Files:

.hop/releases/
โ”œโ”€โ”€ 0.17.0-patches.toml     # Patches in development (TOML format with status)
โ”œโ”€โ”€ 0.17.0-rc1.txt          # First Release Candidate (snapshot)
โ”œโ”€โ”€ 0.17.0-rc2.txt          # Second RC (with fixes, snapshot)
โ”œโ”€โ”€ 0.17.0.txt              # Production version (snapshot)
โ”œโ”€โ”€ 0.17.0-hotfix1.txt      # Urgent production fix (snapshot)
โ””โ”€โ”€ 0.18.0-patches.toml     # Next release in progress

TOML Patches File Format (0.17.0-patches.toml):

[patches]
"1-auth" = "candidate"    # In development
"2-api" = "candidate"     # In development
"3-ui" = "staged"         # Integrated, awaiting RC
"4-tests" = "staged"      # Integrated, awaiting RC

Patch States:

  1. Candidate: Assigned to release, in development ("candidate" in TOML)
  2. Staged: Integrated in ho-release/X.Y.Z, awaiting promotion ("staged" in TOML)
  3. Released: Included in deployed production version (in X.Y.Z.txt snapshot)

Analogy with GitLab/GitHub:

half-orm state File GitLab/GitHub
release new Creates -patches.toml Create milestone
patch create (on ho-release) Adds to TOML as "candidate" Create issue assigned to milestone
Candidate "patch-id" = "candidate" in TOML Open issue assigned to milestone
patch merge Changes to "staged" in TOML Merge MR and close issue
Stage "patch-id" = "staged" in TOML Closed issue in milestone
release promote rc Creates -rcN.txt snapshot Create pre-release
RC -rcN.txt GitHub pre-release
release promote prod Creates X.Y.Z.txt, deletes TOML Create stable release
Production X.Y.Z.txt Stable release

Step 1: Create a New Release

half_orm dev release new minor
# โ†’ Detects current production version (e.g., 0.16.0)
# โ†’ Calculates next minor version: 0.17.0
# โ†’ Creates branch ho-release/0.17.0 from ho-prod
# โ†’ Creates .hop/releases/0.17.0-patches.toml (empty TOML file)
# โ†’ Commits and pushes to reserve version globally
# โ†’ Automatically switches to ho-release/0.17.0

Output:

โœ… Release created successfully!

  Version:          0.17.0
  Release branch:   ho-release/0.17.0
  Patches file:     .hop/releases/0.17.0-patches.toml

๐Ÿ“ Next steps:
  1. Create patches: half_orm dev patch create <patch_id>
  2. Close patches: git checkout ho-patch/<patch_id> && half_orm dev patch merge
  3. Promote to RC: half_orm dev release promote rc

โ„น๏ธ  Patches will be merged into ho-release/0.17.0 for integration testing

Step 2: Create a Candidate Patch

Prerequisites: Must be on ho-release/0.17.0 branch

git checkout ho-release/0.17.0
half_orm dev patch create 6-feature-x
# โ†’ Auto-detects version 0.17.0 from current branch
# โ†’ Creates ho-patch/6-feature-x from ho-release/0.17.0
# โ†’ Adds 6-feature-x to 0.17.0-patches.toml as "candidate"
# โ†’ Switches to ho-patch/6-feature-x

Output:

โœ“ Created patch branch: ho-patch/6-feature-x
โœ“ Created patch directory: Patches/6-feature-x/
โœ“ Added to candidates: .hop/releases/0.17.0-patches.toml
โœ“ Switched to branch: ho-patch/6-feature-x

๐Ÿ“ Next steps:
  1. Add SQL/Python files to Patches/6-feature-x/
  2. Run: half_orm dev patch apply
  3. Test your changes
  4. Run: half_orm dev patch merge (already on ho-patch/6-feature-x)

Step 3: Develop and Test (TDD Approach)

# Apply patch (on ho-patch/* branch)
half_orm dev patch apply
# โ†’ Restores database from production state
# โ†’ Applies all release patches + current patch
# โ†’ Generates Python code
# โ†’ Ready for testing

# FIRST: Write tests
cat > tests/public/users/test_users_feature.py << 'EOF'
def test_feature():
    # Your test here
    assert True
EOF

# Run tests
pytest

# Commit your work
git add .
git commit -m "Implement feature with tests"

Step 4: Close Patch (Integrate to Release)

# Ensure you're on the patch branch
git checkout ho-patch/6-feature-x

half_orm dev patch merge
# Complete workflow:
# โ†’ Detects version from 0.17.0-patches.toml
# โ†’ Validates ho-patch/6-feature-x exists
# โ†’ Creates temporary validation branch
# โ†’ Merges ho-patch/6-feature-x into temp branch
# โ†’ Restores database and applies all patches
# โ†’ Runs tests (pytest)
# โ†’ If PASS: Merges into ho-release/0.17.0
# โ†’ Changes 6-feature-x from "candidate" to "staged" in TOML
# โ†’ Deletes branch ho-patch/6-feature-x
# โ†’ Commits and pushes changes
# โ†’ Notifies other candidate patches to sync

Output:

โœ“ Patch closed successfully!

  Patches file:    .hop/releases/0.17.0-patches.toml
  Patch status:    6-feature-x โ†’ "staged"
  Tests passed:    โœ“
  Notified:        2 active branch(es)

๐Ÿ“ Next steps:
  โ€ข Other developers: git pull && git merge ho-release/0.17.0
  โ€ข Continue development: half_orm dev patch create <next_patch_id>
  โ€ข Promote to RC: half_orm dev release promote rc

Important: patch merge replaces the old patch add command. The semantics are different:

  • OLD: patch add = "I add my validated patch to release" (from ho-prod)
  • NEW: patch merge = "I close my work, it's integrated in release" (merge into ho-release)

Step 5: Synchronize with Other Integrated Patches

When another patch is integrated in the release, candidate patches must update:

git fetch origin
git merge origin/ho-release/0.17.0

Step 6: Promote to RC

Sequentiality Rule: Only the smallest version in preparation can be promoted to RC. This guarantees sequential release order.

Example: If releases 0.17.1, 0.18.0 and 1.0.0 are in preparation, only 0.17.1 can be promoted to RC.

half_orm dev release promote rc
# Complete workflow:
# โ†’ Auto-detects smallest version with -patches.toml containing staged patches
# โ†’ Verifies it's sequential (follows last prod/RC)
# โ†’ Automatically switches to ho-release/X.Y.Z
# โ†’ Finds next RC number (rc1, rc2, etc.)
# โ†’ Creates tag vX.Y.Z-rc1 on ho-release/X.Y.Z (NOT on ho-prod!)
# โ†’ Creates snapshot .hop/releases/X.Y.Z-rc1.txt with staged patches
# โ†’ Resets staged patches to candidates in TOML for potential fixes
# โ†’ Commits and pushes

Output:

โœ“ Success!

  Version:  0.17.0
  Tag:      v0.17.0-rc1
  Branch:   ho-release/0.17.0

๐Ÿ“ Next steps:
  โ€ข Test RC thoroughly
  โ€ข Deploy to production: half_orm dev release promote prod

Important Notes:

  • Tag is created on ho-release/0.17.0, NOT on ho-prod
  • Command auto-detects which version to promote (smallest)
  • Cannot "skip" a version: if 0.17.0 isn't in prod, can't promote 0.18.0

Step 7: Promote to Production

Sequentiality Rule: Only the smallest version in preparation (with a stage file) can be promoted to production.

half_orm dev release promote prod
# Complete workflow:
# โ†’ Auto-detects smallest version with -patches.toml containing staged patches
# โ†’ Verifies strict sequentiality (0.17.0 must follow last prod)
# โ†’ Automatically switches to ho-prod
# โ†’ Merges ho-release/0.17.0 into ho-prod (integrates patch code)
# โ†’ Restores database and applies all staged patches from TOML
# โ†’ Generates .hop/model/schema-0.17.0.sql and metadata-0.17.0.sql
# โ†’ Updates symlink .hop/model/schema.sql โ†’ schema-0.17.0.sql
# โ†’ Creates .hop/releases/0.17.0.txt snapshot with all patches
# โ†’ Deletes .hop/releases/0.17.0-patches.toml (no longer needed)
# โ†’ Preserves .hop/releases/0.17.0-rc*.txt for history (if any)
# โ†’ Creates tag v0.17.0 on ho-prod
# โ†’ Deletes branch ho-release/0.17.0 (mission complete)
# โ†’ Commits and pushes

Output:

โœ“ Success!

  Version:          0.17.0
  Tag:              v0.17.0
  Branches deleted: ho-release/0.17.0

๐Ÿ“ Next steps:
  โ€ข Deploy to production servers
  โ€ข Start next cycle: half_orm dev release new minor

Important Notes:

  • This is when patch code is actually merged into ho-prod, not before
  • Command auto-detects smallest version with stage file
  • Always uses stage file: RC files are preserved for history but not used for promotion
  • RC is optional: Can promote directly from stage to production without creating RC
  • Sequentiality is strictly enforced: impossible to promote 0.18.0 if 0.17.0 isn't already in prod

Step 8/9: Production Upgrade

# On production server (automatically pulls from origin)
# Check available releases
half_orm dev update

# Apply upgrade (with automatic backup and git pull)
half_orm dev upgrade

Hotfix Workflow (Urgent Production Fixes)

Scenario: Critical bug discovered in production (v0.17.0) while new release (v0.18.0) is already in development. Production needs fixing immediately without waiting for v0.18.0.

Step 1: Reopen Production Version

half_orm dev release hotfix
# Workflow:
# โ†’ Detects production version from model/schema.sql (e.g., 0.17.0)
# โ†’ Verifies tag v0.17.0 exists
# โ†’ Reopens branch ho-release/0.17.0 from tag v0.17.0
# โ†’ Automatically switches to ho-release/0.17.0

Output:

โœ“ Reopened ho-release/0.17.0 from v0.17.0
โœ“ Ready for hotfix patches

๐Ÿ“ Next steps:
  1. half_orm dev patch create <patch_id>
  2. git checkout ho-patch/<patch_id> && half_orm dev patch merge
  3. half_orm dev release promote hotfix

Important Note: This is a break from sequential workflow as we now have two active release branches simultaneously (ho-release/0.17.0 and ho-release/0.18.0).

Step 2: Create and Integrate Hotfix Patch

The workflow is identical to normal workflow:

# On ho-release/0.17.0
half_orm dev patch create 999-critical-security-fix
# ... develop ...
half_orm dev patch apply
# ... test ...
git checkout ho-patch/999-critical-security-fix
half_orm dev patch merge

Step 3: Promote Hotfix to Production

Important: Cannot use promote prod as tag v0.17.0 already exists!

git checkout ho-prod
half_orm dev release promote hotfix
# Hotfix-specific workflow:
# โ†’ Detects hotfix context (tag vX.Y.Z already exists)
# โ†’ Finds next hotfix number (hotfix1, hotfix2, etc.)
# โ†’ Merges ho-release/0.17.0 into ho-prod
# โ†’ Generates model/schema-0.17.0-hotfix1.sql and metadata-0.17.0-hotfix1.sql
# โ†’ Updates symlink model/schema.sql โ†’ schema-0.17.0-hotfix1.sql
# โ†’ Creates releases/0.17.0-hotfix1.txt with patch list
# โ†’ Creates tag v0.17.0-hotfix1 on ho-prod
# โ†’ Deletes branch ho-release/0.17.0
# โ†’ Commits and pushes

Output:

โœ“ Hotfix deployed!

  Version:  0.17.0-hotfix1
  Tag:      v0.17.0-hotfix1
  Patches:  999-critical-security-fix

๐Ÿ“ Next steps:
  โ€ข Deploy to production servers immediately
  โ€ข Sync other releases: git checkout ho-release/0.18.0 && git merge ho-prod

Step 4: Sync Other In-Progress Releases

If a release is in development (e.g., 0.18.0), it must integrate the hotfix:

git checkout ho-release/0.18.0
git merge ho-prod
# Resolve any conflicts
git push origin ho-release/0.18.0

This guarantees the bugfix won't be lost in the next release.

๐Ÿ“– Command Reference

NOTE: use half_orm dev command --help for detailed help on each command

Init & Clone

# Create new project
half_orm dev init <package_name> --database <db_name>

# Clone existing project (automatically pulls from origin)
half_orm dev clone <git_origin>

Patch Commands

# Create new patch (must be on ho-release/* branch)
half_orm dev patch create <patch_id> [-d "description"] [--before <other_patch_id>]
# โ†’ Patches are ordered in the TOML file (insertion order = application order)
# โ†’ Use --before to insert a patch before an existing one
# โ†’ Order matters: patches are applied sequentially

# Apply current patch (must be on ho-patch/* branch)
half_orm dev patch apply

# Close patch - integrate to release (AUTOMATIC VALIDATION!)
# Must be on ho-patch/* branch
half_orm dev patch merge

Patch Ordering:

  • Patches are stored in the TOML file in insertion order
  • This order determines the application sequence (critical for dependencies)
  • Use --before option to insert a patch at a specific position
  • Example: If patch B depends on patch A, ensure A is ordered before B

Release Commands

# Prepare next release (patch/minor/major)
# Creates ho-release/X.Y.Z branch + candidates.txt + stage.txt
half_orm dev release new patch
half_orm dev release new minor
half_orm dev release new major

# Promote stage to RC (automatically pushes)
# Tags ho-release/X.Y.Z, renames stage โ†’ rc
half_orm dev release promote rc

# Promote RC to production (automatically pushes)
# Merges ho-release/X.Y.Z โ†’ ho-prod, creates tag
half_orm dev release promote prod

# Reopen production version for hotfix
half_orm dev release hotfix

# Promote hotfix to production
half_orm dev release promote hotfix

Production Commands

# Fetch available releases (automatically pulls from origin)
half_orm dev update

# Apply releases to production (automatically pulls from origin)
half_orm dev upgrade [--to-release X.Y.Z]

# Dry run (simulate upgrade)
half_orm dev upgrade --dry-run

๐ŸŽฏ Common Patterns

Pattern 1: Planned Development with Integration Branch

# Start by creating release integration branch
half_orm dev release new minor  # Creates ho-release/0.17.0

# Now on ho-release/0.17.0, create patch
half_orm dev patch create 123-add-users
# โ†’ Auto-added to 0.17.0-candidates.txt

# Add SQL/Python files
echo "CREATE TABLE users (id SERIAL PRIMARY KEY, username TEXT);" > Patches/123-add-users/01_users.sql

# Write tests
cat > tests/public/test_public_users.py << 'EOF'
def test_user_creation():
    user = User(username='alice').ho_insert()
    assert user['username'] == 'alice'
EOF

# Apply and test
half_orm dev patch apply
pytest  # Tests should pass

# Commit your work
git add .
git commit -m "Implement users table with tests"

# Close patch - integrate to release (tests validated automatically!)
git checkout ho-patch/123-add-users
half_orm dev patch merge
# โ†’ Status changed to "staged" in TOML
# โ†’ Tests run automatically before integration

Pattern 2: Team Collaboration on Same Release

# Integration Manager: Create release
half_orm dev release new minor  # Creates ho-release/0.17.0

# Developer A: Working on feature
git checkout ho-release/0.17.0
half_orm dev patch create 456-dashboard
# โ†’ Added to 0.17.0-patches.toml as candidate
# ... develop and test ...
git checkout ho-patch/456-dashboard
half_orm dev patch merge
# โ†’ Status changed to "staged" in TOML

# Developer B: Must sync with A's changes first
git checkout ho-release/0.17.0
git pull origin ho-release/0.17.0  # Get A's integrated changes

# Then create patch
half_orm dev patch create 789-reports
# โ†’ Added to 0.17.0-patches.toml as candidate
# ... develop and test ...
git merge origin/ho-release/0.17.0  # Sync again before closing
git checkout ho-patch/789-reports
half_orm dev patch merge
# โ†’ Tests run with 456 + 789 together!

# All patches validated together in stage

Pattern 3: Parallel Development of Different Releases

# Parallel development of different versions
# 1. Create multiple release branches
half_orm dev release new minor  # Creates 0.18.0
half_orm dev release new patch  # Creates 0.17.1

# 2. Add patches to specific versions
git checkout ho-release/0.17.1
half_orm dev patch create 123-hotfix
git checkout ho-patch/123-hotfix
half_orm dev patch merge

git checkout ho-release/0.18.0
half_orm dev patch create 456-feature
git checkout ho-patch/456-feature
half_orm dev patch merge

# 3. Sequential promotion (must promote 0.17.1 before 0.18.0)
half_orm dev release promote rc   # Auto-promotes 0.17.1 (smallest)
# ... validate ...
half_orm dev release promote prod  # 0.17.1 to production
# Now can promote 0.18.0
half_orm dev release promote rc   # Auto-promotes 0.18.0

Pattern 4: Incremental RC (Fix Issues)

# RC1 has issues discovered in testing
half_orm dev release promote rc  # Creates 0.17.0-rc1
# โ†’ Creates rc1.txt snapshot
# โ†’ Staged patches remain in TOML

# Found bug in testing, create fix patch
git checkout ho-release/0.17.0  # Back to integration branch
half_orm dev patch create 999-rc1-fix
# โ†’ Added to 0.17.0-patches.toml as candidate
half_orm dev patch apply
# ... fix and test ...

# Close patch - changes status to staged
git checkout ho-patch/999-rc1-fix
half_orm dev patch merge
# โ†’ Status changed to "staged" in TOML
# โ†’ Validated automatically

# Promote again (creates rc2, automatically pushes)
half_orm dev release promote rc  # Creates 0.17.0-rc2
# โ†’ Creates rc2.txt snapshot with all staged patches
# โ†’ Staged patches remain in TOML

# Repeat until RC passes all validation

Pattern 5: Hotfix on Production

# Bug discovered in production (v0.17.0)
# New release (v0.18.0) already in development

# Reopen production version
half_orm dev release hotfix
# โ†’ Reopens ho-release/0.17.0 from tag v0.17.0
# โ†’ Creates 0.17.0-patches.toml for hotfix patches

# Same workflow as normal patch
half_orm dev patch create 999-critical-fix
# โ†’ Added to 0.17.0-patches.toml as candidate (with # HOTFIX marker)
half_orm dev patch apply
# ... fix and test ...
git checkout ho-patch/999-critical-fix
half_orm dev patch merge
# โ†’ Status changed to "staged" in TOML

# Promote as hotfix
git checkout ho-prod
half_orm dev release promote hotfix
# โ†’ Creates v0.17.0-hotfix1 tag
# โ†’ Creates 0.17.0-hotfix1.txt file
# โ†’ Merges into ho-prod

# Sync other in-progress releases
git checkout ho-release/0.18.0
git merge ho-prod  # Integrate the hotfix

Pattern 6: Production Deployment

# On production server (commands automatically pull from origin)

# Check available releases
half_orm dev update

# Simulate upgrade
half_orm dev upgrade --dry-run

# Apply upgrade (creates backup automatically, pulls from origin)
half_orm dev upgrade

# Or apply specific version
half_orm dev upgrade --to-release 1.4.0

๐Ÿ—๏ธ Architecture

Branch Strategy

ho-prod (main production)
โ”‚
โ”œโ”€โ”€ ho-release/0.17.0 (integration branch, deleted after prod promotion)
โ”‚   โ”œโ”€โ”€ ho-patch/6-feature-x    (temporary, deleted after close)
โ”‚   โ”œโ”€โ”€ ho-patch/7-bugfix-y     (temporary, deleted after close)
โ”‚   โ””โ”€โ”€ ho-patch/8-auth-z       (temporary, deleted after close)
โ”‚
โ””โ”€โ”€ ho-release/0.18.0 (next version in parallel)
    โ””โ”€โ”€ ho-patch/10-new-api     (temporary, deleted after close)

Branch types:

  • ho-prod: Main production branch (source of truth, stable)
  • ho-release/X.Y.Z: Integration branch for version X.Y.Z (temporary, deleted after prod promotion)
  • ho-patch/*: Patch development branches created from ho-release/* (temporary, deleted after close)

Branch Lifecycle:

  1. release new creates ho-release/X.Y.Z from ho-prod
  2. patch create creates ho-patch/ID from ho-release/X.Y.Z
  3. patch merge merges ho-patch/ID into ho-release/X.Y.Z and deletes ho-patch/ID
  4. release promote prod merges ho-release/X.Y.Z into ho-prod and deletes ho-release/X.Y.Z

Exception - Hotfix Branches:

  • release hotfix reopens ho-release/X.Y.Z from existing tag vX.Y.Z
  • Multiple ho-release/* branches can exist temporarily (prod version + dev version)
  • After promote hotfix, the hotfix branch is deleted

Release Files

.hop/releases/
โ”œโ”€โ”€ 0.17.0-patches.toml    (patches with status: candidate/staged, mutable)
โ”œโ”€โ”€ 0.17.0-rc1.txt         (first RC snapshot, immutable)
โ”œโ”€โ”€ 0.17.0-rc2.txt         (fixes from rc1 snapshot, immutable)
โ”œโ”€โ”€ 0.17.0.txt             (production snapshot, immutable)
โ”œโ”€โ”€ 0.17.0-hotfix1.txt     (hotfix on production snapshot, immutable)
โ””โ”€โ”€ 0.18.0-patches.toml    (next version patches)

File lifecycle (normal workflow):

patch create โ†’ X.Y.Z-patches.toml (patch added as "candidate")
                    โ†“
patch merge โ†’ X.Y.Z-patches.toml (status changed to "staged")
                    โ†“
                    โ”œโ”€โ†’ promote rc โ†’ X.Y.Z-rc1.txt snapshot created
                    โ”‚                staged patches reset to candidates
                    โ”‚                    โ†“
                    โ”‚             promote rc โ†’ X.Y.Z-rc2.txt (if fixes needed)
                    โ”‚
                    โ””โ”€โ†’ promote prod โ†’ X.Y.Z.txt snapshot created
                                       X.Y.Z-patches.toml deleted

Key point: promote prod creates a .txt snapshot from staged patches in the TOML file, then deletes the TOML file. RC files are kept for history only.

File lifecycle (hotfix workflow):

release hotfix โ†’ Reopens X.Y.Z-patches.toml
                    โ†“
patch merge โ†’ X.Y.Z-patches.toml (adds hotfix patches as staged)
                    โ†“
promote hotfix โ†’ X.Y.Z-hotfixN.txt (snapshot created, TOML deleted)

TOML file format:

[patches]
"patch-id" = "candidate"  # or "staged"

TXT snapshot format (RC/prod/hotfix):

  • Each line contains a patch ID
  • Lines starting with # are comments
  • Empty lines are ignored

Patch Directory Structure

Patches/
โ””โ”€โ”€ 123-feature-name/
    โ”œโ”€โ”€ README.md           (auto-generated description)
    โ”œโ”€โ”€ 01_schema.sql       (schema changes)
    โ”œโ”€โ”€ 02_data.sql         (data migrations)
    โ””โ”€โ”€ 03_indexes.sql      (performance optimizations)

Execution order: Lexicographic (01, 02, 03...)

Semantic Versioning

MAJOR.MINOR.PATCH
  โ”‚     โ”‚     โ”‚
  โ”‚     โ”‚     โ””โ”€โ”€ Bug fixes, minor changes (1.3.5 โ†’ 1.3.6)
  โ”‚     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ New features, backward compatible (1.3.5 โ†’ 1.4.0)
  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Breaking changes (1.3.5 โ†’ 2.0.0)

Workflow Rules

  1. Sequential releases: Must promote 0.17.0 before 0.17.1 or 0.18.0
  2. Auto-detection: Commands automatically detect smallest version to promote
  3. Patch origin: Must create patches from ho-release/* branch, not ho-prod
  4. Patch lifecycle: new โ†’ candidates โ†’ close โ†’ stage โ†’ rc โ†’ prod
  5. Branch cleanup:
    • patch merge deletes ho-patch/* branch
    • promote prod deletes ho-release/* branch
  6. Database restore: patch apply always restores from production state
  7. Immutable releases: RC and production files never modified
  8. Automatic Git operations: Push/pull handled by commands automatically
  9. โš ๏ธ SYSTEMATIC TEST VALIDATION: Tests run before integration (in patch merge)
  10. Hotfix exception: Can reopen production version while other releases in progress
  11. # HOTFIX marker: Candidates file marked with # HOTFIX comment for hotfix releases

๐Ÿ”ง Troubleshooting

Error: "Must be on ho-release/* branch"

# Solution: Create release or switch to release branch
half_orm dev release new minor
# or
git checkout ho-release/0.17.0

Error: "Must be on ho-patch/* branch"

# Solution: Create or switch to patch branch
# First ensure you're on ho-release/*
git checkout ho-release/0.17.0
half_orm dev patch create <patch_id>
# or
git checkout ho-patch/<patch_id>

Error: "Patch not found in candidates file"

# Solution: Patch must be created from ho-release/* branch
# to be automatically added to candidates
git checkout ho-release/0.17.0
half_orm dev patch create <patch_id>

Error: "Repository is not clean"

# Solution: Commit or stash changes
git status
git add .
git commit -m "Your message"
# or
git stash

Error: "Repository not synced with origin"

# This should not happen - commands handle git operations automatically
# If it does occur:
git pull origin ho-prod

Error: "No stage releases found"

# Solution: Prepare a release first
half_orm dev release new patch

Error: "Active RC exists"

# Cannot promote different version while RC exists
# Solution: Promote current RC to production first
half_orm dev release promote prod

# Then promote your stage
half_orm dev release promote rc

Error: "Tests failed for patch integration"

# Tests ran on temp-valid branch and failed
# Solution: Fix your tests or code
half_orm dev patch apply  # Test locally first
pytest  # Verify tests pass

# Fix issues in your patch
vim Patches/123-feature/01_schema.sql
vim tests/test_feature.py

# Try again
git checkout ho-patch/123-feature
half_orm dev patch merge  # Tests will run again

Patch apply failed (SQL error)

# Database automatically rolled back
# Solution: Fix SQL files and re-apply
vim Patches/123-feature/01_schema.sql
half_orm dev patch apply

Lost after conflicts

# View repository state
git status
git log --oneline -10

# View current branch
git branch

# View remote branches
git branch -r

# Return to safe state
git checkout ho-prod
# Commands handle git pull automatically

๐ŸŽ“ Best Practices

Patch Development

โœ… DO:

  • Write tests FIRST (TDD approach)
  • Start with exploratory patches (no release needed initially)
  • Use descriptive patch IDs: 123-add-user-authentication
  • Test patches thoroughly before adding to release
  • Keep patches focused (one feature per patch)
  • Commit generated code with meaningful messages
  • Create release when patches are ready to integrate
  • Run pytest locally before patch merge

โŒ DON'T:

  • Mix multiple features in one patch
  • Skip patch apply validation
  • Add untested patches to release
  • Modify files outside your patch directory
  • Worry about git push/pull (commands handle it automatically)
  • Skip writing tests (validation will fail anyway)

Release Management

โœ… DO:

  • Prepare releases when patches are ready to integrate
  • Trust the automatic test validation system
  • Test RC thoroughly before promoting to production
  • Use semantic versioning consistently
  • Document breaking changes in commit messages
  • Let commands handle git operations automatically
  • Review test failures carefully before retrying

โŒ DON'T:

  • Skip RC validation (always test before prod)
  • Promote multiple RCs simultaneously
  • Skip backup creation in production
  • Force promote without fixing issues
  • Manually push/pull (let commands handle it)
  • Bypass test validation (it's there for your safety)

Production Deployment

โœ… DO:

  • Always run update first to check available releases
  • Use --dry-run to preview changes
  • Verify backups exist before upgrade
  • Monitor application after deployment
  • Schedule deployments during low-traffic periods
  • Trust commands to handle git operations
  • Verify all tests passed in RC before promoting

โŒ DON'T:

  • Deploy without testing in RC first
  • Skip backup verification
  • Deploy during peak usage hours
  • Ignore upgrade warnings
  • Apply patches directly without releases
  • Manually git pull (commands do it automatically)
  • Promote to production if RC tests failed

Testing Best Practices

โœ… DO:

  • Write tests for all business logic
  • Test database constraints and validations
  • Use fixtures for common test scenarios
  • Test edge cases and error handling
  • Keep tests fast and isolated
  • Document test intentions clearly
  • Run tests locally before pushing

โŒ DON'T:

  • Skip tests for "simple" changes
  • Write tests that depend on execution order
  • Ignore test failures
  • Write tests without assertions
  • Test implementation details instead of behavior

๐Ÿ“š Documentation

  • Quick Reference: This README

๐Ÿค Contributing

Contributions are welcome! Please read our contributing guidelines and submit pull requests to our repository.

Development Setup

# Clone repository
git clone https://github.com/half-orm/half-orm-dev.git
cd half-orm-dev

# Install in development mode
pip install -e .

# Run tests
pytest

๐Ÿ“ž Getting Help

# Command help
half_orm dev --help
half_orm dev patch --help
half_orm dev release --help

# Specific command help
half_orm dev patch create --help
half_orm dev release promote --help
half_orm dev update --help
half_orm dev upgrade --help

Support

๐Ÿ“„ License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.


Version: 0.17.0 half-orm: Compatible with half-orm 0.17.x Python: 3.8+ PostgreSQL: Tested with 13+ (might work with earlier versions)

Made with โค๏ธ by the half-orm team

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

half_orm_dev-0.17.3a2.tar.gz (175.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

half_orm_dev-0.17.3a2-py3-none-any.whl (170.1 kB view details)

Uploaded Python 3

File details

Details for the file half_orm_dev-0.17.3a2.tar.gz.

File metadata

  • Download URL: half_orm_dev-0.17.3a2.tar.gz
  • Upload date:
  • Size: 175.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.0

File hashes

Hashes for half_orm_dev-0.17.3a2.tar.gz
Algorithm Hash digest
SHA256 a7c0f49cff906a1d9438981d104feb67c47a4b0eca4f10f7e8cbfd26f3ce70de
MD5 e8880b6919105be1e45bcf1868144f98
BLAKE2b-256 dd6468efcbf20cd071945c85dfa5a68664886c4dfd901f47ddbe631c3c172541

See more details on using hashes here.

File details

Details for the file half_orm_dev-0.17.3a2-py3-none-any.whl.

File metadata

File hashes

Hashes for half_orm_dev-0.17.3a2-py3-none-any.whl
Algorithm Hash digest
SHA256 a7262abe27cf5310e74a12306024f346f1cff3c34de7fd5d8f9caddfd2d3e758
MD5 c331ab5bd2b2c43cb48e4f93e70e93ee
BLAKE2b-256 3e36d93a95eaa6427e59022f703b140ec3e483cb56b8e7f053d2e2f1554e261b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page