SQLDbWrpr 5.1.0

Wrapper for importing CSV and Text files into MySQL and Postgress

SqlDbWrpr

Category	Status' and Links
General
CD/CI
PyPI
Github

Short description

SqlDbWrpr is a Python wrapper that streamlines schema creation plus CSV import/export workflows for SQL backends.

Module Overview

SqlDbWrpr is a Python utility for creating SQL database schemas and moving CSV data into and out of those schemas. It currently provides wrappers for MySQL and PostgreSQL.

Schemas can be supplied in two ways:

1. A legacy db_structure dictionary.
2. SQLAlchemy metadata, either directly through p_sqlalchemy_metadata or through p_sqlalchemy_base.metadata.

When both are supplied, p_db_structure takes precedence for backward compatibility. If no supported schema source is supplied, SchemaSourceError is raised.

Key Features

• Schema Management: Create databases, tables, primary keys, foreign keys, and indexes from a legacy dictionary or SQLAlchemy metadata.
• Data Import/Export:
  • Import CSV data from files or in-memory rows, including single-volume and numbered multi-volume files.
  • Export full tables or custom SQL query results to CSV, including optional multi-volume exports.
• Database Support: Includes MySQL and PostgreSQL wrappers with dialect-specific SQL rendering.
• User and Permission Management: Create MySQL users and grant database rights.
• Batch Processing: Configure import batch sizes for larger CSV loads.

Project Structure

• src/sqldbwrpr/: Core library implementation, including MySQL and PostgreSQL wrappers.
• tests/: Unit and integration-oriented test coverage for wrapper behaviour.
• scripts/: SQL setup/bootstrap assets used for database initialization.
• Root automation scripts (*.ps1) and CI configuration under .github/workflows/ support setup and delivery.

Getting Started

Installation

pip install SqlDbWrpr

Quick Start With A Legacy Structure

from sqldbwrpr.sqldbwrpr import MySQL

field_defaults = {
    "PrimaryKey": ["", ""],
    "FKey": [],
    "Index": [],
    "NN": "",
    "B": "",
    "UN": "",
    "ZF": "",
    "AI": "",
    "G": "",
    "DEF": "",
}

db_structure = {
    "Users": {
        "ID": {
            "Type": ["int"],
            "Params": {
                **field_defaults,
                "PrimaryKey": ["Y", "A"],
                "NN": "Y",
                "AI": "Y",
            },
            "Possible Values": "",
            "Comment": "",
        },
        "Username": {
            "Type": ["varchar", 50],
            "Params": {**field_defaults, "NN": "Y"},
            "Possible Values": "",
            "Comment": "",
        },
    }
}

db = MySQL(
    p_host_name="localhost",
    p_user_name="root",
    p_password="yourpassword",
    p_db_name="my_database",
    p_db_structure=db_structure,
    p_recreate_db=True,
)

db.import_csv("Users", p_csv_db=[("Username",), ("alice",)], p_vol_type="Single")
db.export_to_csv("exported_users.csv", "Users")

Quick Start With SQLAlchemy Metadata

from sqlalchemy import Column
from sqlalchemy import Integer
from sqlalchemy import MetaData
from sqlalchemy import String
from sqlalchemy import Table

from sqldbwrpr.sqldbwrpr import PostgreSQL

metadata = MetaData()
Table(
    "Users",
    metadata,
    Column("ID", Integer, primary_key=True, autoincrement=True),
    Column("Username", String(50), nullable=False),
)

db = PostgreSQL(
    p_host_name="localhost",
    p_user_name="postgres",
    p_password="yourpassword",
    p_db_name="my_database",
    p_sqlalchemy_metadata=metadata,
    p_recreate_db=True,
)

db.import_csv("Users", p_csv_db=[("Username",), ("alice",)], p_vol_type="Single")

---

Updating ReleaseNotes Instructions

1. Run the pushpy.ps1 script or manually commit the current changes.
2. Generate the release notes
3. Use one of the following AI propmpts in Notion to generate the release notes.

• Release - Update - General
• Release - Update - VenvIt

or

1. Use the following template and manually update the ReleaseNotes.md file.

    # Release ?.?.?
    ## Summary of Changes
    - bla, bla, bla
    ## Next Heading
    - bla, bla, bla
    ---
   

2. You can repeat step 1 multiple times.

3. You can repeat step 2 multiple times but update the ReleaseNotes that has not been published.

4. Run the pushpr.ps1 script once you are ready to create the PR to publish the release. TOy can also manually create the tag, touch a file, commit and push the changes.

5. Merge the PR in GitHub.

6. Confirm the following:

7. The release update reflects in GitHub

8. The release update notification was sent

---