Poetiq

A higher level wrapper for poetry that creates templates pre-filled with basic structure and setup that I find convenient as a starting point for my packages.

• Usage: command line usage with examples
• Examples: examples of templates and functionalities results
• development notes: notes on how to add new features to poetiq

Install

pip install poetiq

or for most recent developments:

pip install git+https://github.com/sagitta42/poetiq.git

Usage

$ poetiq -h
usage: poetiq [-h] {new,init,add,update,setup,install} ...

positional arguments:
  {new,init,add,update,setup,install}
    new                 create new template
    init                basic no-interaction init
    add                 poetry add with git+ auto-detect
    update              update current template as is with new poetiq updates
    setup               setup functionality in existing repo/directory
    install             poetry install with added options

options:
  -h, --help            show this help message and exit

Create template

$ poetiq new -h
usage: poetiq new [-h] [--type [{package,app}]] [--db-type [{sqlite,psql,none}]] [--dev-sqlite] [--pydantic-table] [--mongodb] [--settings] [--progressbar] name

positional arguments:
  name                  Template/repository name

options:
  -h, --help            show this help message and exit
  --type [{package,app}]
                        Type of functionality
  --db-type [{sqlite,psql,none}]
                        Database type (app only)
  --dev-sqlite          Development mode switch to SQLite (app only)
  --pydantic-table      Set up pydantic-table for alembic migrations (app only)
  --mongodb             Add MongoDB service (app only)
  --settings            Set up .env Settings class (package only)
  --progressbar         Set up progress bar source code (package only)

Main note: poetry new package-name complains if directory package-name already exists; poetiq new package-name only complains if it is non-empty

Example:

$ poetiq new awesome-package --type pacakge --settings

Available package types:

• package to create a package template (default)
• app to create a simple web app template

Add --db flag to set up alembic migrations and DB of given type (applies to app template type only)

Available DB types:

• sqlite to set up a local SQLite DB (default)
• psql to set up PostgreSQL service in docker-compose.yml

Add --dev-sqlite flag to set up dual psql/SQLite setup with switch to SQLite via .env variables for local development testing.

Add --mongodb flag to set up MongoDB service in docker-compose.yml and related source files and dependencies in the app code (app only).

Add --settings flag to set up pydantic_settings based Settings class containing .env variables (applies to package template only; app template always includes this class / source file)

Add --progressbar flag to set up a simple ProgressBar util class in a package source file.

See detailed examples in Template examples

Init template

poetiq init -h to init a simple template in current direcotry with most basic no-interaction poetry pyproject init. Will treat current directory name as project name.

Add dependency

$ poetiq add -h
usage: poetiq add [-h] [--local [LOCAL]] package

positional arguments:
  package          Package source (name, https, git)

options:
  -h, --help       show this help message and exit
  --local [LOCAL]  Add local dependency to poetiq.toml

Running poetiq add package-name is equivalent to poetry add package-name.

Adding package from a repository, running poetiq add https://github.com/username/awesome-package will automatically add git+ (same for ssh hosted git@...)

Use --local flag and path to a local clone/repository as poetiq add package-name --local /path/to/awesome-package to add local dependency to poetiq.toml - see Install for poetiq install --local usage to handle dual dependencies.

Update template

Run poetiq update inside an existing poetiq template to update it after poetiq itself was updated (new functionalities, bugfixes).

The update will create a special separate update branch, run poetiq template setup in it, and then merge the branch into the one you started from. This way, the updates do not all overwrite the changes you made afterwards. Make sure to handle the merge manually anyway, and be able to recover your original setup in case the merge it too complex.

See detailed examples in Template examples

Set up functionality

$ poetiq setup -h
usage: poetiq setup [-h] [--db-type [{sqlite,psql}]] [--dev-sqlite] [--pydantic-table] [--subfolder [SUBFOLDER]] [--no-commit] {vscode,gitignore,db,logger}

positional arguments:
  {vscode,gitignore,db,logger}
                        Type of functionality

options:
  -h, --help            show this help message and exit
  --db-type [{sqlite,psql}]
                        Database type (db only)
  --dev-sqlite          Development mode switch to SQLite (db only)
  --pydantic-table      Set up pydantic-table for alembic migrations (db only)
  --subfolder [SUBFOLDER]
                        Subfolder of setup (logger only)
  --no-commit           Do not commit changes

Single functionalities set up in current directory:

• poetiq setup vscode - creates/updates .vscode setup
• poetiq setup gitignore - creates/updates .gitignore
• poetiq setup db --db psql --dev-sqlite - sets up psql DB with dev mode switch to SQLite

If directory is a git repository, will commit changes unless --no-commit flag is provided.

See detailed examples in Functionality setup examples

Install

$ poetiq install -h
usage: poetiq install [-h] [--local]

options:
  -h, --help  show this help message and exit
  --local     Install local dependencies defined in poetiq.toml

Perform smart poetry install: automatically add --no-root flag if current directory pyproject.toml states package-mode=false

Add --local flag if you want to install the dependences in pyproject.toml from filepath instead of pyproject information (e.g. a local clone of a dependency, which may be convenient during development)

Provide paths to local dependencies via poetiq.toml file. Format:

[dependency-groups]
local = [
  "my-package @ /path/to/my-package",
  "python-module @ /path/to/my/fork/of/python-module",
]

or by running poetiq add my-package --local /path/to/my-package (see Add dependency section)

Specify a packge to perform local install with poetiq install --local my-package or simply --local to perform local install for all dual packages.

See detailed examples in Install examples

Examples

Templates

poetiq new awesome-package --type package --settings --progressbar

Result

awesome-package
├── .vscode
│   ├── launch.json # debug test setup
│   └── settings.json # pytest, format on save, pylance, auto-import, ...
├── src
│   └── awesome_package
│       ├── __init__.py # imports * from core.py
│       ├── foo.py # example source file
│       ├── logger.py # log with levels based on .env and color/bold functionalities
│       ├── core.py # everything here is imported in __init__ as core functionality
|       ├── py.typed # empty file that enables import suggestions in IDE
|       ├── progressbar.py # ProgressBar wrapper class if requested
│       └── settings.py # pydantic_settings based Settings class containing .env variables if requested
├── tests
|   ├── __init__.py
│   ├── conftest.py # set up to be able to run tests in dev mode
│   └──  test_unit.py # unit tests of awesome_package.foo and awesome_package.models.MyBaseModel
├── .gitignore # standard comprehensive Python .gitignore
├── .env.template
├── poetry.lock
├── pyproject.toml
├── README.md
└── venv # venv with pyproject.toml dependencies: dotenv; poetry and pytest (dev)

poetiq new awesome-app --type app --db psql --dev-sqlite --mongodb

Result

awesome-app
├── .vscode
│   ├── launch.json # debug test setup
│   └── settings.json # pytest, format on save, pylance, auto-import, ...
├── alembic_migrations # migrations for SQLite if requested; adaptation for PostgreSQL coming soon
│   ├── versions
│   ├── env.py # auto DB URL using settings.py - compatible with SQLite/psql via only .env change
│   ├── README
│   └── script.py.mako
├── app
│   ├── api
│   │   ├── routes
│   │   │   └── dummy.py
│   │   └── router.py # main API router that includes dummy router
│   ├── schemas
│   │   └── dummy.py # dummy request and response schemas
│   └── services
│       └── dummy.py # dummy service using dummy core logic
├── core
│   ├── models
│   │   ├── example.py # DeclarativeBase for sqlalchemy session
│   │   └── mongo_document.py
│   ├── db.py # DB session with automatic dual SQLite/psql switch based on .env
│   ├── db_mongo.py # dummy MongoDB client
│   ├── dummy.py # dummy core logic
│   └── mongo_config.py # MongoDB client config using settings.py
├── db
│   └── database.db # initial SQLite DB file, not tracked
├── venv # venv with pyproject.toml dependencies installed: dotenv, fastapi, pydantic, ...
├── .env.template # controls switch from SQLite to psql with just a few variables
├── .gitignore # standard comprehensive Python .gitignore
├── alembic.ini
├── app_info.py # app info extraction from pyproject
├── docker-compose.yml # app, psql, and mongodb services, env variables set based on .env
├── dockerfile # app service dockerfile
├── main.py # main API launcher
├── poetiq.toml.template
├── poetry.lock
├── pyproject.toml
├── README.md
└── settings.py  # pydantic_settings based Settings class containing .env variables for sqlite/psql dual setup and MongoDB; shared by alembic migrations, SQLAlchemy Session, and MongoDB client

Update template

poetiq update

On the first update, will create a branch dedicated to poetiq updates starting from the first commit.

$ git branch
  dev-poetiq-update
* main

The standard setup is run in the update branch, and the differences/additions are committed and merged with the active branch.

commit cac874d5f2cf07199f00890c4d4cefbb57d3206b (HEAD -> main)
Merge: a01eefc b4bb541
    Merge branch 'dev-poetiq-update'

commit b4bb54109bf78b85618566c3df082128a3f93dd8 (dev-poetiq-update)
    poetiq update
    commit: d68f600af54ae2410557d19a5f72b09ed63aadbe
    message: <last poetiq commit message>

commit a01eefcfe8fff4372c9ad337d40e9ad991b32f9d
    readme update

commit d68f600af54ae2410557d19a5f72b09ed63aadbe
    template made with poetiq

Functionalities

$ poetiq setup vscode
VSCode update with [poetiq](https://pypi.org/project/poetiq)
├── settings.json
└── launch.json

Install examlpes

Poetiq automatically determines --no-root flag analyzing pyproject.toml for package-mode=false:

$ poetiq install --local
Local install requested but no dual dependencies found in poetiq.toml
poetiq: poetry install --no-root
Installing dependencies from lock file

No dependencies to install or update

Poetiq uninstalls and re-installs dual dependencies:

$ poetiq install --local
poetiq: poetry install --no-root
Installing dependencies from lock file

No dependencies to install or update
Replacing dual packages with local dependencies
poetiq: pip uninstall python-module
Found existing installation: python-module 2.13.4
Uninstalling python-module-2.13.4:
  Would remove:
    /home/user/path/to/repo/venv/lib/python3.12/site-packages/python-module-2.13.4.dist-info/*
    /home/user/path/to/repo/venv/lib/python3.12/site-packages/python-module/*
Proceed (Y/n)? 

...

poetiq: pip install /path/to/my/fork/of/python-module
Processing /path/to/my/fork/of/python-module

...

Successfully installed python-module-2.14.0a1 ...

with .poetiq.toml:

[dependency-groups]
local = [
  "python-module @ /path/to/my/fork/of/python-module",
]

development notes

implement new independent functionality item setup (setup)

1. Create new SetupType e.g. SetupType.foo (settings.setup)
2. Add SetupType.foo to choices for type argument of the microfunctionality subparser in add_microfunctionality_arguments() (cli/cli.py)
3. Create item settings FooSettings in poetiq.settings.item inheriting from SetupSettings with type as Literal[SetupType.foo]
4. Add additional settings field if any under FooSettings e.g. field
5. Create function adding those settings to given CLI parser in cli/cli.py e.g. add_foo_arguments(parser) utilizing FooSettings to translate them to CLI arguments. Append call to this function under add_microfunctionality_arguments()
6. Add FooSettings to accepted setup settings ( settings.options)
7. Create item setup class FooSetup in a new source file poetiq.item.foo inheriting from a base setup (e.g. BaseFunctionalitySetup, BaseVenvSetup, or BaseDependencySetup ) with [FooSettings] (Generic) depending on if item includes python library dependency setup etc. For convenience, define __init__() with settings=FooSettings()
8. Define setup() method, calling parent setup(), and adding specific setup actions for this item. This method must return bool representing whether this setup already existed before.
9. In case of dependency setup, add dependencies in setup_dependencies() using _poetry_add("package-name")
10. Add FooSetup under ItemSetupClass enum in item.builder, matching enum name with SetupType name (foo)

After this, this setup is now usable with poetiq add foo

implement new DB setup

1. Define new DB type in DBType e.g. DBType.foo
2. Create DB setup class FooDBSetup inheriting from BaseDBSetup
3. Define its setup_db() method with actions for this DB setup. Return bool representing whether this setup existed before
4. Define DB URL under db_url property
5. Add FooDBSetup under DBSetupClass in poetiq.item.db.builder using the same enum name as defined DBType (foo)

After this, this setup is now usable with

• poetiq new awesome-app --db psql
• poetiq setup db --db foo

pydantic <-> argparse adapter

Template and setup settings fields are used to set argparse descriptions, defaults, and options to avoid duplications.

For this reason, even if otherwise unnecessary:

• default for type is always set
• field type annotation is always set
• field description is always set

build assets

To run tests locally, need to first run poetry build to generate src/poetiq/_build_assets (see build.py) for non-src assets