Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Automate creation of sources

Project Description
author:Lele Gaifax
contact:lele@metapensiero.it
license:GNU General Public License version 3 or later

This is a simple tool that takes a YAML declarative configuration of actions, each describing the set of steps needed to accomplish it, and realizes one or more action possibly interpolating a set of dynamic values inserted by the user.

I’ve written it with the goal of taking away, as much as possible, the tediousness of creating new content in a PatchDB setup.

There is a myriad of similar tools around, but most of them are either targeted at bootstrapping a hierarchy of files and directories (that is, a one-shot operation, for example to create the skeleton of a new Python package, like cookiecutter to mention one) or belong to a very specific domain (an example of this category may be ZopeSkel).

To fulfill my goal I needed a tool able to perform an arbitrary list of operations, primarily:

  • create a directory
  • create a file with a particular content
  • change the content of an existing file

It obviously need to be parametric in almost every part: the pathnames and the actual content of the files must be determined by a template interpolated with a set of values. This is what Jinja2 excels at. Since the values are mostly volatile (that is, almost all of them change from one invocation to the other), I used the whaaaaat library to prompt the user.

Configuration

The YAML configuration file may be composed by one or two documents: an optional global section and a mandatory one containing the list of available actions, each of them being basically a list of steps.

Global section

The first document contains a set of optional global settings, in particular:

  • the options to instantiate the Jinja2 environment
  • a set of file headers, one for each kind of file
  • a list of steps to be performed at every invocation

Jinja2 environment

To avoid clashes with the syntax used by Python’s format() minilanguage, the default environment settings are the following:

block_start_string
The start of a block is <<, instead of the default {%
block_end_string
The end of a block is >>, instead of the default %}
variable_start_string
A variable is introduced by «, instead of the classic {{
variable_end_string
A variable ends with », and not with the classic }}
extensions
The extension jinja2_time.TimeExtension is automatically selected
keep_trailing_newline
By default the trailing newline is not discarded

Obviously any Jinja2 template defined by the configuration must adhere to these settings. If you don’t like them, or if you want define different ones, you can put an entry like the following in the globals section of the configuration:

jinja:
  block_start_string: "[["
  block_end_string: "]]"
  variable_start_string: "<<"
  variable_end_string: ">>"

File headers

This is a map between a file suffix and a template that will be automatically injected in the render’s context as the variable header. For example, should you need to generate a Pascal unit, you could insert the following:

headers:
  pas: |
    (*« "*" * 70 »
     * Project: «package_name»
     * Created: «timestamp»
     *« "*" * 70 »)

Initial steps

This is a list of steps to be executed unconditionally at startup. In particular it may be used to gather some values from arbitrary places to initialize the answers, to prompt the user for a common set of values and to define custom steps. As an example, you could say:

steps:
  - python:
      script: |
        import os, pwd, sys

        class InitCustomAnswersWithoutPrompt(Step):
            def __init__(self, state, config):
                super().__init__(state, config)
                self.name = config['name']
                self.value = config['value']

            def announce(self):
                self.state.announce('*', "Inject %s=%s", self.name, self.value)

            def __call__(self):
                return {self.name: self.value, python_version: sys.version}

        register_step('initcustom', InitCustomAnswersWithoutPrompt)

        myself = pwd.getpwuid(os.getuid())
        state.answers['author_username'] = myself.pw_name
        state.answers['author_fullname'] = myself.pw_gecos.split(',')[0]

  ## Here you can execute the new kind of operation defined above

  - initcustom:
      name: "myextravar"
      value: "thevalue"

Actions

An action is identified by a unique name and carries an optional description, an optional set of prompts specific to the action and a list of one or more steps.

The following is a complete example:

create_top_level_setup_py:
  description: Create top level setup.py

  prompt:
    - package_name:
        message: The name of the Python package

  steps:
    - createdir:
        directory: src

    - createfile:
        directory: src
        filename: setup.py
        content: |
          # Hi, I'm the setup.py file for «package_name»

Steps

A step is some kind of operation that must be carried out. The name of the step identifies the kind of operation, and its value is used to specify the needed parameters. So, in the example above, we have two steps, createdir and createfile, each requiring its specific arguments map.

A step may be conditionally skipped specifying an expression in its when setting: if present, the operation will be performed only when the expression evaluates to true.

This is the list of available operation kinds:

changefile

Perform some quite simple changes to the content of an existing file.

Required configuration:

directory
The directory containing the file to be changed
filename
The name of the existing file within the given directory
changes
A list of tweaks: there are currently just three types, one that add some content before a given marker, one to add the content after it and one that insert some content between a marker and another marker keeping the block sorted

Example:

- changefile:
    directory: src
    filename: listofitems.txt
    changes:
      - add: "«newitemname»\n"
        before: "\n;; items delimiter\n"
      - add: "«newitemname»\n"
        after: "\n;; reversed insertion order\n"

- changefile:
    directory: src
    filename: __init__.py
    changes:
      - insert: "from .«table_name» import «table_name»\n"
        between: "\n## ⌄⌄⌄ tinject import marker ⌄⌄⌄, please don't remove!\n"
        and: "\n## ⌃⌃⌃ tinject import marker ⌃⌃⌃, please don't remove!\n"
createdir

Create a directory and its parents.

Required configuration:

directory
The directory to be created

Example:

- createdir:
    directory: src/my/new/package
createfile

Create a new file with a particular content.

Required configuration:

directory
The directory contained the file to be created
filename
The name of the new file
content
A Jinja2 template that will be rendered and written to the new file

Example:

- createfile:
    directory: "«docs_dir»/«schema_name»/tables"
    filename: "«table_name».sql"
    description: Structure of table «schema_name».«table_name»
    ## The template may be either inline or included from an external file
    content: !include 'table.sql'
prompt

Ask the user for some information bits.

Required configuration: a list of dictionaries, each representing a whaaaaat's question.

Example:

- prompt:
    - name_of_the_variable:
        message: Tell me the value
        default: "default value"

    - different_kind_of_input:
        message: Select the variant
        kind: list
        choices:
          - Big
          - Medium
          - Small
python

Execute an arbitrary Python script.

Required configuration:

script
The code of the script

The script is executed with a context containing the class Step, the function register_step and the global state of the program.

See the initial steps above for an example.

repeat

Repeat a list of substeps

Required configuration:

steps
The list of substeps to repeat

Optional configuration:

description
A message string, emitted at the start, if given
answers
The name of variable holding a list of answers, when one substep is a prompt
count
The number of iterations
when
A Jinja boolean expression: if given it’s evaluated once before the loop begins, that gets executed only when it expression’s value is true, otherwise no repetition happens at all; the expression may refer to previous answers, even those collected while looping (that is, the variable specified by the answers option)
until
A Jinja boolean expression: if given (and count is not), then the loop is terminated when the condition is false
again_message
When neither count nor until are specified, the step will explicitly ask confirmation about looping again, at the end of all substeps execution

See examples/repeat.yml for an example.

Sample session

Create a new schema with a new table:

$ tinject --verbose apply examples/patchdb.yml new_schema new_table

* Execute Python script
[?] Author fullname (author_fullname): Lele Gaifax
[?] Author username (author_username): lele
[?] Author email (author_email): «author_username»@example.com
[?] Fully qualified package name (package_name): package.qualified.name
[?] Timestamp (timestamp): << now 'local', '%a %d %b %Y %H:%M:%S %Z' >>
[?] Year (year): << now 'local', '%Y' >>
[?] Distribution license (license): GNU General Public License version 3 or later
[?] Copyright holder (copyright): © «year» «author_fullname»
[?] Root directory of Sphinx documentation (docs_dir): docs/database
[?] Root directory of SQLAlchemy model sources (model_dir): src/«package_name|replace(".","/")»

=====================
 Create a new schema
=====================
[?] Name of the new schema (schema_name): public

* Create directory docs/database/public

* Create file docs/database/public/index.rst

* Create directory docs/database/public/tables

* Create file docs/database/public/tables/index.rst

* Create directory src/package/qualified/name/entities/public

* Create file src/package/qualified/name/entities/public/__init__.py

* Create directory src/package/qualified/name/tables/public

* Create file src/package/qualified/name/tables/public/__init__.py

====================
 Create a new table
====================
[?] Schema name of the new table (schema_name): public
[?] Name of the new table (table_name): things
[?] Description of the new table (table_description): The table ``«schema_name».«table_name»`` contains...
[?] Name of the corresponding entity (entity_name): Thing

* Create file docs/database/public/tables/things.rst

* Create file docs/database/public/tables/things.sql

* Create file src/package/qualified/name/entities/public/thing.py

* Change file src/package/qualified/name/entities/public/__init__.py

  - insert “from .thing import T…” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”

  - add “mapper(Thing, t.thi…” after “## ⌃⌃⌃ tinject impo…”

* Create file src/package/qualified/name/tables/public/things.py

* Change file src/package/qualified/name/tables/public/__init__.py

  - insert “from .things import …” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”

Verify:

$ cat src/package/qualified/name/entities/public/__init__.py
# -*- coding: utf-8 -*-
# :Project:   package.qualified.name -- Entities in schema public
# :Created:   mer 15 giu 2016 13:24:54 CEST
# :Author:    Lele Gaifax <lele@example.com>
# :License:   GNU General Public License version 3 or later
# :Copyright: © 2016 Lele Gaifax
#

from sqlalchemy.orm import mapper

from ...tables import public as t

## ⌄⌄⌄ tinject import marker ⌄⌄⌄, please don't remove!
from .thing import Thing

## ⌃⌃⌃ tinject import marker ⌃⌃⌃, please don't remove!

mapper(Thing, t.things, properties={
})

Add another table:

$ tinject --verbose apply examples/patchdb.yml new_table

* Execute Python script
[?] Author fullname (author_fullname): Lele Gaifax
[?] Author username (author_username): lele
[?] Author email (author_email): «author_username»@example.com
[?] Fully qualified package name (package_name): package.qualified.name
[?] Timestamp (timestamp): << now 'local', '%a %d %b %Y %H:%M:%S %Z' >>
[?] Year (year): << now 'local', '%Y' >>
[?] Distribution license (license): GNU General Public License version 3 or later
[?] Copyright holder (copyright): © «year» «author_fullname»
[?] Root directory of Sphinx documentation (docs_dir): docs/database
[?] Root directory of SQLAlchemy model sources (model_dir): src/«package_name|replace(".","/")»

====================
 Create a new table
====================
[?] Schema name of the new table (schema_name): public
[?] Name of the new table (table_name): thangs
[?] Description of the new table (table_description): The table ``«schema_name».«table_name»`` contains...
[?] Name of the corresponding entity (entity_name): Thang

* Create file docs/database/public/tables/thangs.rst

* Create file docs/database/public/tables/thangs.sql

* Create file src/package/qualified/name/entities/public/thang.py

* Change file src/package/qualified/name/entities/public/__init__.py

  - insert “from .thang import T…” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”

  - add “mapper(Thang, t.tha…” after “## ⌃⌃⌃ tinject impo…”

* Create file src/package/qualified/name/tables/public/thangs.py

* Change file src/package/qualified/name/tables/public/__init__.py

  - insert “from .thangs import …” between “## ⌄⌄⌄ tinject impo…” and “## ⌃⌃⌃ tinject impo…”

Verify:

$ cat src/package/qualified/name/entities/public/__init__.py
# -*- coding: utf-8 -*-
# :Project:   package.qualified.name -- Entities in schema public
# :Created:   mer 15 giu 2016 13:24:54 CEST
# :Author:    Lele Gaifax <lele@example.com>
# :License:   GNU General Public License version 3 or later
# :Copyright: © 2016 Lele Gaifax
#

from sqlalchemy.orm import mapper

from ...tables import public as t

## ⌄⌄⌄ tinject import marker ⌄⌄⌄, please don't remove!
from .thang import Thang
from .thing import Thing

## ⌃⌃⌃ tinject import marker ⌃⌃⌃, please don't remove!

mapper(Thang, t.thangs, properties={
})

mapper(Thing, t.things, properties={
})

Changes

0.7 (2017-06-02)

0.6 (2017-03-22)

  • Minor tweak, no externally visible changes

0.5 (2016-11-07)

  • All steps support a “when” condition

0.4 (2016-06-16)

  • New changefile tweak to insert a line in a sorted block

0.3 (2016-05-22)

  • New ability to repeat a list of substeps

0.2 (2016-05-19)

  • First release on PyPI

0.1 (2016-04-26)

  • Initial effort
Release History

Release History

This version
History Node

0.7

History Node

0.6

History Node

0.5

History Node

0.4

History Node

0.3

History Node

0.2

History Node

0.1

History Node

0.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
metapensiero.tool.tinject-0.7.tar.gz (19.6 kB) Copy SHA256 Checksum SHA256 Source Jun 2, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting