knockoff 4.1.1

Library for generating and bootstrapping mock data

Knockoff Factory

A library for generating mock data and creating database fixtures that can be used for unit testing.

Table of content

• Installation
• Changelog
• Documentation
• Unit Tests
• Future Work
• Legacy YAML Based CLI

Installation

pip install knockoff

Changelog

See the changelog for a history of notable changes to knockoff.

Documentation

We are working on adding more documentation and examples!

• Knockoff SDK
  • KnockoffTable
  • KnockoffDB
• TempDatabaseService
• Knockoff CLI
• Unit Testing Example: Sample App

Unit Tests

Prerequisites

• docker
• poetry (curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python)

Some of the unit tests depend on a database instance. Knockoff will create ephemeral databases within that instance and clean them up when tests have completed. By default it will attempt to connect to an existing instance at postgresql://postgres@localhost:5432/postgres and will create and destroy databases per test. This postgres location can be overridden with the KNOCKOFF_TEST_DB_URI environment variable.

If no external postgres instance is available for testing, but postgresql is installed, the TEST_USE_EXTERNAL_DB environment variable can be set to 0. The fixtures will then rely on the testing.postgresql library to create ephemeral postgres instances per fixture.

If postgres is not available, dependent tests can be disabled with the following: export TEST_POSTGRES_ENABLED=0.

Create the database instance using docker:

docker run --rm  --name pg-docker -e POSTGRES_HOST_AUTH_METHOD=trust -d -p 5432:5432  postgres:11.9

Install poetry:

poetry install

Run unit test:

poetry run pytest

Future work

• Further documentation and examples for SDK
• Add yaml based configuration for SDK
• Make extensible generic output for KnockffDB.insert (csv, parquet, etc)
• Enable append option for KnockoffDB.insert
• Autodiscover and populate all tables by using reflection and building dependency graph with foreign key relationships
• Parallelize execution of dag. (e.g. https://ipython.org/ipython-doc/stable/parallel/dag_dependencies.html)