dynamic, declarative data transformations with automatic code generation

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for westandskif from gravatar.com westandskif

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License (The MIT License (MIT))
  • Author: Nikita Almakov
  • Maintainer: Nikita Almakov
  • Tags etl , converters , codegen , convtools
  • Requires: Python >=3.6
  • Provides-Extra: test , lint , doc
Classifiers
Report project as malware

Project description

convtools — write transformations as expressions, run them as Python

convtools lets you declare data transformations in plain Python, then compiles them into tiny, optimized Python functions at runtime. You keep your data in native iterables (lists, dicts, generators, CSV streams)—no heavy container required.

License codecov Tests status Docs status PyPI Twitter Downloads Python versions

Why pick convtools?

  • Stay in Python. Compose transformations as expressions: pipes, filters, joins, group‑bys, reducers, window functions, and more. Then call .gen_converter() to get a real Python function.
  • Stream‑friendly. Works directly on iterators and files; the Table helper processes CSV‑like data without loading everything into memory.
  • Powerful aggregations. Rich reducers (Sum, CountDistinct, MaxRow, ArraySorted, Dict*, TopK…) with per‑reducer where filters and defaults. Nested aggregations are first‑class.
  • Debuggable & inspectable. Print the generated code with debug=True or set global options via c.OptionsCtx. Works with pdb/pydevd.
  • Plays nicely with Pandas/Polars. It’s not a DataFrame; it’s a code‑generation layer. Use it when you want lean, composable transforms over native Python data.

Installation

pip install convtools

Documentation

convtools.readthedocs.io

Group by example

from convtools import conversion as c

input_data = [
    {"a": 5, "b": "foo"},
    {"a": 10, "b": "foo"},
    {"a": 10, "b": "bar"},
    {"a": 10, "b": "bar"},
    {"a": 20, "b": "bar"},
]

conv = (
    c.group_by(c.item("b"))
    .aggregate(
        {
            "b": c.item("b"),
            "a_first": c.ReduceFuncs.First(c.item("a")),
            "a_max": c.ReduceFuncs.Max(c.item("a")),
        }
    )
    .pipe(
        c.aggregate({
            "b_values": c.ReduceFuncs.Array(c.item("b")),
            "mode_a_first": c.ReduceFuncs.Mode(c.item("a_first")),
            "median_a_max": c.ReduceFuncs.Median(c.item("a_max")),
        })
    )
    .gen_converter()
)

assert conv(input_data) == {
    'b_values': ['foo', 'bar'],
    'mode_a_first': 10,
    'median_a_max': 15.0
}
Built-in reducers like c.ReduceFuncs.First
* Sum
* SumOrNone
* Max
* MaxRow
* Min
* MinRow
* Count
* CountDistinct
* First
* Last
* Average
* Median
* Percentile
* Mode
* TopK
* Array
* ArrayDistinct
* ArraySorted

DICT REDUCERS ARE IN FACT AGGREGATIONS THEMSELVES, BECAUSE VALUES GET REDUCED.
* Dict
* DictArray
* DictSum
* DictSumOrNone
* DictMax
* DictMin
* DictCount
* DictCountDistinct
* DictFirst
* DictLast

AND LASTLY YOU CAN DEFINE YOUR OWN REDUCER BY PASSING ANY REDUCE FUNCTION
OF TWO ARGUMENTS TO ``c.reduce``.

When should I reach for convtools?

  • You need composable transforms over native Python data (lists/dicts/generators/CSV), not a DataFrame.
  • You want to express business rules declaratively and generate fast, readable Python functions.
  • You need aggregations/joins/pipes that you can reuse across scripts and services.

Contributing

  • Star the repo and share use‑cases in Discussions -- it really helps.

  • To report a bug or suggest enhancements, please open an issue and/or submit a pull request.

  • Reporting a Security Vulnerability: see the security policy.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for westandskif from gravatar.com westandskif

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License (The MIT License (MIT))
  • Author: Nikita Almakov
  • Maintainer: Nikita Almakov
  • Tags etl , converters , codegen , convtools
  • Requires: Python >=3.6
  • Provides-Extra: test , lint , doc
Classifiers

Release history Release notifications | RSS feed

This version

1.17.0

1.16.0

1.15.0

1.14.8

1.14.7

1.14.6

1.14.5

1.14.4

1.14.3

1.14.2

1.14.1

1.14.0

1.13.2

1.13.1

1.13.0 yanked

Reason this release was yanked:

aggregate CSE optimizer is bugged

1.12.2

1.12.1

1.11.0

1.10.1

1.10.0

1.9.0

1.8.0

1.7.0

1.6.0

1.5.1

1.5.0

1.4.0

1.3.0

1.2.0

1.1.2

1.1.1

1.1.0

1.0.0

0.42.4

0.42.3

0.42.2

0.42.1

0.42.0

0.41.0

0.40.2

0.40.1

0.40.0

0.39.0

0.38.0

0.37.0

0.36.0

0.35.0

0.34.0

0.33.2

0.33.1

0.33.0

0.32.0

0.31.0

0.30.0

0.29.0

0.28.0

0.27.0

0.26.0

0.25.2

0.25.1

0.25.0

0.24.1

0.24.0

0.23.3

0.23.2

0.23.1

0.23.0

0.22.0

0.21.0

0.20.2

0.20.1

0.20.0 yanked

Reason this release was yanked:

inconsistent API

0.19.0

0.18.0

0.17.0

0.16.0

0.15.4

0.15.3

0.15.2

0.15.1

0.15.0

0.14.1

0.14.0

0.13.4

0.13.3

0.13.2

0.13.1

0.13.0

0.12.1

0.12.0

0.11.2

0.11.1

0.11.0

0.10.0

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.8.0

0.7.2

0.7.1

0.7.0

0.6.1

0.6.0

0.5.3

0.5.2

0.5.1

0.5.0

0.4.0

0.3.3

0.3.2

0.3.1

0.3.0

0.2.3

0.2.2

0.2.1

0.2.0

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

convtools-1.17.0.tar.gz (142.9 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

convtools-1.17.0-cp314-cp314t-win_amd64.whl (103.3 kB view details)

Uploaded CPython 3.14tWindows x86-64

convtools-1.17.0-cp314-cp314t-musllinux_1_2_x86_64.whl (110.4 kB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

convtools-1.17.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (110.7 kB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

convtools-1.17.0-cp314-cp314t-macosx_11_0_arm64.whl (99.9 kB view details)

Uploaded CPython 3.14tmacOS 11.0+ ARM64

convtools-1.17.0-cp314-cp314t-macosx_10_15_x86_64.whl (99.7 kB view details)

Uploaded CPython 3.14tmacOS 10.15+ x86-64

convtools-1.17.0-cp310-abi3-win_amd64.whl (102.8 kB view details)

Uploaded CPython 3.10+Windows x86-64

convtools-1.17.0-cp310-abi3-musllinux_1_2_x86_64.whl (104.4 kB view details)

Uploaded CPython 3.10+musllinux: musl 1.2+ x86-64

convtools-1.17.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (104.4 kB view details)

Uploaded CPython 3.10+manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

convtools-1.17.0-cp310-abi3-macosx_11_0_arm64.whl (99.5 kB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

convtools-1.17.0-cp310-abi3-macosx_10_9_x86_64.whl (99.2 kB view details)

Uploaded CPython 3.10+macOS 10.9+ x86-64

File details

Details for the file convtools-1.17.0.tar.gz.

File metadata

  • Download URL: convtools-1.17.0.tar.gz
  • Upload date:
  • Size: 142.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for convtools-1.17.0.tar.gz
Algorithm Hash digest
SHA256 da5a2c3e375c1b5e849d6ea2718cb5f65ad45e4f83623c44dd5ba150624d5ce5
MD5 52e06fb8d76435243c2bdade69938203
BLAKE2b-256 3e41696fbe0af880adb0d3db28e067f81768cf9962db2e45c4c6b7cf7a3a230d

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0.tar.gz:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp314-cp314t-win_amd64.whl.

File metadata

  • Download URL: convtools-1.17.0-cp314-cp314t-win_amd64.whl
  • Upload date:
  • Size: 103.3 kB
  • Tags: CPython 3.14t, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for convtools-1.17.0-cp314-cp314t-win_amd64.whl
Algorithm Hash digest
SHA256 378c37f28994e507c33f4e9c623bf1b7564479ce44a45a8d017f64c263d78e5f
MD5 3c4e881bdd5fb9d75111752250f0b1e2
BLAKE2b-256 0c2b8e3b99dc50c3dd44ac17fc6b2b2c4d19a0a57f3458e4ec12e5ee65e318b6

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp314-cp314t-win_amd64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 e67ac83d964133038ccdd4339b43f593c9d66e1ac3a010d4458256c960831047
MD5 c5f0decf3f52f32c5eac76826b78d67d
BLAKE2b-256 b8c813340682de4fb38ca3917b1f8fbf5f020405fc5adfddcac9b1531aed007f

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp314-cp314t-musllinux_1_2_x86_64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 c2e7848a35dbedf7f6ea17453b070a8a3178cd2b6255f9f44c6cf477deeae23f
MD5 5a1d6d51f3e451d22b20ce2430cd8173
BLAKE2b-256 738e5b916947d71bcd0ec5f4c5797ce7e819db2a75e4a8bef9009eff5336e1d3

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp314-cp314t-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp314-cp314t-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e0b83010425e1da72bbea4bfd76c67246a292a1c1e02305ee33cad4c3d86b010
MD5 6c46178acd23729b7fc068587aa8fa03
BLAKE2b-256 bf87e6384321a4fb28eab708c3076c02d124a2614aece9f47063f06b34fcfc07

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp314-cp314t-macosx_11_0_arm64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp314-cp314t-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp314-cp314t-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 9a7f63017ca68f73c4da4beded7e608e159f3fbbbb718859041cc125d99b7e7c
MD5 f552b56e888490e8f6ce58cf55e80119
BLAKE2b-256 ba17d475d603ecbc3173b0afc0bcab1b851e72fe91d56eea95334215c3072936

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp314-cp314t-macosx_10_15_x86_64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: convtools-1.17.0-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 102.8 kB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for convtools-1.17.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 9912a83eb7fd56246480ba25863c3c2f1ab298b05905e16725c0128988e8b2ba
MD5 dec7303c7a0f1c0807498a5dd754d95b
BLAKE2b-256 9f91d4c4263b0a1861ff8fcd9c3f261b49c015bcb7a75901deb1de9541d46612

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp310-abi3-win_amd64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp310-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp310-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 b9402f2e08293dec060e1191bc06abfdda35aea7c83292f3595ab63b41ebf54d
MD5 f81c213f929fd07a996bd9e35b38a6f7
BLAKE2b-256 aa911954e4c0aa901491d0d8a8e0a42487e128056c69c565b08e3574de21be06

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp310-abi3-musllinux_1_2_x86_64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 028362c74352ff092025e10cecded46a7a6e5565ca55abe591086e3fee92000f
MD5 a712c98d946b3475469130d3cc7ad105
BLAKE2b-256 cfc920b29324f495b7f8f06daf918cb47df6fe8b6f45c5bd2de2253353e557f9

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 15f76427ab09eed7dc11a69fc372d294d3bab3d49edd0d5d9158303442b76c7f
MD5 aa628d351b81b916f82c151f66eaedb5
BLAKE2b-256 08ef6036708e3196e6337c487f3fe2ea9c5e68be8cdfb348dd75427eafd66d78

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file convtools-1.17.0-cp310-abi3-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for convtools-1.17.0-cp310-abi3-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 d357ab641fe116223bc8b9532e9bf23a2adb3a399b391b241209910a649c1f71
MD5 746823e8966e8e8b95601e61ce7a1d81
BLAKE2b-256 abfcd5bd3925118255326fbf1b2ccfc13d224a1ff98d518e8a523aa4a4795509

See more details on using hashes here.

Provenance

The following attestation bundles were made for convtools-1.17.0-cp310-abi3-macosx_10_9_x86_64.whl:

Publisher: pytest.yml on westandskif/convtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page