Feature engineering on polars and pandas dataframes for machine learning!

PyPI Read the Docs GitHub GitHub last commit GitHub issues Build Binder

tubular implements pre-processing steps for tabular data commonly used in machine learning pipelines.

The transformers are compatible with scikit-learn Pipelines. Each has a transform method to apply the pre-processing step to data and a fit method to learn the relevant information from the data, if applicable.

The transformers in tubular are written in narwhals narwhals, so are agnostic between pandas and polars dataframes, and will utilise the chosen (pandas/polars) API under the hood.

There are a variety of transformers to assist with;

  • capping
  • dates
  • imputation
  • mapping
  • categorical encoding
  • numeric operations

Here is a simple example of applying capping to two columns;

import polars as pl

transformer = CappingTransformer(
    capping_values={"a": [10, 20], "b": [1, 3]},
)

test_df = pl.DataFrame({"a": [1, 15, 18, 25], "b": [6, 2, 7, 1], "c": [1, 2, 3, 4]})

transformer.transform(test_df)
# ->
# shape: (4, 3)
# ┌─────┬─────┬─────┐
# │ a   ┆ b   ┆ c   │
# │ --- ┆ --- ┆ --- │
# │ i64 ┆ i64 ┆ i64 │
# ╞═════╪═════╪═════╡
# │ 10  ┆ 3   ┆ 1   │
# │ 15  ┆ 2   ┆ 2   │
# │ 18  ┆ 3   ┆ 3   │
# │ 20  ┆ 1   ┆ 4   │
# └─────┴─────┴─────┘

Tubular also supports saving/reading transformers and pipelines to/from json format (goodbye .pkls!), which we demo below:

import polars as pl
from tubular.imputers import MeanImputer, MedianImputer
from sklearn.pipeline import Pipeline
from tubular.pipeline import dump_pipeline_to_json, load_pipeline_from_json

# Create a simple dataframe

df = pl.DataFrame({"a": [1, 5], "b": [10, None]})

# Add imputers
median_imputer = MedianImputer(columns=["b"])
mean_imputer = MeanImputer(columns=["b"])

# Create and fit the pipeline
original_pipeline = Pipeline(
    [("MedianImputer", median_imputer), ("MeanImputer", mean_imputer)]
)
original_pipeline = original_pipeline.fit(df)

# Dumping the pipeline to JSON
pipeline_json = dump_pipeline_to_json(original_pipeline)
pipeline_json

# Printed value:
# ->
# {
# 'MedianImputer': {
#     'tubular_version': '2.6.1',
#     'classname': 'MedianImputer',
#     'init': {
#          'columns': ['b'],
#          'copy': False,
#          'verbose': False,
#          'return_native': True,
#          'weights_column': None
#          },
#     'fit': {
#           'impute_values_': {'b': 10.0}
#           }
#      },
# 'MeanImputer': {
#      'tubular_version': '2.6.1',
#      'classname': 'MeanImputer',
#      'init': {
#          'columns': ['b'],
#          'copy': False,
#          'verbose': False,
#          'return_native': True,
#          'weights_column': None
#           },
#      'fit': {
#          'impute_values_': {
#          'b': 10.0
#          }
#     }
# }

# Load the pipeline from JSON
pipeline = load_pipeline_from_json(pipeline_json)

# Verify the reconstructed pipeline
print(pipeline)

# Printed value:
# Pipeline(steps=[('MedianImputer', MedianImputer(columns=['b'])),
#                 ('MeanImputer', MeanImputer(columns=['b']))])

We are currently in the process of rolling out support for polars lazyframes!

track our progress below:

polars_compatible pandas_compatible jsonable lazyframe_compatible
AggregateColumnsOverRowTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
AggregateRowsOverColumnTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ArbitraryImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
BetweenDatesTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
BooleanImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
CappingTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
CategoricalImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ColumnDtypeSetter :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
CompareTwoColumnsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DateDifferenceTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DatetimeComponentExtractor :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DatetimeInfoExtractor :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DatetimeSinusoidCalculator :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DifferenceTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ExtractStringComponentsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
GroupRareLevelsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
LowerCaseTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MappingTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MeanImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MeanResponseTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MedianImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ModeImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
NullIndicator :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
NumberImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
OneDKmeansTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :x:
OneHotEncodingTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
OutOfRangeNullTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
RatioTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
RemoveCharactersTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
RenameColumnsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
SetValueTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
StringContainsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
StringImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ToDatetimeTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
WhenThenOtherwiseTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:

Installation

The easiest way to get tubular is directly from pypi with;

pip install tubular

Documentation

The documentation for tubular can be found on readthedocs.

Instructions for building the docs locally can be found in docs/README.

Examples

We utilise doctest to keep valid usage examples in the docstrings of transformers in the package, so please see these for getting started!

Issues

For bugs and feature requests please open an issue.

Build and test

The test framework we are using for this project is pytest. To build the package locally and run the tests follow the steps below.

First clone the repo and move to the root directory;

git clone https://github.com/azukds/tubular.git
cd tubular

Next install tubular and development dependencies;

pip install . -r requirements-dev.txt

Finally run the test suite with pytest;

pytest

Contribute

tubular is under active development, we're super excited if you're interested in contributing!

See the CONTRIBUTING file for the full details of our working practices.

Project links

Data verified by PyPI on Aug 14, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for LVGIG from gravatar.com LVGIG

Credits

Author: Allianz UK Data Science Team

GitHub Statistics

Data verified by PyPI on Aug 14, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

BSD License (BSD 3-Clause License)

Requires

Python >=3.10

Provides Extra

dev

Tags

polars pandas feature engineering pre-processing machine learning ML data science data transforms pipeline sklearn DS

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tubular-4.0.1.tar.gz (254.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tubular-4.0.1-py3-none-any.whl (89.8 kB view details)

Uploaded Python 3

File details

Details for the file tubular-4.0.1.tar.gz.

File metadata

  • Download URL: tubular-4.0.1.tar.gz
  • Upload date:
  • Size: 254.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tubular-4.0.1.tar.gz
Algorithm Hash digest
SHA256 028dfe9c52066e15ed3ee204f582dc98c365d0c3f9cd54b9b74d98e8832ea87a
MD5 fde54b0b7a074a27bf416d529040e9a3
BLAKE2b-256 0784827269c4600cfbc7244064a039ac5858914eacdfeb155637531b411f9d4b

See more details on using hashes here.

Provenance

The following attestation bundles were made for tubular-4.0.1.tar.gz:

Publisher: release.yml on azukds/tubular

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tubular-4.0.1-py3-none-any.whl.

File metadata

  • Download URL: tubular-4.0.1-py3-none-any.whl
  • Upload date:
  • Size: 89.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tubular-4.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 08a13cc271a3be8b30fd26aff4f78af86d3cd41cedcbb11c47faa6f8e460e93f
MD5 aecd804a0cf5d9c9a7a5812c657cff6c
BLAKE2b-256 41ea1549e7f0e2ae8fda7f7bae0075b5352e592d111b83951a57223f1a85564e

See more details on using hashes here.

Provenance

The following attestation bundles were made for tubular-4.0.1-py3-none-any.whl:

Publisher: release.yml on azukds/tubular

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

4.0.1 This release

2 files

4.0.0

2 files

3.9.0

2 files

3.8.4

2 files

3.8.3

2 files

3.8.2

2 files

3.8.1

2 files

3.8.0

2 files

3.7.1

2 files

3.7.0

2 files

3.6.0

2 files

3.5.0

2 files

3.4.0

2 files

3.3.0

2 files

3.2.0

2 files

3.1.0

2 files

3.0.0

2 files

2.8.0

2 files

2.7.0

2 files

2.6.0

2 files

2.5.0

2 files

2.4.0

2 files

2.3.0

2 files

2.2.0

2 files

2.1.0

2 files

2.0.0

2 files

1.4.8

2 files

1.4.7

2 files

1.4.6

2 files

1.4.5

2 files

1.4.4

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.1

2 files

1.3.0

2 files

1.2.2

2 files

1.2.1

2 files

1.2.0

2 files

1.1.1

2 files

1.1.0

2 files

1.0.0

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.15

2 files

0.2.14

2 files

0.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page