Feature engineering on polars and pandas dataframes for machine learning!

PyPI Read the Docs GitHub GitHub last commit GitHub issues Build Binder

tubular implements pre-processing steps for tabular data commonly used in machine learning pipelines.

The transformers are compatible with scikit-learn Pipelines. Each has a transform method to apply the pre-processing step to data and a fit method to learn the relevant information from the data, if applicable.

The transformers in tubular are written in narwhals narwhals, so are agnostic between pandas and polars dataframes, and will utilise the chosen (pandas/polars) API under the hood.

There are a variety of transformers to assist with;

  • capping
  • dates
  • imputation
  • mapping
  • categorical encoding
  • numeric operations

Here is a simple example of applying capping to two columns;

import polars as pl

transformer = CappingTransformer(
    capping_values={"a": [10, 20], "b": [1, 3]},
)

test_df = pl.DataFrame({"a": [1, 15, 18, 25], "b": [6, 2, 7, 1], "c": [1, 2, 3, 4]})

transformer.transform(test_df)
# ->
# shape: (4, 3)
# ┌─────┬─────┬─────┐
# │ a   ┆ b   ┆ c   │
# │ --- ┆ --- ┆ --- │
# │ i64 ┆ i64 ┆ i64 │
# ╞═════╪═════╪═════╡
# │ 10  ┆ 3   ┆ 1   │
# │ 15  ┆ 2   ┆ 2   │
# │ 18  ┆ 3   ┆ 3   │
# │ 20  ┆ 1   ┆ 4   │
# └─────┴─────┴─────┘

Tubular also supports saving/reading transformers and pipelines to/from json format (goodbye .pkls!), which we demo below:

import polars as pl
from tubular.imputers import MeanImputer, MedianImputer
from sklearn.pipeline import Pipeline
from tubular.pipeline import dump_pipeline_to_json, load_pipeline_from_json

# Create a simple dataframe

df = pl.DataFrame({"a": [1, 5], "b": [10, None]})

# Add imputers
median_imputer = MedianImputer(columns=["b"])
mean_imputer = MeanImputer(columns=["b"])

# Create and fit the pipeline
original_pipeline = Pipeline(
    [("MedianImputer", median_imputer), ("MeanImputer", mean_imputer)]
)
original_pipeline = original_pipeline.fit(df)

# Dumping the pipeline to JSON
pipeline_json = dump_pipeline_to_json(original_pipeline)
pipeline_json

# Printed value:
# ->
# {
# 'MedianImputer': {
#     'tubular_version': '2.6.1',
#     'classname': 'MedianImputer',
#     'init': {
#          'columns': ['b'],
#          'copy': False,
#          'verbose': False,
#          'return_native': True,
#          'weights_column': None
#          },
#     'fit': {
#           'impute_values_': {'b': 10.0}
#           }
#      },
# 'MeanImputer': {
#      'tubular_version': '2.6.1',
#      'classname': 'MeanImputer',
#      'init': {
#          'columns': ['b'],
#          'copy': False,
#          'verbose': False,
#          'return_native': True,
#          'weights_column': None
#           },
#      'fit': {
#          'impute_values_': {
#          'b': 10.0
#          }
#     }
# }

# Load the pipeline from JSON
pipeline = load_pipeline_from_json(pipeline_json)

# Verify the reconstructed pipeline
print(pipeline)

# Printed value:
# Pipeline(steps=[('MedianImputer', MedianImputer(columns=['b'])),
#                 ('MeanImputer', MeanImputer(columns=['b']))])

We are currently in the process of rolling out support for polars lazyframes!

track our progress below:

polars_compatible pandas_compatible jsonable lazyframe_compatible
AggregateColumnsOverRowTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
AggregateRowsOverColumnTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ArbitraryImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
BetweenDatesTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
BooleanImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
CappingTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
CategoricalImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ColumnDtypeSetter :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
CompareTwoColumnsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DateDifferenceTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DatetimeComponentExtractor :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DatetimeInfoExtractor :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DatetimeSinusoidCalculator :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
DifferenceTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ExtractStringComponentsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
GroupRareLevelsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
LowerCaseTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MappingTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MeanImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MeanResponseTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
MedianImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ModeImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
NullIndicator :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
NumberImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
OneDKmeansTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :x:
OneHotEncodingTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
OutOfRangeNullTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
RatioTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
RemoveCharactersTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
RenameColumnsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
SetValueTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
StringContainsTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
StringImputer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
ToDatetimeTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:
WhenThenOtherwiseTransformer :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: :heavy_check_mark:

Installation

The easiest way to get tubular is directly from pypi with;

pip install tubular

Documentation

The documentation for tubular can be found on readthedocs.

Instructions for building the docs locally can be found in docs/README.

Examples

We utilise doctest to keep valid usage examples in the docstrings of transformers in the package, so please see these for getting started!

Issues

For bugs and feature requests please open an issue.

Build and test

The test framework we are using for this project is pytest. To build the package locally and run the tests follow the steps below.

First clone the repo and move to the root directory;

git clone https://github.com/azukds/tubular.git
cd tubular

Next install tubular and development dependencies;

pip install . -r requirements-dev.txt

Finally run the test suite with pytest;

pytest

Contribute

tubular is under active development, we're super excited if you're interested in contributing!

See the CONTRIBUTING file for the full details of our working practices.

Project links

Data verified by PyPI on Aug 14, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (4.0.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for LVGIG from gravatar.com LVGIG

Credits

Author: Allianz UK Data Science Team

GitHub Statistics

Data verified by PyPI on Aug 14, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

BSD License (BSD 3-Clause License)

Requires

Python >=3.10

Provides Extra

dev

Tags

polars pandas feature engineering pre-processing machine learning ML data science data transforms pipeline sklearn DS

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tubular-4.0.0.tar.gz (254.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tubular-4.0.0-py3-none-any.whl (89.8 kB view details)

Uploaded Python 3

File details

Details for the file tubular-4.0.0.tar.gz.

File metadata

  • Download URL: tubular-4.0.0.tar.gz
  • Upload date:
  • Size: 254.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tubular-4.0.0.tar.gz
Algorithm Hash digest
SHA256 902829c68e0044ccb7ebae32c63eaa144ce9aeb0896c4fd7225d9efae62b425a
MD5 443a5dc33006d22a18e51b23d927196a
BLAKE2b-256 d8b36bc9761f9ed6020ae689148fad2645003a554ead175cdfe158a62bc9baa6

See more details on using hashes here.

Provenance

The following attestation bundles were made for tubular-4.0.0.tar.gz:

Publisher: release.yml on azukds/tubular

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file tubular-4.0.0-py3-none-any.whl.

File metadata

  • Download URL: tubular-4.0.0-py3-none-any.whl
  • Upload date:
  • Size: 89.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tubular-4.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f8f24914aa478eb0a8af041517609fe20b522bdf76677dec1ecfb6a0c438acf7
MD5 945bda46d068b1a3ab840999b2db058a
BLAKE2b-256 4a3fe118a44413c96c5b220d92501213d5e5cacd4415cb73344f014cec12ee59

See more details on using hashes here.

Provenance

The following attestation bundles were made for tubular-4.0.0-py3-none-any.whl:

Publisher: release.yml on azukds/tubular

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

4.0.1

2 files

This release

4.0.0 This release

2 files

3.9.0

2 files

3.8.4

2 files

3.8.3

2 files

3.8.2

2 files

3.8.1

2 files

3.8.0

2 files

3.7.1

2 files

3.7.0

2 files

3.6.0

2 files

3.5.0

2 files

3.4.0

2 files

3.3.0

2 files

3.2.0

2 files

3.1.0

2 files

3.0.0

2 files

2.8.0

2 files

2.7.0

2 files

2.6.0

2 files

2.5.0

2 files

2.4.0

2 files

2.3.0

2 files

2.2.0

2 files

2.1.0

2 files

2.0.0

2 files

1.4.8

2 files

1.4.7

2 files

1.4.6

2 files

1.4.5

2 files

1.4.4

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.1

2 files

1.3.0

2 files

1.2.2

2 files

1.2.1

2 files

1.2.0

2 files

1.1.1

2 files

1.1.0

2 files

1.0.0

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.15

2 files

0.2.14

2 files

0.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page