django-polars-tools 0.1.0

Tools for converting Django QuerySets to Polars DataFrames with correct schema handling, nullable field support, and high-performance data extraction.

django-polars-tools

Utilities for integrating Django and Polars, including safe QuerySet → DataFrame conversion, correct schema inference, and nullable field handling.

This package solves the common issue where Polars incorrectly infers nullable fields when converting Django QuerySets, especially when using infer_schema_length. django_polars_tools provides reliable schema handling and high-performance data extraction from Django models.

🚀 Features

• Safe QuerySet → Polars DataFrame conversion
• Correct handling of nullable fields
• Improved schema inference compared to Polars defaults
• Fast extraction path for large querysets
• Simple API designed to “just work”
• Django-friendly, Polars-native

More features will be added as the project grows toward deeper Django ↔ Polars interoperability.

📦 Installation

pip install django_polars_tools

Or with uv:

uv add django_polars_tools

📚 Usage

Basic Example

Convert any Django QuerySet to a Polars DataFrame:

from django_polars_tools import django_queryset_to_dataframe
from myapp.models import MyModel

# Get a queryset
queryset = MyModel.objects.filter(active=True)

# Convert to Polars DataFrame
df = django_queryset_to_dataframe(queryset)

With Annotations

Works seamlessly with Django annotations:

from django.db.models import Count, F
from django_polars_tools import django_queryset_to_dataframe

queryset = MyModel.objects.annotate(
    total=Count('items'),
    display_name=F('first_name') + ' ' + F('last_name')
)

df = django_queryset_to_dataframe(queryset)

Using .values()

Also supports QuerySets with .values():

queryset = MyModel.objects.values('id', 'name', 'created_at')
df = django_queryset_to_dataframe(queryset)

Custom Field Mapping

Override the default Django → Polars type mapping:

import polars as pl
from django.db import models
from django_polars_tools import django_queryset_to_dataframe, DJANGO_MAPPING

# Create custom mapping
custom_mapping = DJANGO_MAPPING.copy()
custom_mapping[models.JSONField] = pl.Object  # Store JSON as Object instead of String

df = django_queryset_to_dataframe(queryset, mapping=custom_mapping)

Additional Polars Options

Pass any pl.read_database() kwargs:

df = django_queryset_to_dataframe(
    queryset,
    batch_size=10000,
    # Other polars.read_database options...
)

📝 Why this library?

Polars' schema inference works great for many cases, but with Django querysets it can:

• infer nullable fields incorrectly
• misclassify types with limited sample size

This library provides consistent handling tailored for the Django ORM.

🤝 Contributing

Contributions are welcome!

Open an issue or submit a PR if you’d like to help improve the project.