Skip to main content

A Django library for importing CSVs and other structured data quickly using Django's ModelForm for validation and deserialisation into an instance.

Project description

django-model-import

PyPI version

Django Model Import is a light weight CSV importer built for speed.

It uses a standard Django ModelForm to parse each row, giving you a familiar API to work with for data validation and model instantiation. In most cases, if you already have a ModelForm for the ContentType you are importing you do not need to create an import specific form.

To present feedback to the end-user running the import you can easily generate a preview of the imported data by toggling the commit parameter.

It also provides some import optimized fields for ForeignKey's, allowing preloading all possible values, or caching each lookup as it occurs, or looking up a model where multiple fields are needed to uniquely identify a resource.

Installation

poetry add django-model-import

Quickstart

import djangomodelimport

class BookImporter(djangomodelimport.ImporterModelForm):
    name = forms.CharField()
    author = CachedChoiceField(queryset=Author.objects.all(), to_field='name')

    class Meta:
        model = Book
        fields = (
            'name',
            'author',
        )

with default_storage.open('books.csv', 'rb') as fh:
    data = fh.read().decode("utf-8")

# Use tablib
parser = djangomodelimport.TablibCSVImportParser(BookImporter)
headers, rows = parser.parse(data)

# Process
importer = djangomodelimport.ModelImporter(BookImporter)
preview = importer.process(headers, rows, commit=False)
errors = preview.get_errors()

if errors:
    print(errors)

importresult = importer.process(headers, rows, commit=True)
for result in importresult.get_results():
    print(result.instance)

Composite key lookups

Often a relationship cannot be referenced via a single unique string. For this we can use a CachedChoiceField with a CompositeLookupWidget. The widget looks for the values under the type and variant columns in the source CSV, and does a unique lookup with the field names specified in to_field, e.g. queryset.get(type__name=type, name=variant).

The results of each get are cached internally for the remainder of the import minimising any database access.

class AssetImporter(ImporterModelForm):
    site = djangomodelimport.CachedChoiceField(queryset=Site.objects.active(), to_field='ref')
    type = djangomodelimport.CachedChoiceField(queryset=AssetType.objects.filter(is_active=True), to_field='name')
    type_variant = djangomodelimport.CachedChoiceField(
        queryset=InspectionItemTypeVariant.objects.filter(is_active=True),
        required=False,
        widget=djangomodelimport.CompositeLookupWidget(source=('type', 'variant')),
        to_field=('type__name', 'name'),
    )
    contractor = djangomodelimport.CachedChoiceField(queryset=Contractor.objects.active(), to_field='name')

Flat related fields

Often you'll have a OneToOneField or just a ForeignKey to another model, but you want to be able to create/update that other model via this one. You can flatten all of the related model's fields onto this importer using FlatRelatedField.

class ClientImporter(ImporterModelForm):
    primary_contact = FlatRelatedField(
        queryset=ContactDetails.objects.all(),
        fields={
            'contact_name': {'to_field': 'name', 'required': True},
            'email': {'to_field': 'email'},
            'email_cc': {'to_field': 'email_cc'},
            'mobile': {'to_field': 'mobile'},
            'phone_bh': {'to_field': 'phone_bh'},
            'phone_ah': {'to_field': 'phone_ah'},
            'fax': {'to_field': 'fax'},
        },
    )

    class Meta:
        model = Client
        fields = (
            'name',
            'ref',
            'is_active',
            'account',

            'primary_contact',
        )

Tests

Run tests with python example/manage.py test testapp

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django_model_import-0.7.6.tar.gz (15.0 kB view details)

Uploaded Source

Built Distribution

django_model_import-0.7.6-py3-none-any.whl (18.3 kB view details)

Uploaded Python 3

File details

Details for the file django_model_import-0.7.6.tar.gz.

File metadata

  • Download URL: django_model_import-0.7.6.tar.gz
  • Upload date:
  • Size: 15.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/2.0.1 CPython/3.12.1 Darwin/24.3.0

File hashes

Hashes for django_model_import-0.7.6.tar.gz
Algorithm Hash digest
SHA256 95e4c859519938fd42eddf171dfdf27a6a8b8375c906f7b0b6c986531160f474
MD5 c13739719991d47e68680d722a0a74c2
BLAKE2b-256 4e3aa2fb3935ff4895c79399f8ea53fb6a7a387729ae512cf6202c2cfd718c3f

See more details on using hashes here.

File details

Details for the file django_model_import-0.7.6-py3-none-any.whl.

File metadata

File hashes

Hashes for django_model_import-0.7.6-py3-none-any.whl
Algorithm Hash digest
SHA256 4bae113cda54d20de4800ea53151638466cfa72719c1664933a23a0797744008
MD5 efed2f8412cb1c522f0f74e3f0924eb4
BLAKE2b-256 3193332266c4d53ef4d28334a052e1a0bafbe4e54afa884df96be4904c52f591

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page