A Django library for validating and process lines from a pandas DataFrame
Project description
Django Dataframe Processor
A django library for importing/processing files or other data that is read as a DataFrame
Setup
pip install django-dataframe-processor
Add dataframe_processor
to your Django settings:
INSTALLED_APPS = (
...
'dataframe_processor',
...
)
Usage
Create a class extending from DataframeProcessor
from dataframe_processor.processor import DataframeProcessor
from dataframe_processor.table_schema.schema import TableSchema, Column
from dataframe_processor.table_schema.validators import date_range_validator, choices_validator
from dateutil.relativedelta import relativedelta
from django.utils import timezone
class MyFileProcessor(DataframeProcessor):
today = timezone.now().date()
schema = TableSchema([
Column('name', label='Name*', ctype=str),
Column('document_number', label='Document*', ctype=int),
Column('birth_date', label='Date of birth*', ctype=pd.Timestamp, validators=[
date_range_validator(begin=today - relativedelta(years=120), end=today),
]),
Column('gender', label='Gender*', ctype=str, validators=[choices_validator(['M', 'F'])], required=False),
])
def clean_row(self, row):
row = super().clean_row(row)
# make any aditional validation that you may need here
# if anything goes wrong you may raise a FileProcessor.ProcessException
return row
def save(self, data):
# save your data to the database or do any other processing you may need
return 'Ok'
Then you can process your files:
import pandas as pd
import MyFileProcessor
df = pd.read_csv('mydata.csv')
processor = MyFileProcessor(df)
success = processor.process()
# the processor generates a report for each line
print(processor.report)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for django-dataframe-processor-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | f9696d24b6f436871dc5f7fa01f6fc661b0dfed9c30b79eff0ddf66370987e0b |
|
MD5 | fc73ba9c82cd5fb78d2f1d7ed488371b |
|
BLAKE2b-256 | a9d4c1d4e982f9ca7c4ff50213114a8cdb6ee81a9241911fcca22723b825caa3 |
Close
Hashes for django_dataframe_processor-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0e77fd078da838106afe59ec20a382316f78fef7f7b06797e0020d7937e80769 |
|
MD5 | 8250dbd0e6df460c4ca44bb76c98b82b |
|
BLAKE2b-256 | 4c4309b423427bf4f100da090a485aa41d62f749b3350a59a20d12019ed8ce43 |