Pandas-Oop

(Also known as Poop), is a package that uses Pandas dataframes with object oriented programming style

Installation:

  pip install pandas-oop

Some examples

from pandas_oop import models
from pandas_oop.fields import StringColumn, IntegerColumn, FloatColumn, DateColumn, BoolColumn

DB_CONNECTION = models.Connection('sqlite:///pandas_oop.db') # this is the same con_string for sqlalchemy engine

@models.sql(table='people', con=DB_CONNECTION) # Use this decorator if you want to connect your class to a database
@models.Data
class People(models.DataFrame):
    name = StringColumn(unique=True)
    age = IntegerColumn()
    money = FloatColumn(target_name="coins") # target_name if the name in the csv or table is coins and you want to have a different variable name
    insertion_date = DateColumn(format='%d-%m-%Y')
    is_staff = BoolColumn(true='yes', false='no')

Now when instantiating this class, it will return a custom dataframe with all the functionalities of a Pandas dataframe and some others

people = People()
"""-----------------------------------------------------------"""
people = People(from_csv=DATA_FILE, delimiter=";")
"""-----------------------------------------------------------"""
people = People(from_sql_query='select * from people')
"""-----------------------------------------------------------"""
people = People(from_df=some_dataframe)
"""-----------------------------------------------------------"""
people = People(from_iterator=some_function_that_yield_values)
"""-----------------------------------------------------------""" 
for people_chunk in People(from_csv=DATA_FILE, delimiter=";", chunksize=10):
    ...

example of function that yield values:

def some_function_that_yield_values():
    while something:
        ...
        yield name, age, money, insertion_date, is_staff

You can also save it to the database with the save() method (if the dtypes of the columns change, this will raise a ValidationError):

people.save()

You can upsert to the database and this will automatically look at the unique fields that were declared in the class

people.save(if_row_exists='update')
or
people.save(if_row_exists='ignore')

If you want to revalidate your dataframe (convert the columns dtypes to the type that was declared in the class), you can call the validate() method:

people.validate()

You can also validate from another class. For example, you can do something like this:

people = People(from_csv=DATA_FILE)
jobs = Jobs(from_sql_query='select * from jobs')
people_with_jobs = people.merge(jobs, on='name').validate(from_class=PeopleWithJobs)

This is the list of the overriten methods that return a pandas_oop custom dataframe

• 'isnull'
• 'head'
• 'abs'
• 'merge'
• 'loc' and dataframe slicing

I will add more and more methods on this list.

New features

Alembic Database migration support added:

• On your main application package, import Base (this is a declarative_base from sqlalchemy)

from pandas_oop import Base

• Add this configuration on the env.py file of your alembic config

from your_app import Base
target_metadata = Base.metadata

• And finaly, update your database url on your alembic.ini file