pynorma 1.0.0a1

A smart tool for preprocessing messy tabular data.

PyNorma

This project is currently under construction

"You gotta do it, you can do it, but you just don't wanna do it."

PyNorma is a Python library that provides insights and automation for preprocessing messy, real-world tabular data. It's designed for data scientists, analysts, and anyone who's tired of the tedious task of cleaning up unstructured spreadsheets.

Key Features

• Smart Table Detection: Automatically finds the main data table within messy Excel or CSV files, ignoring surrounding comments and empty spaces.
• Advanced Preprocessing: Includes powerful tools like:
  • Flattener: Converts wide, multi-level header tables into a tidy, long format.
  • Atomizer: Splits cells with multiple values into distinct rows or columns.
  • Clarifier: Standardizes data based on a custom dictionary.
  • ...and more.
• Developer-Friendly: Designed by a lazy developer for lazy (but smart) developers.

Installation

pip install pynorma

Quickstart

Here's a simple example of reading a messy Excel file and automatically trimming it to the core data table.

from pynorma.io import parser
from pynorma.preprocessor import trimmer

# 1. Parse the file - PyNorma automatically detects the file type.
raw_df = parser.parse("examples/townbusiness1.csv")

# 2. Automatically trim the dataframe to the main table area.
clean_df = trimmer.trim_dataframe(raw_df, trim_mode="auto")

print("Successfully cleaned the dataframe!")
print(clean_df.head())

Author

nash-dir (https://github.com/nash-dir)

License

This project is licensed under the MIT License.