agro-data-intelligence 0.2.0

Agricultural data auditing, visualization and explainable machine learning framework

🌱 Agro-Data-Intelligence (v0.2.0)

Welcome to the agro-data-intelligence framework! This package is designed to make data auditing, cleaning, and Explainable Machine Learning (XAI) for agricultural datasets incredibly simple, robust, and professional.

🏆 Test Results & Reliability

We take data integrity seriously. This package is fully tested:

• Unit & Integration Tests: ✅ Passed (100% success rate)
• Real Dataset Tests: ✅ Passed (Handles missing, duplicate, and extreme bounds seamlessly)
• Code Coverage: 🛡️ 87% Code Coverage

---

📥 1. Installation & Setup

Install the package via PyPI and import it alongside pandas.

# Install from PyPI (Terminal)
pip install agro-data-intelligence

# In your Python script / Jupyter Notebook
import pandas as pd
import data_audit  # Registers the 'audit' accessor automatically

---

🔍 2. Scanning Data for Issues (scan)

The first step is to scan the dataset for missing values, duplicates, and outliers.

# Load your dataset
df = pd.read_csv("agricultural_data.csv")

# Scan the data
issues = df.audit.scan()

# View the issues
print(issues)

[!TIP] Advanced Scanning: Users can customize outlier detection:

df.audit.scan(outlier_method='zscore', zscore_thresh=2.5)
# Or using custom bounds for a specific column like pH level:
df.audit.scan(custom_bounds={'Soil_pH': (5.5, 7.5)})

---

🛠️ 3. Fixing the Data (fix)

Once the data is scanned, users can fix the issues. There are three modes:

A. Auto Mode (Default)

Automatically fills missing values (median/mode) and clips outliers.

df.audit.fix(mode='auto')

B. Suggest Mode

Doesn't change the data, but suggests what should be done.

suggestions = df.audit.fix(mode='suggest')
print(suggestions)

C. Manual Mode (Professional Syntax)

Allows the user to precisely fix data by using the sid (Specific ID) helper.

from data_audit import sid

df.audit.fix(
    mode="manual",
    fixes=[
        sid(10, 120.0),             # Fix Issue ID 10 -> Replace with 120.0
        sid("Soil_Type", "Loamy"),  # Fill missing values in 'Soil_Type' with 'Loamy'
        sid((15, "Age"), 99.0),     # Change row 15, column 'Age' directly to 99.0
        sid(12, "ignore")           # Explicitly ignore Issue ID 12
    ]
)

---

📈 4. Data Visualization (plot)

Generate professional visualizations for specific columns in your dataset effortlessly.

# Plot distribution of 'Crop_Yield'
df.audit.plot("Crop_Yield")

# Plot categories in 'Soil_Type'
df.audit.plot("Soil_Type")

---

📊 5. Generating Reports (summary & report)

Users can generate detailed statistical summary and audit reports.

# Quick statistical summary of the dataframe
print(df.audit.summary())

# Full audit report (Including ML status)
print(df.audit.report())

---

🤖 6. Machine Learning & AI (ml)

The package has an integrated Machine Learning module that automatically configures models based on your data.

Recommend a Model

Find the best model for your target variable based on dataset size and distribution:

print(df.audit.ml.recommend(target='Crop_Yield'))

Train a Model

# Tell the model what column you want to predict
df.audit.ml.train(target='Crop_Yield')

Evaluate the Model

See how well the model performed (Accuracy, F1-Score, R2 Score, RMSE, etc.).

print(df.audit.ml.evaluate())

Explain Predictions (XAI)

Explain why the model made a specific prediction for a specific row of data.

# Get a single row of data (e.g., the first farm in the dataset)
local_farm_data = df.iloc[0:1]

# Explain the prediction
explanation = df.audit.ml.explain(local_data=local_farm_data)
print(explanation)

---

👽 7. Finding Anomalies (anomaly)

Find completely weird or unusual rows in the dataset using Unsupervised Machine Learning (Isolation Forests).

# Get the rows that are considered severe anomalies
weird_farms = df.audit.anomaly()
print(weird_farms)

---

📜 8. Viewing History Log (history)

Users can see a trail of every action the auditor performed on their data.

print(df.audit.history())

📜 License

This project is licensed under the strict open-source AGPL-3.0 License. Any commercial online service utilizing this package must disclose their modified source code.