Get clean datasets from DataHerb to boost your data science and data analysis projects
Project description
The Python Package for DataHerb
A DataHerb Core Service to Create and Load Datasets.
Install
pip install dataherb
The DataHerb Command-Line Tool
Requires Python 3
The DataHerb cli provides tools to create dataset metadata, validate metadata, search dataset in flora, and download dataset.
Search and Download
Search by keyword
dataherb search covid19
# Shows the minimal metadata
Search by dataherb id
dataherb search -i covid19_eu_data
# Shows the full metadata
Download dataset by dataherb id
dataherb download covid19_eu_data
# Downloads this dataset: http://dataherb.io/flora/covid19_eu_data
Create Dataset Using Command Line Tool
We provide a template for dataset creation.
Before creating a dataset, it is recommended that the user reads the intro.
Use the following command line tool to create the metadata template.
dataherb create
Use DataHerb in Your Code
Load Data into DataFrame
# Load the package
from dataherb.flora import Flora
# Initialize Flora service
# The Flora service holds all the dataset metadata
dataherb = Flora()
# Search datasets with keyword(s)
geo_datasets = dataherb.search("geo")
print(geo_datasets)
# Get a specific file from a dataset and load as DataFrame
tz_df = dataherb.herb(
"geonames_timezone"
).leaves.get(
"dataset/geonames_timezone.csv"
).data
print(tz_df)
The DataHerb Project
What is DataHerb
DataHerb is an open data initiative to make the access of open datasets easier.
- A DataHerb or Herb is a dataset. A dataset comes with the data files, and the metadata of the data files.
- A DataHerb Leaf or Leaf is a data file in the DataHerb.
- A Flora is the combination of all the DataHerbs.
In many data projects, finding the right datasets to enhance your data is one of the most time consuming part. DataHerb adds flavor to your data project.
What is DataHerb Flora
We desigined the following workflow to share and index datasets.
This repository is being used for listing of datasets (Listings in DataHerb flora repository).
How to Add Your Dataset
Simply create a yml
file in the flora
folder to link to your dataset repository. Your dataset repository should have a .dataherb
folder and a metadata.yml
file in it.
The indexing part will be done by GitHub Actions.
Development
- Create a conda environment.
- Install requirements:
pip install -r requirements.txt
Documentation
The documentation for this package is located at docs
.
HISTORY.rst
is used to list changes of the package.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.