hestia-earth-utils 0.17.23

HESTIA's utils library

HESTIA Utils

Install

1. Install the module:

pip install hestia_earth.utils

2. Add this to your environment variables:

API_URL=https://api.hestia.earth
WEB_URL=https://www.hestia.earth

Usage

1. To download a file from the HESTIA API:

from hestia_earth.schema import SchemaType
from hestia_earth.utils.api import download_hestia

cycle = download_hestia('cycleId', SchemaType.CYCLE)
sandContent = download_hestia('sandContent', SchemaType.TERM)

2. To search for a specific Node on HESTIA:

from hestia_earth.schema import SchemaType
from hestia_earth.utils.api import find_node_exact

source = find_node_exact(SchemaType.SOURCE, {'bibliography.title': 'My Bibliography'})

3. To get a lookup table from local file system:

from hestia_earth.schema import SchemaType
from hestia_earth.utils.lookup import load_lookup

df = load_lookup('path/to/my/lookup.csv')

4. To get a lookup table from HESTIA:

from hestia_earth.schema import SchemaType
from hestia_earth.utils.lookup import download_lookup

df = download_lookup('crop.csv')

LSRS Format

You can use this library to convert HESTIA data into the LSRS format.

convert() accepts Impact Assessment, Cycle, Site and Source nodes and returns one simplified dict per node:

• Impact Assessment → the 88 LSRS indicator columns plus metadata (including land transformation and deforestation read from its indicators).
• Cycle → yield, EVS, seed, pesticide, diesel, electricity, crop residues, and (using the nutrient/unit helpers in lsrs/models_utils.py) fertiliser kg N / kg P2O5 / kg K2O, lime kg CaCO3 and irrigation m3.
• Site → site type, country, region and current land cover.
• Source → bibliography title and year.

These JSON outputs can be rendered to Excel or stored elsewhere. lsrs/models_utils.py holds nutrient/unit/land-cover helpers copied from hestia-earth-models so this library has no dependency on it.

For example, this will let you download all verified aggregation on HESTIA:

1. Add the env variable API_ACCESS_TOKEN and use the API Key from your account.
2. Create a file to store the search query:

{
  "bool": {
    "must": [
      { "match": { "@type": "ImpactAssessment" } },
      { "match": { "aggregatedDataValidated": true } }
    ]
  }
}

3. Run python3 convert_to_lsrs.py --query-path test.json --data-state recalculated

This will create 2 files:

• query-results-lsrs.json: contains the raw JSON data needed to generate the Excel file
• query-results-lsrs.xlsx: cotains the LSRS data in Excel format.

Note: if you make some changes to the Excel generation, you can run this to use the json data and re-generate the Excel file quicker: python3 render_lsrs.py --filepath query-results-lsrs.json

Converting multiple files and rendering into a single Excel

You can use the convert script to create all the JSON files containing the raw data, then render everything together at once:

1. Run all your conversion, using --skip-rendering parameter:

python3 convert_to_lsrs.py --query-path test1.json --data-state recalculated --skip-rendering
python3 convert_to_lsrs.py --query-path test1.json --data-state recalculated --skip-rendering

2. This will write multiple JSON files in the samples folder. Once done, render all together: python3 render_lsrs.py --folder samples. This will create the final samples.xlsx file.