Skip to main content

HESTIA Utils

Install

  1. Install the module:
pip install hestia_earth.utils
  1. Add this to your environment variables:
API_URL=https://api.hestia.earth
WEB_URL=https://www.hestia.earth

Usage

  1. To download a file from the HESTIA API:
from hestia_earth.schema import SchemaType
from hestia_earth.utils.api import download_hestia

cycle = download_hestia('cycleId', SchemaType.CYCLE)
sandContent = download_hestia('sandContent', SchemaType.TERM)
  1. To search for a specific Node on HESTIA:
from hestia_earth.schema import SchemaType
from hestia_earth.utils.api import find_node_exact

source = find_node_exact(SchemaType.SOURCE, {'bibliography.title': 'My Bibliography'})
  1. To get a lookup table from local file system:
from hestia_earth.schema import SchemaType
from hestia_earth.utils.lookup import load_lookup

df = load_lookup('path/to/my/lookup.csv')
  1. To get a lookup table from HESTIA:
from hestia_earth.schema import SchemaType
from hestia_earth.utils.lookup import download_lookup

df = download_lookup('crop.csv')

LSRS Format

You can use this library to convert HESTIA data into the LSRS format.

convert() accepts Impact Assessment, Cycle, Site and Source nodes and returns one simplified dict per node:

  • Impact Assessment → the LSRS indicator columns (the accounting categories for all scopes and again for Scope 1, 2 and 3, gas types, removals, and the other impact categories) plus metadata (including land transformation and deforestation read from its indicators).
  • Cycle → yield, EVS, seed, pesticide, diesel, electricity, crop residues, and (using the nutrient/unit helpers in lsrs/models_utils.py) fertiliser kg N / kg P2O5 / kg K2O, lime kg CaCO3 and irrigation m3. These are the Cycle's own inputs and products: on an animal Cycle, fertiliser, lime and irrigation are the farm's pasture and forage inputs, never upstream feed, and seed and crop residues show - (not applicable).
  • Cycle, animal production (lsrs/animal.py, only for Cycles with animals or animal products; - on other rows) → animal type (the live animal behind the primary product), number of animals (average over the Cycle), liveweight and carcass weight produced (kg), milk (kg), eggs (kg), herd replacement rate (%), stocking density (animals per m2); feed as kg fed weight, kg dry matter, MJ and kg N (from the inputs' dryMatter, energyContentHigherHeatingValue and nitrogenContent or crudeProteinContent / 6.25 properties), feed conversion ratio (the feedConversionRatioDryMatter practice), feed produced on farm and grazed forage (% of dry matter); the five largest feeds by dry matter, each with its origin (the input's country or region, On farm when produced in the Cycle, else -) and share (% of dry matter), and the rest as other feed; system and manure management (the practice with the largest value), excreta (kg N). Feed inputs are those flagged isAnimalFeed; when none is (aggregated Cycles), the crop, forage, processed food and feed additive inputs count as feed.
  • Site → site type, country, region and current land cover.
  • Source → bibliography title and year.

These JSON outputs can be rendered to Excel or stored elsewhere. lsrs/models_utils.py holds nutrient/unit/land-cover helpers copied from hestia-earth-models so this library has no dependency on it.

For example, this will let you download all verified aggregation on HESTIA:

  1. Add the env variable API_ACCESS_TOKEN and use the API Key from your account.
  2. Create a file to store the search query:
{
  "bool": {
    "must": [
      { "match": { "@type": "ImpactAssessment" } },
      { "match": { "aggregatedDataValidated": true } }
    ]
  }
}
  1. Run python3 convert_to_lsrs.py --query-path test.json --data-state recalculated

This will create 2 files:

  • query-results-lsrs.json: contains the raw JSON data needed to generate the Excel file
  • query-results-lsrs.xlsx: cotains the LSRS data in Excel format.

Note: if you make some changes to the Excel generation, you can run this to use the json data and re-generate the Excel file quicker: python3 render_lsrs.py --filepath query-results-lsrs.json

Converting multiple files and rendering into a single Excel

You can use the convert script to create all the JSON files containing the raw data, then render everything together at once:

  1. Run all your conversion, using --skip-rendering parameter:
python3 convert_to_lsrs.py --query-path test1.json --data-state recalculated --skip-rendering
python3 convert_to_lsrs.py --query-path test1.json --data-state recalculated --skip-rendering
  1. This will write multiple JSON files in the samples folder. Once done, render all together: python3 render_lsrs.py --folder samples. This will create the final samples.xlsx file.

Metadata

Release files for hestia-earth-utils 0.18.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for hestia-earth-utils 0.18.0
File Size Uploaded
hestia_earth_utils-0.18.0.tar.gz 106.8 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for hestia-earth-utils 0.18.0
File Interpreter ABI Platform
hestia_earth_utils-0.18.0-py3-none-any.whl Python 3 none any Details

Total release size: 205.8 kB

Release files / hestia_earth_utils-0.18.0.tar.gz

Download URL hestia_earth_utils-0.18.0.tar.gz
Size 106.8 kB
Tags Source
SHA-256 checksum
How to use checksums
4639771a4d0344c09b06ba9b0a418745066eda6c12c30d9f6ff3f0573a4f0563
BLAKE2b-256 checksum
How to use checksums
b514a230a463e7c97940f6788f1aad9f2c140f8718b642300b87ce94e53dee4a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.15

Release files / hestia_earth_utils-0.18.0-py3-none-any.whl

Download URL hestia_earth_utils-0.18.0-py3-none-any.whl
Size 99.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2bccc21be9290adffb50889b01fb8ca4d265ad2e199a917b34b9c95cc0b0e205
BLAKE2b-256 checksum
How to use checksums
bbef5b2326f37e476203c47d4a59c85828208cf512c31511ce80241daadb578e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.15

Release history Release notifications | RSS feed

This release

0.18.0 This release

2 release files

0.17.5

2 release files

0.17.4

2 release files

0.16.9

2 release files

0.16.8

2 release files

0.16.7

2 release files

0.16.6

2 release files

0.16.5

2 release files

0.16.4

2 release files

0.16.3

2 release files

0.16.2

2 release files

0.16.1

2 release files

0.15.9

2 release files

0.15.3

2 release files

0.15.2

2 release files

0.15.1

2 release files

0.15.0

2 release files

0.14.7

2 release files

0.14.6

2 release files

0.14.5

2 release files

0.14.3

2 release files

0.14.2

2 release files

0.14.1

2 release files

0.14.0

2 release files

0.13.9

2 release files

0.13.8

2 release files

0.13.7

2 release files

0.13.6

2 release files

0.13.5

2 release files

0.13.3

2 release files

0.13.2

2 release files

0.13.1

2 release files

0.13.0

2 release files

0.12.7

2 release files

0.12.6

2 release files

0.12.5

2 release files

0.12.3

2 release files

0.12.2

2 release files

0.12.1

2 release files

0.12.0

2 release files

0.11.7

2 release files

0.11.6

2 release files

0.11.5

2 release files

0.11.4

2 release files

0.11.3

2 release files

0.11.2

2 release files

0.11.1

2 release files

0.11.0

2 release files

0.10.9

2 release files

0.10.8

2 release files

0.10.7

2 release files

0.10.6

2 release files

0.10.5

2 release files

0.10.1

2 release files

0.10.0

2 release files

0.9.2

2 release files

0.9.1

2 release files

0.9.0

2 release files

0.8.1

2 release files

0.8.0

2 release files

0.7.4

2 release files

0.7.3

2 release files

0.7.2

2 release files

0.7.1

2 release files

0.7.0

2 release files

0.6.1

2 release files

0.6.0

2 release files

0.5.1

2 release files

0.5.0

2 release files

0.4.10

2 release files

0.4.9

2 release files

0.4.8

2 release files

0.4.7

2 release files

0.4.6

2 release files

0.4.5

2 release files

0.4.4

2 release files

0.4.3

2 release files

0.4.2

2 release files

0.4.1

2 release files

0.4.0

2 release files

0.3.3

2 release files

0.3.2

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.8

2 release files

0.1.7

2 release files

0.1.6

2 release files

0.1.5

2 release files

0.1.4

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.1.0

2 release files

0.0.7

2 release files

0.0.6

2 release files

0.0.5

2 release files

0.0.4

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page