A data tool for CEMC.

These details have not been verified by PyPI

Project links

Project description

reki

A data tool for CEMC.

Load GRIB2 data using eccodes

or cfgrib.

Installation

Download the latest source code from GitHub and install using pip.

If you are using system python, such as apps/python/3.6.3/gnu on HPC CMA-PI,

please use --user option to install on user directory.

reki uses ecCodes to decode GRIB files

(which is needed by eccodes and cfgrib).

Please install ecCodes through conda or other package source.

Getting started

reki has several functions to help users to find one filed from a local GRIB 2 file.

load_message_from_file from reki.format.grib.eccodes returns a GRIB handler.

Users can use it to get attrs or values with functions from eccodes .

For example, load 850hPa temperature from GRAPES GFS and get values from the returned message.

>>> from reki.format.grib.eccodes import load_message_from_file

>>> t = load_message_from_file(

...     file_path="/g1/COMMONDATA/OPER/NWPC/GRAPES_GFS_GMF/Prod-grib/2020031721/ORIG/gmf.gra.2020031800105.grb2",

...     parameter="t",

...     level_type="isobaricInhPa",

...     level=850,

... )

>>> data = eccodes.codes_get_double_array(t, "values")

>>> data = data.reshape([720, 1440])

>>> data

array([[249.19234375, 249.16234375, 249.16234375, ..., 249.15234375,

    249.19234375, 249.14234375],

   [249.45234375, 249.45234375, 249.42234375, ..., 249.45234375,

    249.44234375, 249.44234375],

   [249.69234375, 249.68234375, 249.68234375, ..., 249.70234375,

    249.67234375, 249.68234375],

   ...,

   [235.33234375, 235.45234375, 235.62234375, ..., 235.47234375,

    235.63234375, 235.48234375],

   [235.78234375, 235.91234375, 235.64234375, ..., 235.80234375,

    235.72234375, 235.82234375],

   [235.66234375, 235.86234375, 235.82234375, ..., 235.85234375,

    235.68234375, 235.70234375]])

NOTE: Please release the handler using eccodes.codes_release manually.

eccodes engine also provides some functions to load array from GRIB2 file

in which GRIB2 message is loaded by eccodes-python

and converted into xarray.DataArray by nwpc-data.

WARNING: This feature is under construction.

>>> from reki.format.grib.eccodes import load_field_from_file

>>> load_field_from_file(

...     file_path="/sstorage1/COMMONDATA/OPER/NWPC/GRAPES_GFS_GMF/Prod-grib/2020031721/ORIG/gmf.gra.2020031800105.grb2",

...     parameter="t",

...     level_type="isobaricInhPa",

...     level=850,

... )

<xarray.DataArray 't' (latitude: 720, longitude: 1440)>

array([[249.19234375, 249.16234375, 249.16234375, ..., 249.15234375,

        249.19234375, 249.14234375],

       [249.45234375, 249.45234375, 249.42234375, ..., 249.45234375,

        249.44234375, 249.44234375],

       [249.69234375, 249.68234375, 249.68234375, ..., 249.70234375,

        249.67234375, 249.68234375],

       ...,

       [235.33234375, 235.45234375, 235.62234375, ..., 235.47234375,

        235.63234375, 235.48234375],

       [235.78234375, 235.91234375, 235.64234375, ..., 235.80234375,

        235.72234375, 235.82234375],

       [235.66234375, 235.86234375, 235.82234375, ..., 235.85234375,

        235.68234375, 235.70234375]])

Coordinates:

    time           datetime64[ns] 2020-03-18

    step           timedelta64[ns] 4 days 09:00:00

    valid_time     datetime64[ns] 2020-03-22T09:00:00

    isobaricInhPa  int64 850

  * latitude       (latitude) float64 89.88 89.62 89.38 ... -89.38 -89.62 -89.88

  * longitude      (longitude) float64 0.0 0.25 0.5 0.75 ... 359.2 359.5 359.8

Attributes:

    GRIB_edition:                    2

    GRIB_centre:                     babj

    GRIB_subCentre:                  0

    GRIB_tablesVersion:              4

    GRIB_localTablesVersion:         1

    GRIB_dataType:                   fc

    GRIB_dataDate:                   20200318

    GRIB_dataTime:                   0

    GRIB_validityDate:               20200322

    GRIB_validityTime:               900

    GRIB_step:                       105

    GRIB_stepType:                   instant

    GRIB_stepUnits:                  1

    GRIB_stepRange:                  105

    GRIB_endStep:                    105

    GRIB_name:                       Temperature

    GRIB_shortName:                  t

    GRIB_cfName:                     air_temperature

    GRIB_discipline:                 0

    GRIB_parameterCategory:          0

    GRIB_parameterNumber:            0

    GRIB_gridType:                   regular_ll

    GRIB_gridDefinitionDescription:  Latitude/longitude 

    GRIB_typeOfFirstFixedSurface:    pl

    GRIB_typeOfLevel:                isobaricInhPa

    GRIB_level:                      850

    GRIB_numberOfPoints:             1036800

    GRIB_missingValue:               9999

    GRIB_units:                      K

    long_name:                       Temperature

    units:                           K

Engines

reki loads GRIB2 file using eccodes by default and also supports cfgrib.

cfgrib

If you don't care about loading speed, please use cfgrib engine

with option engine="cfgrib".

Please install cfgrib before using this engine.

Read 850hPa temperature from a GRAEPS GFS grib2 file using shortName key t.

( shortName is an ecCodes key. )

>>> load_field_from_file(

...     file_path="/g1/COMMONDATA/OPER/NWPC/GRAPES_GFS_GMF/Prod-grib/2020031721/ORIG/gmf.gra.2020031800105.grb2",

...     parameter="t",

...     level_type="isobaricInhPa",

...     level=850,

...     engine="cfgrib",

... )

<xarray.DataArray 't' (latitude: 720, longitude: 1440)>

[1036800 values with dtype=float32]

Coordinates:

    time           datetime64[ns] ...

    step           timedelta64[ns] ...

    isobaricInhPa  int64 ...

  * latitude       (latitude) float64 89.88 89.62 89.38 ... -89.38 -89.62 -89.88

  * longitude      (longitude) float64 0.0 0.25 0.5 0.75 ... 359.2 359.5 359.8

    valid_time     datetime64[ns] ...

Attributes:

    GRIB_paramId:                             130

    GRIB_shortName:                           t

    GRIB_units:                               K

    GRIB_name:                                Temperature

    GRIB_cfName:                              air_temperature

    GRIB_cfVarName:                           t

    GRIB_dataType:                            fc

    GRIB_missingValue:                        9999

    GRIB_numberOfPoints:                      1036800

    GRIB_typeOfLevel:                         isobaricInhPa

    GRIB_NV:                                  0

    GRIB_stepUnits:                           1

    GRIB_stepType:                            instant

    GRIB_gridType:                            regular_ll

    GRIB_gridDefinitionDescription:           Latitude/longitude

    GRIB_Nx:                                  1440

    GRIB_iDirectionIncrementInDegrees:        0.25

    GRIB_iScansNegatively:                    0

    GRIB_longitudeOfFirstGridPointInDegrees:  0.0

    GRIB_longitudeOfLastGridPointInDegrees:   359.75

    GRIB_Ny:                                  720

    GRIB_jDirectionIncrementInDegrees:        0.25

    GRIB_jPointsAreConsecutive:               0

    GRIB_jScansPositively:                    0

    GRIB_latitudeOfFirstGridPointInDegrees:   89.875

    GRIB_latitudeOfLastGridPointInDegrees:    -89.875

    long_name:                                Temperature

    units:                                    K

    standard_name:                            air_temperature

If the field doesn't have a shortName, use dict for parameter.

Load a filed without shortName.

>>> load_field_from_file(

...     file_path="/g1/COMMONDATA/OPER/NWPC/GRAPES_GFS_GMF/Prod-grib/2020031721/ORIG/gmf.gra.2020031800105.grb2",

...     parameter={

...         "discipline": 0,

...         "parameterCategory": 2,

...         "parameterNumber": 225,

...     },

...     level_type="isobaricInhPa",

...     level=850,

... )

<xarray.DataArray 'paramId_0' (latitude: 720, longitude: 1440)>

[1036800 values with dtype=float32]

Coordinates:

    time           datetime64[ns] ...

    step           timedelta64[ns] ...

    isobaricInhPa  int64 ...

  * latitude       (latitude) float64 89.88 89.62 89.38 ... -89.38 -89.62 -89.88

  * longitude      (longitude) float64 0.0 0.25 0.5 0.75 ... 359.2 359.5 359.8

    valid_time     datetime64[ns] ...

Attributes:

    GRIB_paramId:                             0

    GRIB_dataType:                            fc

    GRIB_missingValue:                        9999

    GRIB_numberOfPoints:                      1036800

    GRIB_typeOfLevel:                         isobaricInhPa

    GRIB_NV:                                  0

    GRIB_stepUnits:                           1

    GRIB_stepType:                            instant

    GRIB_gridType:                            regular_ll

    GRIB_gridDefinitionDescription:           Latitude/longitude

    GRIB_Nx:                                  1440

    GRIB_iDirectionIncrementInDegrees:        0.25

    GRIB_iScansNegatively:                    0

    GRIB_longitudeOfFirstGridPointInDegrees:  0.0

    GRIB_longitudeOfLastGridPointInDegrees:   359.75

    GRIB_Ny:                                  720

    GRIB_jDirectionIncrementInDegrees:        0.25

    GRIB_jPointsAreConsecutive:               0

    GRIB_jScansPositively:                    0

    GRIB_latitudeOfFirstGridPointInDegrees:   89.875

    GRIB_latitudeOfLastGridPointInDegrees:    -89.875

    GRIB_discipline:                          0

    GRIB_parameterCategory:                   2

    GRIB_parameterNumber:                     225

    long_name:                                original GRIB paramId: 0

    units:                                    1

Examples

See example directory for more examples.

LICENSE

reki is licensed under GPL v3.0

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

2024.10.0

Oct 28, 2024

2024.4.0

Apr 9, 2024

This version

0.3.0

Nov 24, 2021

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

reki-0.3.0.tar.gz (51.4 kB view details)

Uploaded Nov 24, 2021 Source

Built Distribution

reki-0.3.0-py3-none-any.whl (77.5 kB view details)

Uploaded Nov 24, 2021 Python 3

File details

Details for the file reki-0.3.0.tar.gz.

File metadata

Download URL: reki-0.3.0.tar.gz
Upload date: Nov 24, 2021
Size: 51.4 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/0.0.0 importlib_metadata/4.8.1 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for reki-0.3.0.tar.gz
Algorithm	Hash digest
SHA256	`622a5ca70e04b30e0002e39e178687f0f40325ba8c463f68bc4840c712ad1bad`
MD5	`ce036ff2eba8f3f69be582a101598aaf`
BLAKE2b-256	`a7ce2d1855d894776435911d5177b4b0fc2131c261bd011050d91730c9b4082b`

See more details on using hashes here.

File details

Details for the file reki-0.3.0-py3-none-any.whl.

File metadata

Download URL: reki-0.3.0-py3-none-any.whl
Upload date: Nov 24, 2021
Size: 77.5 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/0.0.0 importlib_metadata/4.8.1 pkginfo/1.8.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for reki-0.3.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`9d04c00a6ab7613131786924bf80ad33a945c7c700b08246eee457966886bbf6`
MD5	`573bf7006d6568368b76467cb7005f43`
BLAKE2b-256	`0db6ff17aeeb891a1524e36541577188551ce269604ecba2f6540f901c0e5705`