planetarypy 0.18.2

Python Tools for Planetary Science

PlanetaryPy

Beta release: This will become (part of) the core package of the PlanetaryPy organisation.

Potential logo:

Install

pip install planetarypy

This will pull in these other dependencies and their dependencies:

tomlkit pandas pvl numpy python-dateutil tqdm lxml yarl hirise-tools kalasiris

Suggested standard abbreviations:

• Inside these docs the package will be called PLPY for brevity.
• A standard Python import could be: plp or plpy
  • because the last p in plp can be pronounced out, we consider these equivalent for human conversation and pronounce these "plippy".

General scope

First and foremost this package should provide support in working with planetary science data.

With working we mean:

• locating
• retrieving
• reading
• further processing

of data.

Locating

This library manages, via its PDS tools, multiple PDS3 index files per instrument that can be used for identifying data of interest. These index files are automatically downloaded and converted to the very performant (and cloud-ready) parquet file format.

Parquet is able to store advanced datatypes like nan-capable integer and full datetime objects, as opposed to HDF5.

Retrieving

The interface to getting data is via a path-retrieving function based on a PDS product-id. If that product-id is available locally, the path will be returned. If it is not, it will previously be downloaded, stored in a systematic fashion organized by mission and instrument, and then the local path will be returned.

Reading

For now, the library only returns the path to the object and the user needs to sort out the reading process. A recently funded NASA project Planetary Data Reader will be integrated here, so that basic reading into memory can be provided.

As such, we anticipate two classes of reading support:1. basic reading into numpy and/or xarray1. added reader functionality like basic plots and basic geospatial processing, as supported by interested parties

There will exist larger other packages that focus on working with a given instrument's data, in which case that package could become an affiliated package with the planetarypy GitHub organization, if so desired.

Further processing

In the future, additional frequently used procedures will be added to this library, e.g.

• frequently used GDAL/rasterio procedures
• frequently used SPICE operations
  • like surface illumination on a given body

PDS tools

Look at the Apps docs to see what pds.apps exist for easily getting PDS indexes. The find_index app is specifically useful when you don't know what index files exist.

So far, the following indexes are supported (but not necessarily all tested within PLPY):

• Cassini
  • ISS (all)
  • UVIS (all)
• MRO
  • CTX EDR
  • HiRISE
    • EDR, RDR, DTM
      • EDR index has a bug (as delivered by the team, reported), where I need to activate an existing fix for it.
• LRO
  • Diviner (DLRE)
    • EDR, RDR
  • LOLA
    • EDR, RDR

More indexes

More indexes of other instruments can be easily added by following the existing structure of what has been copied into your config at ~/.planetarypy_config.toml.

Please consider submitting a pull request for adding further PDS index files into the config file at its source: https://github.com/michaelaye/nbplanetary/blob/master/planetarypy/data/planetarypy_config.toml

Utils

Find something in Utils for working with NASA timestamps and a well working URL download function url_retrieve, among other stuff.

Experiment/Instrument Specific

So far, planetarypy supports CTX EDR and HiRISE RGB.NOMAP data. Look at the CTX and HiRISE pages for descriptions of classes for working with these data.

Bug reports

Please submit bug reports at https://github.com/michaelaye/nbplanetary/issues

How to use

Indexes

See PDS apps for more details.

from planetarypy.pds.apps import get_index

ctrindex = get_index("mro.ctx", "edr")
ctrindex.sample(5, random_state=42)  # setting random_state to always get same files for docs

Stored index is up-to-date.

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	VOLUME_ID	FILE_SPECIFICATION_NAME	ORIGINAL_PRODUCT_ID	PRODUCT_ID	IMAGE_TIME	INSTRUMENT_ID	INSTRUMENT_MODE_ID	LINE_SAMPLES	LINES	SPATIAL_SUMMING	...	SUB_SOLAR_LATITUDE	SUB_SPACECRAFT_LONGITUDE	SUB_SPACECRAFT_LATITUDE	SOLAR_DISTANCE	SOLAR_LONGITUDE	LOCAL_TIME	IMAGE_SKEW_ANGLE	RATIONALE_DESC	DATA_QUALITY_DESC	ORBIT_NUMBER
26491	MROX_0964	DATA/B17_016473_1740_XN_06S018W.IMG	4A_04_1050062800	B17_016473_1740_XN_06S018W	2010-01-31 02:01:44.947	CTX	NIFL	5056	52224	1	...	17.77	16.8	-5.91	246673937.3	45.19	14.83	89.6	Ride-along with HiRISE	OK	16473
63218	MROX_2077	DATA/F01_036289_1668_XN_13S296W.IMG	4A_04_10B603B000	F01_036289_1668_XN_13S296W	2014-04-24 04:14:02.581	CTX	NIFL	5056	9216	1	...	21.67	296.3	-13.17	240471585.2	120.81	15.5	90.2	Ride-along with HiRISE	OK	36289
108231	MROX_3472	DATA/K15_059422_2018_XN_21N035W.IMG	4A_04_112C030F00	K15_059422_2018_XN_21N035W	2019-03-31 15:54:32.383	CTX	NIFL	5056	7168	1	...	1.75	34.46	21.99	234469987.0	4.06	14.08	90.1	Ride-along with HiRISE	OK	59422
70723	MROX_2321	DATA/F09_039344_1423_XN_37S165W.IMG	4A_04_10C6018D00	F09_039344_1423_XN_37S165W	2014-12-18 04:29:08.464	CTX	NIFL	3776	8192	1	...	-24.5	165.27	-37.76	206661098.9	254.74	15.34	90.1	Terra Sirenum	OK	39344
28889	MROX_1056	DATA/B19_017097_1318_XN_48S126W.IMG	4A_04_1054040700	B19_017097_1318_XN_48S126W	2010-03-20 16:44:39.958	CTX	NIFL	5056	52224	1	...	23.24	126.77	-48.24	249141981.2	66.58	15.32	90.4	Terra Sirenum	OK	17097

5 rows × 51 columns

hirise_rdr = get_index("mro.hirise", "rdr")
hirise_rdr.sample(5, random_state=42)

Stored index is up-to-date.

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	VOLUME_ID	FILE_NAME_SPECIFICATION	INSTRUMENT_HOST_ID	INSTRUMENT_ID	OBSERVATION_ID	PRODUCT_ID	PRODUCT_VERSION_ID	TARGET_NAME	ORBIT_NUMBER	MISSION_PHASE_NAME	...	LINE_PROJECTION_OFFSET	SAMPLE_PROJECTION_OFFSET	CORNER1_LATITUDE	CORNER1_LONGITUDE	CORNER2_LATITUDE	CORNER2_LONGITUDE	CORNER3_LATITUDE	CORNER3_LONGITUDE	CORNER4_LATITUDE	CORNER4_LONGITUDE
68663	MROHR_0001	RDR/ESP/ORB_036800_036899/ESP_036809_2640/ESP_...	MRO	HIRISE	ESP_036809_2640	ESP_036809_2640_COLOR	1	MARS	36809	Extended Science Phase	...	801810.0	1260530.0	83.6808	237.701	83.6727	237.516	83.9234	236.565	83.9318	236.757
56127	MROHR_0001	RDR/ESP/ORB_030800_030899/ESP_030831_1335/ESP_...	MRO	HIRISE	ESP_030831_1335	ESP_030831_1335_RED	1	MARS	30831	Extended Science Phase	...	-5451830.0	11875900.0	-46.3981	38.0734	-46.406	37.9435	-46.1322	37.9086	-46.1242	38.0378
18294	MROHR_0001	RDR/ESP/ORB_011400_011499/ESP_011400_1680/ESP_...	MRO	HIRISE	ESP_011400_1680	ESP_011400_1680_COLOR	2	MARS	11400	Extended Science Phase	...	-1402640.0	-7165880.0	-12.0828	241.439	-12.0848	241.421	-11.8342	241.39	-11.8321	241.407
96217	MROHR_0001	RDR/ESP/ORB_049600_049699/ESP_049673_1000/ESP_...	MRO	HIRISE	ESP_049673_1000	ESP_049673_1000_COLOR	1	MARS	49673	Extended Science Phase	...	-247898.5	-2325763.5	-80.0898	96.4902	-80.0948	96.3953	-79.8123	95.909	-79.8074	96.0015
30823	MROHR_0001	RDR/ESP/ORB_018300_018399/ESP_018301_2505/ESP_...	MRO	HIRISE	ESP_018301_2505	ESP_018301_2505_COLOR	1	MARS	18301	Extended Science Phase	...	1078760.0	-4561720.0	70.0663	103.136	70.0631	103.073	70.3287	102.954	70.332	103.018

5 rows × 54 columns

Instrument tools

CTX

from planetarypy.ctx import CTXEDR

pid = ctrindex.sample().squeeze().PRODUCT_ID
pid

'B09_013187_1492_XN_30S303W'

ctxedr = CTXEDR(pid)

ctxedr.local_path

Path('/home/maye/big_drive/planetary_data/missions/mro/ctx/edr/B09_013187_1492_XN_30S303W/B09_013187_1492_XN_30S303W.IMG')

ctxedr

PRODUCT_ID: B09_013187_1492_XN_30S303W
URL: https://pds-imaging.jpl.nasa.gov/data/mro/mars_reconnaissance_orbiter/ctx/mrox_0817/data/B09_013187_1492_XN_30S303W.IMG
Local: /home/maye/big_drive/planetary_data/missions/mro/ctx/edr/B09_013187_1492_XN_30S303W/B09_013187_1492_XN_30S303W.IMG
Not downloaded yet.

ctxedr.download()

ctxedr

PRODUCT_ID: B09_013187_1492_XN_30S303W
URL: https://pds-imaging.jpl.nasa.gov/data/mro/mars_reconnaissance_orbiter/ctx/mrox_0817/data/B09_013187_1492_XN_30S303W.IMG
Local: /home/maye/big_drive/planetary_data/missions/mro/ctx/edr/B09_013187_1492_XN_30S303W/B09_013187_1492_XN_30S303W.IMG
Shape: (1, 7168, 5056)