PyRasterFrames

PyRasterFrames enables access and processing of geospatial raster data in PySpark DataFrames.

Getting started

The quickest way to get started is to pip install the pyrasterframes package.

pip install pyrasterframes

You can then access a pyspark SparkSession using the local[*] master in your python interpreter as follows.

from pyrasterframes.utils import create_rf_spark_session
spark = create_rf_spark_session()

Then you can read a raster and do some work with it.

from pyrasterframes.rasterfunctions import *
from pyspark.sql.functions import lit
# Read a MODIS surface reflectance granule
df = spark.read.raster('https://modis-pds.s3.amazonaws.com/MCD43A4.006/11/08/2019059/MCD43A4.A2019059.h11v08.006.2019072203257_B02.TIF')
# Add 3 element-wise, show some rows of the dataframe
df.select(rf_local_add(df.tile, lit(3))).show(5, False)

Support

Reach out to us on gitter!

Issue tracking is through github.

Contributing

Community contributions are always welcome. To get started, please review our contribution guidelines, code of conduct, and developer's guide. Reach out to us on gitter so the community can help you get started!

Development environment setup

For best results, we suggest using conda and the conda-forge channel to install the compiled dependencies before installing the packages in setup.py. Assuming you're in the same directory as this file:

conda create -n rasterframes python==3.7
conda install --file ./requirements-condaforge.txt

Then you can install the source dependencies:

pip install -e .