pymrgeo 1.0.2

MrGeo (pronounced "Mister Geo") is an open source geospatial toolkit designed to provide raster-based geospatial processing capabilities performed at scale. MrGeo enables global geospatial big data image processing and analytics.

Origin

MrGeo was developed at the National Geospatial-Intelligence Agency (NGA) in collaboration with DigitalGlobe. The government has “unlimited rights” and is releasing this software to increase the impact of government investments by providing developers with the opportunity to take things in new directions. The software use, modification, and distribution rights are stipulated within the Apache 2.0 license.

Pull Requests

All pull request contributions to this project will be released under the Apache 2.0 license.

Software source code previously released under an open source license and then modified by NGA staff is considered a “joint work” (see 17 USC 101); it is partially copyrighted, partially public domain, and as a whole is protected by the copyrights of the non-government authors and must be released according to the terms of the original open source license.

MrGeo Overview

MrGeo (pronounced “Mister Geo”) is an open source geospatial toolkit designed to provide raster-based geospatial processing capabilities performed at scale. MrGeo enables global geospatial big data image processing and analytics.

MrGeo is built upon the Apache Spark distributed processing frarmework to leverage the storage and processing of 100’s of commodity computers. Functionally, MrGeo stores large raster datasets as a collection of individual tiles stored in Hadoop to enable large-scale data and analytic services. The co-location of data and analytics offers the advantage of minimizing the movement of data in favor of bringing the computation to the data; a more favorable compute method for Geospatial Big Data. This framework has enabled the servicing of terabyte scale raster databases and performed terrain analytics on databases exceeding 100’s of gigabytes in size.

MrGeo has been fully deployed and tested in Amazon EMR.

Unique features/solutions of MrGeo:

Scalable storage and processing of raster data Application ready data: data is stored in MrGeo in a format that is ready for computation, eliminating several data pre-processing steps from production workflows. A suite of robust Spark analytics that that include algebraic math operations, focal operations (e.g. slope and gaussian) A third generation data storage model that Maintains data locality via spatial indexing. An abstraction layer between the analytics and storage methods to enables a diverse set of cloud storage options such as HDFS, Accumulo, HBASE etc. A Map algebra interface that enables the development of custom algorithms in a simple scripting API A plugin architecture that facilitates a modular software development and deployment strategies Data and Analytic capabilities provisioned by OGC and REST service end points Exemplar MrGeo Use Cases:

Raster Storage and Provisioning: MrGeo has been used to store, index, tile, and pyramid multi-terabyte scale image databases. Once stored, this data is made available through a simple Tiled Map Services (TMS) and Web Mapping Services (WMS) and can be made available through GeoServer via a MrGeo plugin. Large Scale Batch Processing and Serving: MrGeo has been used to pre-compute global 1 ArcSecond (nominally 30 meters) elevation data (300+ GB) into derivative raster products : slope, aspect, relative elevation, terrain shaded relief (collectively terabytes in size), and Tobler and Pingel friction surfaces Global Computation of Cost Distance: Given all pub locations in OpenStreetMap, compute 2 hour drive times from each location. The full resolution is 1 ArcSecond (30 meters nominally)

Dependencies

It is recommended these dependencies are installed for the package to function properly:

Java 7 or higher Apache Spark GDAL 1.10.1 or higher Python 2.7 or higher py4j pip