Skip to main content

Extract, Transform and Load to OpenStreetMap.

Project description

ETL2OSM
=======

Extract, Transform and Load to OpenStreetMap

[![Build Status](https://travis-ci.org/osmottawa/etl2osm.svg?branch=master)](https://travis-ci.org/osmottawa/etl2osm)
[![Coverage Status](https://coveralls.io/repos/osmottawa/etl2osm/badge.svg?branch=master&service=github)](https://coveralls.io/github/osmottawa/etl2osm?branch=master)

Features
--------

- Extracts & Loads data from multiple formats:

- Shapefile
- GeoJSON
- OSM
- KML (Coming Soon)

- Replace direction fields (NE > Northeast, SW > Southwest)
- Replace street suffix fields (AVE > Avenue, ST > Street)
- Change & remove attribute names
- Convert text fields into proper titlecase (ottawa ONTARIO > Ottawa Ontario)
- Transform data into WGS84 (EPSG:4326)


API Overview
------------

Step by Step doing a typical `Extract Transform Load` processing.

```python
>>> import etl2osm
>>> data = etl2osm.extract("infile.shp")
>>> data.transform("config.json")
>>> data.save("outfile.osm")
```

Doing the entire process in a single line

```python
>>> import etl2osm
>>> etl2osm.process("infile.shp", "config.json", "outfile.osm")
```

Command Line Interface
----------------------

Reading the a file, the standard output will be in a GeoJSON format.

```bash
$ etl2osm "infile.shp" --config "config.json" --outfile "outfile.osm"
```

See [Examples](https://github.com/osmottawa/etl2osm/tree/master/examples) for more information.

Making a Config.json
--------------------

Whenever you want to perform a transformation, include a config file in a JSON format.

### Road Data

```json
{
"conform": {
"addr:street": [
{"function": "direction", "field":"DIRECTION"},
{"function": "title", "field": "ST_NAME"},
{"function": "suffix", "field": "ST_EXT"}
],
"maxspeed": {"function": "mph", "field": "Speed_Limi"}
}
}
```

### Address Data

```json
{
"conform": {
"addr:number": {"int": "True", "field": "NUMBER_"},
"addr:street": [
{"function": "direction", "field": "PREDIR"},
{"function": "title", "field": "ST_NAME"},
{"function": "suffix", "field": "STSUFFIX"}
],
"addr:postcode": {"int": "True", "field": "ZIP_CODES"},
"addr:unit": "UNIT"
}
}
```

Installation
------------

### GDAL & Fiona

ETL2OSM depends on GDAL, Fiona & LXML; to install those packages using Windows.
Download the appropriate library from [Unofficial Windows Binaries for Python Extension Packages](http://www.lfd.uci.edu/~gohlke/pythonlibs/) matching your Python version (Python27/34) & 64/32bit.

```bash
$ cd packages
$ pip install lxml‑3.4.4‑cp27‑none‑win32.whl
$ pip install Fiona-1.6.2-cp27-none-win32.whl
$ pip install GDAL-1.11.3-cp27-none-win32.whl
```

### PyPi Install

Install ETL2OSM from PyPi's online packages.

```bash
$ pip install etl2osm
```

### GitHub Install

Install the latest version directly from Github.

```bash
$ git clone https://github.com/osmottawa/etl2osm/
$ cd etl2osm
$ pip install .
```

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

etl2osm-0.2.0.tar.gz (13.5 kB view details)

Uploaded Source

Built Distribution

etl2osm-0.2.0-py2.py3-none-any.whl (14.1 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file etl2osm-0.2.0.tar.gz.

File metadata

  • Download URL: etl2osm-0.2.0.tar.gz
  • Upload date:
  • Size: 13.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for etl2osm-0.2.0.tar.gz
Algorithm Hash digest
SHA256 7c69965873fceae815a44c76c37573242c5c3e5d35a1647d8240b54871a1c818
MD5 63cd94fdf6b0ae0a3a3330c6e04fb5bd
BLAKE2b-256 1f7ecc80d880a045c44236879c8b4692c2c96878678f5c4baa8ee985e3fafe42

See more details on using hashes here.

File details

Details for the file etl2osm-0.2.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for etl2osm-0.2.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 6392640725b202653a3ddcb5efd9eb3a35779a5066732b1ea51e17e8c3109906
MD5 9e58f532101e0c6f7019d8467f33719b
BLAKE2b-256 0e350fef40889c84c3277d220c404e0c690bf7f1dadcd2d8461421f19ad2966a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page