Skip to main content
ETL2OSM
=======

Extract, Transform and Load to OpenStreetMap

[![Build Status](https://travis-ci.org/osmottawa/etl2osm.svg?branch=master)](https://travis-ci.org/osmottawa/etl2osm)
[![Coverage Status](https://coveralls.io/repos/osmottawa/etl2osm/badge.svg?branch=master&service=github)](https://coveralls.io/github/osmottawa/etl2osm?branch=master)

Features
--------

- Extracts & Loads data from multiple formats:

- Shapefile
- GeoJSON
- OSM
- KML (Coming Soon)

- Replace direction fields (NE > Northeast, SW > Southwest)
- Replace street suffix fields (AVE > Avenue, ST > Street)
- Change & remove attribute names
- Convert text fields into proper titlecase (ottawa ONTARIO > Ottawa Ontario)
- Transform data into WGS84 (EPSG:4326)


API Overview
------------

Step by Step doing a typical `Extract Transform Load` processing.

```python
>>> import etl2osm
>>> data = etl2osm.extract("infile.shp")
>>> data.transform("config.json")
>>> data.save("outfile.osm")
```

Doing the entire process in a single line

```python
>>> import etl2osm
>>> etl2osm.process("infile.shp", "config.json", "outfile.osm")
```

Command Line Interface
----------------------

Reading the a file, the standard output will be in a GeoJSON format.

```bash
$ etl2osm "infile.shp" --config "config.json" --outfile "outfile.osm"
```

See [Examples](https://github.com/osmottawa/etl2osm/tree/master/examples) for more information.

Making a Config.json
--------------------

Whenever you want to perform a transformation, include a config file in a JSON format.

### Road Data

```json
{
"conform": {
"addr:street": [
{"function": "direction", "field":"DIRECTION"},
{"function": "title", "field": "ST_NAME"},
{"function": "suffix", "field": "ST_EXT"}
],
"maxspeed": {"function": "mph", "field": "Speed_Limi"}
}
}
```

### Address Data

```json
{
"conform": {
"addr:number": {"int": "True", "field": "NUMBER_"},
"addr:street": [
{"function": "direction", "field": "PREDIR"},
{"function": "title", "field": "ST_NAME"},
{"function": "suffix", "field": "STSUFFIX"}
],
"addr:postcode": {"int": "True", "field": "ZIP_CODES"},
"addr:unit": "UNIT"
}
}
```

Installation
------------

### GDAL & Fiona

ETL2OSM depends on GDAL, Fiona & LXML; to install those packages using Windows.
Download the appropriate library from [Unofficial Windows Binaries for Python Extension Packages](http://www.lfd.uci.edu/~gohlke/pythonlibs/) matching your Python version (Python27/34) & 64/32bit.

```bash
$ cd packages
$ pip install lxml‑3.4.4‑cp27‑none‑win32.whl
$ pip install Fiona-1.6.2-cp27-none-win32.whl
$ pip install GDAL-1.11.3-cp27-none-win32.whl
```

### PyPi Install

Install ETL2OSM from PyPi's online packages.

```bash
$ pip install etl2osm
```

### GitHub Install

Install the latest version directly from Github.

```bash
$ git clone https://github.com/osmottawa/etl2osm/
$ cd etl2osm
$ pip install .
```

Metadata

Release files for etl2osm 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for etl2osm 0.2.0
File Size Uploaded Provenance Actions
etl2osm-0.2.0.tar.gz 13.5 kB Without provenance Details for etl2osm-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for etl2osm 0.2.0
File Interpreter ABI Platform Actions
etl2osm-0.2.0-py2.py3-none-any.whl Python 2, Python 3 none any Details for etl2osm-0.2.0-py2.py3-none-any.whl

Total release size: 27.6 kB

Release files / etl2osm-0.2.0.tar.gz

Download URL etl2osm-0.2.0.tar.gz
Size 13.5 kB
Tags Source
SHA-256 checksum
How to use checksums
7c69965873fceae815a44c76c37573242c5c3e5d35a1647d8240b54871a1c818
BLAKE2b-256 checksum
How to use checksums
1f7ecc80d880a045c44236879c8b4692c2c96878678f5c4baa8ee985e3fafe42
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / etl2osm-0.2.0-py2.py3-none-any.whl

Download URL etl2osm-0.2.0-py2.py3-none-any.whl
Size 14.1 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
6392640725b202653a3ddcb5efd9eb3a35779a5066732b1ea51e17e8c3109906
BLAKE2b-256 checksum
How to use checksums
0e350fef40889c84c3277d220c404e0c690bf7f1dadcd2d8461421f19ad2966a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for etl2osm 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files No provenance

0.0.2

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page