Skip to main content

Python package to detect suspicious OpenStreetMap changesets

Project description

osmcha
=======

OSM Changeset Analyser, ``osmcha``, is a Python package to detect suspicious OSM changesets.
It was designed to be used with `osmcha-django <https://github.com/willemarcel/osmcha-django>`_,
but also can be used standalone or in other projects.

.. image:: https://badge.fury.io/py/osmcha.svg
:target: http://badge.fury.io/py/osmcha

.. image:: https://travis-ci.org/willemarcel/osmcha.svg
:target: https://travis-ci.org/willemarcel/osmcha

.. image:: https://coveralls.io/repos/willemarcel/osmcha/badge.svg
:target: https://coveralls.io/r/willemarcel/osmcha

Installation
============

pip install osmcha

Usage
=====

Python Library
--------------

You can read a `replication changeset file <https://planet.openstreetmap.org/replication/changesets/>`_
directly from the web:

.. code-block:: python

c = ChangesetList('https://planet.openstreetmap.org/replication/changesets/002/236/374.osm.gz')

or from your local filesystem.

.. code-block:: python

c = ChangesetList('tests/245.osm.gz')


``c.changesets`` will return a list containing data of all the changesets listed in the file.

You can filter the changesets passing a `GeoJSON` file with a polygon with your
interest area to `ChangesetList` as the second argument.

Finally, to analyse an especific changeset, do:

.. code-block:: python

ch = Analyse(changeset_id)
ch.full_analysis()

Customizing Detection Rules
~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can customize the detection rules by defining your prefered values when
initializing the ``Analyze`` class. See below the default values.

.. code-block:: python

ch = Analyse(changeset_id, create_threshold=200, modify_threshold=200,
delete_threshold=30, percentage=0.7, top_threshold=1000,
suspect_words=[...], illegal_sources=[...], excluded_words=[...])

Command Line Interface
----------------------

The command line interface can be used to verify an especific changeset directly
from the terminal.

Usage: ``osmcha <changeset_id>``

Detection Rules
===============

``osmcha`` works by analysing how many map features the changeset created, modified
or deleted, and by verifying the presence of some suspect words in the ``comment``,
``source`` and ``imagery_used`` fields of the changeset. Furthermore, we also
consider if the software editor used allows to import data or to do mass edits.
We consider ``powerfull editors``: JOSM, Merkaartor, level0, QGIS and ArcGis.

In the ``Usage`` section, you can see how to customize some of these detection rules.

Possible Import
---------------

We tag a changeset as a ``possible import`` if the number of created elements is
greater than 70% of the sum of elements created, modified and deleted and if it
creates more than 1000 elements or 200 elements case it used one of the ``powerfull editors``.

Mass Modification
-----------------

We consider a changeset as a ``mass modification`` if the number of modified elements
is greater than 70% of the sum of elements created, modified and deleted and if it
modifies more than 200 elements.

Mass Deletion
-------------

All changesets that delete more than 1000 elements are considered a ``mass deletion``.
If the changeset deletes between 200 and 1000 elements and the number of deleted
elements is greater than 70% of the sum of elements created, modified and deleted
it's also tagged as a ``mass deletion``.

Suspect words
-------------

The suspect words are loaded from a `yaml file <osmcha/suspect_words.yaml>`_.
You can customize the words by setting another default file with a environment
variable:

.. code-block:: console
export SUSPECT_WORDS=<path_to_the_file>

or pass a list of words to the ``Analyse`` class, more information on the section
``Customizing Detection Rules``. We use a list of illegal sources to analyse the
``source`` and ``imagery_used`` fields and another more general list to examine
the comment field. We have also a list of excluded words to avoid false positives.


Unknown iD instance
-------------------

Verify the changesets made in iD editor to check the host instance. The trusted
iD instances are: `OSM.org <http://osm.org/>`_, `Strava <https://strava.github.io/iD/>`_
and `ImproveOSM <http://improveosm.org>`_.


Tests
======

To run the tests on `osmcha`:

.. code-block:: console

git clone https://github.com/willemarcel/osmcha.git
cd osmcha
pip install -e .[test]
py.test -v

Changelog
=========

Check `CHANGELOG <CHANGELOG.rst>`_ for the version history.

Contributors
============

* Wille Marcel
* Sanjay Banghar
* Bhargav Kowshik

License
=======

GPLv3

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

osmcha-0.3.7.tar.gz (27.1 kB view details)

Uploaded Source

Built Distribution

osmcha-0.3.7-py2.py3-none-any.whl (11.2 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file osmcha-0.3.7.tar.gz.

File metadata

  • Download URL: osmcha-0.3.7.tar.gz
  • Upload date:
  • Size: 27.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for osmcha-0.3.7.tar.gz
Algorithm Hash digest
SHA256 dee547a4c2a9bf9f4fa61fe5b76d2fc9845d9388405ba2dcb46de3a183603fcc
MD5 dbc394ed9be3dc604745b66687be5e4f
BLAKE2b-256 8d89ffe31e32411f5fdbf1a3c8ee0965e8e5d25ff5f621d9cf7d07661d3802da

See more details on using hashes here.

File details

Details for the file osmcha-0.3.7-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for osmcha-0.3.7-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 0e31a6c729eed5de5eccf870306705030b3b253cb50008d420703e641b6fad1f
MD5 9432570b8e166417f6ac40cb2ce716dd
BLAKE2b-256 5802133566130fec2314a9276fa9ea217c3c948d5fec830b5aa852a7e278b143

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page