Skip to main content

Python library to manipulate Open Travel Data

Project description

Minimalist GeoBases implementation:

  • no dependencies

  • compatible with Python 3.9+, CPython and PyPy

  • one data source: opentraveldata

  • one Python module for easier distribution on clusters (like Hadoop)

  • faster load time (5x)

  • tested with pytest and tox

>>> from neobase import NeoBase
>>> b = NeoBase()
>>> b.get('ORY', 'city_code_list')
['PAR']
>>> b.get('ORY', 'city_name_list')
['Paris']
>>> b.get('ORY', 'country_code')
'FR'
>>> b.distance('ORY', 'CDG')
34.87...
>>> b.get_location('ORY')
LatLng(lat=48.72..., lng=2.35...)

Installation

Use the Python package:

pip install neobase

Docs

Check out readthedocs for the API.

You can customize the source data when initializing:

with open("file.csv") as f:
    N = NeoBase(f)

Otherwise the loaded file will be the embedded one, unless the OPTD_POR_FILE environment variable is set. In that case, it will load from the path defined in that variable.

You can manually retrieve the latest data source yourself too, but you expose yourself to some breaking changes if they occur in the data.

from io import StringIO
from urllib.request import urlopen

from neobase import NeoBase, OPTD_POR_URL

data = urlopen(OPTD_POR_URL).read().decode('utf8')
N = NeoBase(StringIO(data))
N.get("PAR")

The reference date of validity can be changed as well:

N = NeoBase(date="2000-01-01")
N.get("AIY")  # was decommissioned in 2015

By default, the reference date will be set to today, unless the OPTD_POR_DATE environment variable is set. In that case, it will use that value.

You can customize the behavior regarding duplicates: points sharing the same IATA code, like NCE as airport and NCE as city. By default everything is kept, but you can set it so that only the first point with an IATA code is kept:

N = NeoBase(duplicates=False)
len(N)  # about 10,000 "only"

Note that you can use the OPTD_POR_DUPLICATES environment variable to control this as well: set it to 0 to drop duplicates.

Finally, you can customize fields loaded by subclassing.

class SubNeoBase(NeoBase):
    KEY = 0  # iata_code

    # Those loaded fields are the default ones
    FIELDS = (
        ("name", 6, None),
        ("lat", 8, None),
        ("lng", 9, None),
        ("page_rank", 12, lambda s: float(s) if s else None),
        ("country_code", 16, None),
        ("country_name", 18, None),
        ('continent_name', 19, None),
        ("timezone", 31, None),
        ("city_code_list", 36, lambda s: s.split(",")),
        ('city_name_list', 37, lambda s: s.split('=')),
        ('location_type', 41, None),
        ("currency", 46, None),
    )

N = SubNeoBase()

Command-line interface

You can query the data using:

python -m neobase PAR NCE

Tests

tox

A note about performance

The geographical operations like N.find_near("ORY", 100) or N.find_closest_from("ORY") perform a full scan of the data, and are not optimized (remember that this library has no dependencies, by design).

If you want a more efficient solution, you should use a spatial index like a BallTree, for example using scikit-learn:

import numpy as np
from sklearn.neighbors import BallTree
from neobase import NeoBase

N = NeoBase()

iata_codes = []
coords = []
for key in N:
    lat, lon = N.get_location(key)
    if lat is not None and lon is not None:
        iata_codes.append(N.get(key, "iata_code"))
        coords.append([np.radians(lat), np.radians(lon)])
coords = np.array(coords)

tree = BallTree(coords, metric="haversine")

def find_closest_with_balltree(coord):
    point = np.radians(coord)
    _, idx = tree.query([point], k=1)
    iata_code = iata_codes[idx[0][0]]
    return iata_code

paris = (48.8566, 2.3522)
print(find_closest_with_balltree(paris))  # <0.1ms
print(list(N.find_closest_from_location(paris)))  # ~30ms

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

neobase-0.34.10.tar.gz (4.0 MB view details)

Uploaded Source

Built Distribution

neobase-0.34.10-py3-none-any.whl (4.0 MB view details)

Uploaded Python 3

File details

Details for the file neobase-0.34.10.tar.gz.

File metadata

  • Download URL: neobase-0.34.10.tar.gz
  • Upload date:
  • Size: 4.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for neobase-0.34.10.tar.gz
Algorithm Hash digest
SHA256 8cbb45473bde9f4c38a8c60223d1117103615541223ea5c205ba0adb0a470c25
MD5 6fea47345488b9b462201b81ddf38286
BLAKE2b-256 6909f39e25b5a72dcecf73f8ae18c97d3717be4bebed875c9a61b5223f42ec08

See more details on using hashes here.

Provenance

The following attestation bundles were made for neobase-0.34.10.tar.gz:

Publisher: python-package.yml on alexprengere/neobase

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file neobase-0.34.10-py3-none-any.whl.

File metadata

  • Download URL: neobase-0.34.10-py3-none-any.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for neobase-0.34.10-py3-none-any.whl
Algorithm Hash digest
SHA256 4b4da8f38cb358770a9fac3aa8eaaf8965cd604ee1c0f32b5ce17ad8a8b80753
MD5 fbeffff91c1e5a5f65070404d2b3083c
BLAKE2b-256 a4ffb107019cadbc5224c213954e5175257c287efa3139221759eb7f961ca774

See more details on using hashes here.

Provenance

The following attestation bundles were made for neobase-0.34.10-py3-none-any.whl:

Publisher: python-package.yml on alexprengere/neobase

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page