Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Glottolog languoid tree as SQLite database

Project description

Latest PyPI Version License Supported Python Versions Format

This tool loads the content of the languoids/tree directory from the Glottolog master repo into a normalized SQLite database.

Each file under in that directory contains the definition of one Glottolog languoid. Loading their content into a relational database allows to perform some advanced consistency checks (example) and in general to execute queries that inspect the languoid tree relations in a compact and performant way (e.g. without repeatedly traversing the directory tree).

See pyglottolog for the more general official Python API to work with the repo without a mandatory initial loading step (also provides programmatic access to the references and a convenient command-line interface).

The database can be exported into a ZIP file containing one CSV file for each database table, or written into a single denormalized CSV file with one row per languoid (via a provided SQL query).

As sqlite is the most widely used database, the database file itself (e.g. treedb.sqlite3) can be queried directly from most programming environments. It can also be examined using graphical interfaces such as DBeaver, or via the sqlite3 cli.

Python users can also use the provided SQLAlchemy models to build queries or additional abstractions programmatically using SQLAlchemy core or the ORM (as more maintainable alternative to hand-written SQL queries).

Quickstart

Clone this repository side-by-side to your clone of the glottolog master repo:

$ git clone https://github.com/glottolog/glottolog.git
$ git clone https://github.com/glottolog/treedb.git
$ cd treedb

Note: treedb expects to find it under ../glottolog/ relative to its repository root.

Optional: Create and activate a venv for installation into an isolated Python environment:

$ python -m venv .venv  # PY3
$ source .venv/bin/activate  # Windows: .venv/Scripts/activate.bat

Install treedb (and dependencies) directly from your clone (editable install):

$ pip install -e .

Load ../glottolog/languoids/tree/**/md.ini into an in-memory sqlite3 database. Write the denormalized example query into treedb.csv:

$ python -c "import treedb; treedb.load(); treedb.write_csv()"

Usage from Python

Start a Python shell:

$ python

Import the package:

>>> import treedb

Use treedb.iterlanguoids() to iterate over languoids as (<path>, dict) pairs:

>>> next(treedb.iterlanguoids())
(('abin1243',), {'id': 'abin1243', 'parent_id': None, 'level': 'language', ...

Note: This is a low-level interface, which does not require loading.

Load the database into treedb.sqlite3 (and set the default engine):

>>> treedb.load('treedb.sqlite3')
...
<treedb.proxies.SqliteEngineProxy filename='treedb.sqlite3' ...>

Run consistency checks:

>>> treedb.check()
...
True

Export into a ZIP file containing one CSV file per database table:

>>> treedb.export()
...Path('treedb.zip')

Execute the example query and write it into a CSV file with one row per languoid:

>>> treedb.write_csv()
...Path('treedb.csv')

Rebuild the database (e.g. after an update):

>>> treedb.load(rebuild=True)
...
<treedb.proxies.SqliteEngineProxy filename='treedb.sqlite3' ...>

Execute a simple query with sqlalchemy core and write it to a CSV file:

>>> treedb.write_csv(treedb.select([treedb.Languoid]), filename='languoids.csv')
...Path('languoids.csv')

Get one row from the languoid table via sqlalchemy core:

>>> treedb.select([treedb.Languoid]).execute().first()
('abin1243', 'language', 'Abinomn', None, 'bsa', 'bsa', -2.92281, 138.891)

Get one Languoid model instance via sqlalchemy orm:

>>> session = treedb.Session()
>>> session.query(treedb.Languoid).first()
<Languoid id='abin1243' level='language' name='Abinomn' hid='bsa' iso639_3='bsa'>
>>> session.close()

See also

License

This tool is distributed under the Apache license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for treedb, version 0.1.2
Filename, size File type Python version Upload date Hashes
Filename, size treedb-0.1.2-py2.py3-none-any.whl (37.1 kB) File type Wheel Python version py2.py3 Upload date Hashes View hashes
Filename, size treedb-0.1.2.zip (176.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page