Library to retrieve and manipulate data about IXPs
Project description
django-ixp-tracker
Library to retrieve and manipulate data about IXPs
Installation
Install this library using pip
:
pip install django-ixp-tracker
Usage
- Add to your INSTALLED_APPS setting like this:
INSTALLED_APPS = [
...,
"ixp_tracker",
]
Note: this app has no web-facing components so you don't need to add anything to urls.py
etc
- Run
python manage.py migrate
to create the models. - Add the relevant settings to your config.
IXP_TRACKER_PEERING_DB_URL
will use a default if you don't provide a value so you probably don't need that. But you will need to setIXP_TRACKER_PEERING_DB_KEY
to authenticate against the API. - Add
IXP_TRACKER_GEO_LOOKUP_FACTORY
to config with the path to your factory (see below). - Run the management command to import the data:
python manage.py ixp_tracker_import
(This will sync the current data, if you want historical data you need to backfill first)
ASN country and status data
The lib uses an external component to look up the country of registration (why?) and the status of an ASN. This status is used for the logic to identify when members have left an IXP.
If you don't provide this service yourself, it will default to a noop version. This will mean you will get no country of registration data and the marking of members having left an IXP will not be as efficient.
In order to implement such a component yourself, you should implement the Protocol ixp_tracker.importers.ASNGeoLookup
and provide a factory function for your class.
Backfilling data
You have the option of backfilling data from archived PeeringDb data. This can be done by running the import command with the --backfill
option for each month you want to backfill:
python manage.py ixp_tracker_import --backfill <YYYMM>
The backfill currently process a single month at a time and will look for the earliest file for the relevant month at https://publicdata.caida.org/datasets/peeringdb/
IMPORTANT NOTE: due to the way the code tries to figure out when a member left an IXP, you should run the backfill strictly in date order and before syncing the current data.
IXP stats
The import process also generates monthly stats per IXP and per country. These are generated as of the 1st of the month used to import the data.
Running programmatically
If you'd like to run the import from code, rather than from the management command, you can call importers.import_data()
directly.
It's not recommended to call any other functions yourself.
Development
To contribute to this library, first checkout the code. Then create a new virtual environment:
cd django-ixp-tracker
python -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
pip install -e '.[test]'
To run the tests:
pytest
We use pre-commit for linting etc on push. Run:
pre-commit install
from the repo top-level dir to set it up.
Peering Db libraries
PeeringDb provide their own vanilla Python and Django libs, but we have decided not to use these.
Both libs are designed to keep a local copy of the current data and to keep that copy in sync with the central copy via the API.
As we need to keep a historical record (e.g. for IXP growth stats over time), we would have to provide some sort of wrapper over those libs anyway.
In addition to that, the historical archives of PeeringDb data use flat lists of the different object types in json. We can retrieve the data from the API directly in the same way, so it makes it simpler to implement.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for django_ixp_tracker-0.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b5b74e30c378e98f0accac8e78961f04b0b12c5dfce296c064fb0281de8d0da |
|
MD5 | 39d1b6e09da738bc2ddcd80b7141b222 |
|
BLAKE2b-256 | 028465d219786fb203ce293ae180791977f196d5bf887ca3c7ff01d1c0a3be43 |