RPSL lexer for Internet Route Verification.
Project description
Parse RPSL Policy
WIP
TODO: update this README.
Debugging
-
Enable logging:
export RUST_LOG=route_verification=trace
-
Enable backtrace in error messages:
export RUST_BACKTRACE=1
Produce a parsed dump using both lexer and parser
TODO: We use Rye now, so we can just initialize the virtual environment and run python
.
-
Put the database file at
data/ripe.db
. -
Make sure
pypy3
is in thePATH
. -
Make sure you have
rpsl-lexer
installed and it can be found bypypy3
(e.g., inPYTHONPATH
).python3 -m pip install rpsl-lexer
-
Run at
route_verification/
:cargo r --release -- parse ../data/ripe.db ../parsed
The parsed dump will be distributed in
parsed/
.
Produce a spread parsed dump from both priority and backup registries
Obtain IRR data
Download from all FTP servers on IRR List of Routing Registries.
Download priority registries to data/irrs/priority/
:
ftp://ftp.afrinic.net/pub/dbase/ ftp://ftp.altdb.net/pub/altdb/ ftp://ftp.apnic.net/pub/apnic/whois/ ftp://ftp.arin.net/pub/rr/ ftp://irr.bboi.net/ https://whois.canarie.ca/dbase/ ftp://irr-mirror.idnic.net/ ftp://ftp.nic.ad.jp/jpirr/ ftp://irr.lacnic.net/ ftp://ftp.nestegg.net/irr ftp://rr1.ntt.net/nttcomRR/ ftp://ftp.panix.com/pub/rrdb ftp://ftp.ripe.net/ripe/dbase/
Download backup registries to data/irrs/backup/
:
ftp://ftp.radb.net/radb/dbase/
Decompress all files.
Run the parser with parse_priority
Run at route_verification/
:
cargo r --release -- parse_priority ../data/irrs/priority/ ../data/irrs/backup/ ../parsed_all/
The above command parses all IRR DB files in data/irrs/priority/
and
data/irrs/backup/
,
overrides any duplicated information with the version from the former,
and writes the result to multiple JSON files in parsed_all/
.
Produce a parsed IR dump using IRRs in ordered priorities
Run at route_verification/
:
cargo r --release -- parse_ordered \
../data/irrs/priority/apnic.db.* ../data/irrs/priority/afrinic.db ../data/irrs/priority/arin.db ../data/irrs/priority/lacnic.db\
../data/irrs/priority/ripe.db ../data/irrs/backup/radb.db\
../data/irrs/backup/altdb.db ../data/irrs/backup/idnic.db ../data/irrs/backup/jpirr.db ../data/irrs/backup/level3.db ../data/irrs/backup/nttcom.db ../data/irrs/backup/reach.db ../data/irrs/backup/tc.db\
../parsed_all/
Running interactively in Jupyter Notebook
- Finish the previous section. Your parsed dump should be cached in
parsed/
. - Install Evcxr Jupyter Kernel.
- Open the notebook at
./
, and try outparse_test.ipynb
.
Produce a lexed dump using lexer
If the database file is at data/ripe.db
, for example:
pypy3 -m rpsl_lexer.dump data/ripe.db > dump.json
In the command above, we use PyPy for faster performance,
and pipe the dumped JSON to dump.json
.
Test lexer
Run at ./
:
pytest
Test lexer against ripe.db
To test against ripe.db
using rpsl_lexer/tests/mp_import_w_db.py
,
put the database file at data/ripe.db
:
python3 -m rpsl_lexer.tests.mp_import_w_db
Similarly, to test with rpsl_lexer/tests/mp_export_w_db.py
,
or one of the other lexers:
python3 -m rpsl_lexer.tests.mp_export_w_db
python3 -m rpsl_lexer.tests.mp_peering_w_db
python3 -m rpsl_lexer.tests.mp_filter_w_db
python3 -m rpsl_lexer.tests.action_w_db
python3 -m rpsl_lexer.tests.import_w_db
python3 -m rpsl_lexer.tests.export_w_db
Test parser and comparator
Run at route_verification/
:
cargo t --workspace
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for rpsl_lexer-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99f8f50622c530de14c70af907127c5dd3dc37134ece74e84f7c56f1f655a7ba |
|
MD5 | 1d2677d49a1b8438d2f7a5ae92a17e83 |
|
BLAKE2b-256 | 37f98f110cb48af8735c44ab25f0c09fd98e4c6ded9e3120fea60bfcda7ef400 |