A Python parser for BGP data in MRT or looking glass format

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for js9401 from gravatar.com js9401
Meta

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License
  • Tags BGP , MRT , parser
  • Requires: Python >=3.6
Classifiers
Report project as malware

Project description

ftlbgp

A Python parser for BGP data in MRT or looking glass format.

Key features

  • Support for all MRT entry types, BGP messages, and BGP attributes
  • Customizable record and attribute types (no parsing of unneeded data)
  • Programmatic data access with optional CSV and JSON serialization
  • Raw values and human-readable output (e.g. integers vs. strings)
  • Rapid prototyping (namedtuple) and high-performance mode (tuple)
  • Context manager with built-in statistics and flexible error handling
  • Zero-copy operations on all data items (as fast as it gets in Python)

Compatibility

This package is compatible with python3.6 and pypy3.6-v7.0.0 or greater.

Installation

Run pip install ftlbgp or

~$ git clone https://github.com/leitwert-net/ftlbgp.git`
~$ cd ftlbgp.git/src

to download and run the source code manually.

Usage

Parsing MRT files

~$ python3 -m ftlbgp -h
usage: python3 -m ftlbgp [-h] [--pkg-help] [--json] <FILE> [<FILE> ...]

ftlbgp [v1.0.3] - Parse BGP archives in MRT or looking glass format

positional arguments:
  <FILE>      input file with BGP data (supports bz2/gz)

optional arguments:
  -h, --help  show this help message and exit
  --pkg-help  show package help for BgpParser usage
  --json      output BGP records in JSON format

Programmatic use

# Import parser
from ftlbgp import BgpParser

# Prepare input file
filename = ...

# Parse default records and attributes
with BgpParser() as parse:
    for record in parse(filename):
        print(record)

Customization

# Parse all records
with BgpParser(bgp_records=BgpParser.bgp.records.ALL) as parse:
    for record in parse(filename):
        print(record)

# Parse specific records (route and error)
with BgpParser(bgp_records=BgpParser.bgp.records.route | BgpParser.bgp.records.error) as parse:
    for record in parse(filename):
        print(record)

# Parse all route attributes
with BgpParser(bgp_route=BgpParser.bgp.route.ALL) as parse:
    for record in parse(filename):
        print(record)

# Parse specific route attributes (default and local_pref)
with BgpParser(bgp_route=BgpParser.bgp.route.DEFAULT | BgpParser.bgp.route.local_pref) as parse:
    for record in parse(filename):
        print(record)

Full specification

BgpParser()
      
This @FtlParser instance is used to read input files and generate a set of BGP records (Python tuples).
It must be used with a context manager (see sample usage below) and accepts the following arguments.

Keyword arguments:
  named_records     - Return named tuples instead of plain unnamed tuple records. [Default: True]
  serialize         - Convert output records to JSON (if named) or CSV (if unnamed). [Default: False]
  use_cache         - Cache expensive low-entropy values (memory consumption < 1MB). [Default: True]
  raise_on_errors   - Raise exceptions and stop parsing in case of data errors. [Default: False]
  bgp_records       - Select the types of BgpRecord entries to be returned by the parser.
                      Supports bitwise logical operators OR, AND, NOT to specify multiple records.
                      [Default:
                        BgpParser.bgp.records.route |
                        BgpParser.bgp.records.stats |
                        BgpParser.bgp.records.error
                       Available:
                        BgpParser.bgp.records.peer_table
                        BgpParser.bgp.records.state_change
                        BgpParser.bgp.records.keep_alive
                        BgpParser.bgp.records.route_refresh
                        BgpParser.bgp.records.notification
                        BgpParser.bgp.records.open
                        BgpParser.bgp.records.ALL
                        BgpParser.bgp.records.NONE
                        BgpParser.bgp.records.DEFAULT]
  bgp_peer_table    - Select [optionally human-readable] attribute to be included in BgpPeerTableRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.peer_table[.human].type |
                        BgpParser.bgp.peer_table[.human].peer_protocol |
                        BgpParser.bgp.peer_table[.human].peer_bgp_id |
                        BgpParser.bgp.peer_table[.human].peer_as |
                        BgpParser.bgp.peer_table[.human].peer_ip
                       Available:
                        BgpParser.bgp.peer_table[.human].collector_bgp_id
                        BgpParser.bgp.peer_table[.human].view_name
                        BgpParser.bgp.peer_table[.human].ALL
                        BgpParser.bgp.peer_table[.human].NONE
                        BgpParser.bgp.peer_table[.human].DEFAULT]
  bgp_state_change  - Select [optionally human-readable] attribute to be included in BgpStateChangeRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.state_change[.human].type |
                        BgpParser.bgp.state_change[.human].timestamp |
                        BgpParser.bgp.state_change[.human].peer_protocol |
                        BgpParser.bgp.state_change[.human].peer_as |
                        BgpParser.bgp.state_change[.human].peer_ip |
                        BgpParser.bgp.state_change[.human].old_state |
                        BgpParser.bgp.state_change[.human].new_state
                       Available:
                        BgpParser.bgp.state_change[.human].ALL
                        BgpParser.bgp.state_change[.human].NONE
                        BgpParser.bgp.state_change[.human].DEFAULT]
  bgp_route         - Select [optionally human-readable] attribute to be included in BgpRouteRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.route[.human].type |
                        BgpParser.bgp.route[.human].source |
                        BgpParser.bgp.route[.human].sequence |
                        BgpParser.bgp.route[.human].timestamp |
                        BgpParser.bgp.route[.human].peer_protocol |
                        BgpParser.bgp.route[.human].peer_bgp_id |
                        BgpParser.bgp.route[.human].peer_as |
                        BgpParser.bgp.route[.human].peer_ip |
                        BgpParser.bgp.route[.human].nexthop_protocol |
                        BgpParser.bgp.route[.human].nexthop_ip |
                        BgpParser.bgp.route[.human].prefix_protocol |
                        BgpParser.bgp.route[.human].prefix |
                        BgpParser.bgp.route[.human].path_id |
                        BgpParser.bgp.route[.human].aspath |
                        BgpParser.bgp.route[.human].origin |
                        BgpParser.bgp.route[.human].communities |
                        BgpParser.bgp.route[.human].large_communities
                       Available:
                        BgpParser.bgp.route[.human].extended_communities
                        BgpParser.bgp.route[.human].multi_exit_disc
                        BgpParser.bgp.route[.human].atomic_aggregate
                        BgpParser.bgp.route[.human].aggregator_protocol
                        BgpParser.bgp.route[.human].aggregator_as
                        BgpParser.bgp.route[.human].aggregator_ip
                        BgpParser.bgp.route[.human].only_to_customer
                        BgpParser.bgp.route[.human].originator_id
                        BgpParser.bgp.route[.human].cluster_list
                        BgpParser.bgp.route[.human].local_pref
                        BgpParser.bgp.route[.human].attr_set
                        BgpParser.bgp.route[.human].as_pathlimit
                        BgpParser.bgp.route[.human].aigp
                        BgpParser.bgp.route[.human].attrs_unknown
                        BgpParser.bgp.route[.human].ALL
                        BgpParser.bgp.route[.human].NONE
                        BgpParser.bgp.route[.human].DEFAULT]
  bgp_keep_alive    - Select [optionally human-readable] attribute to be included in BgpKeepAliveRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.keep_alive[.human].type |
                        BgpParser.bgp.keep_alive[.human].timestamp |
                        BgpParser.bgp.keep_alive[.human].peer_protocol |
                        BgpParser.bgp.keep_alive[.human].peer_as |
                        BgpParser.bgp.keep_alive[.human].peer_ip
                       Available:
                        BgpParser.bgp.keep_alive[.human].ALL
                        BgpParser.bgp.keep_alive[.human].NONE
                        BgpParser.bgp.keep_alive[.human].DEFAULT]
  bgp_route_refresh - Select [optionally human-readable] attribute to be included in BgpRouteRefreshRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.route_refresh[.human].type |
                        BgpParser.bgp.route_refresh[.human].timestamp |
                        BgpParser.bgp.route_refresh[.human].peer_protocol |
                        BgpParser.bgp.route_refresh[.human].peer_as |
                        BgpParser.bgp.route_refresh[.human].peer_ip |
                        BgpParser.bgp.route_refresh[.human].refresh_protocol
                       Available:
                        BgpParser.bgp.route_refresh[.human].ALL
                        BgpParser.bgp.route_refresh[.human].NONE
                        BgpParser.bgp.route_refresh[.human].DEFAULT]
  bgp_notification  - Select [optionally human-readable] attribute to be included in BgpNotificationRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.notification[.human].type |
                        BgpParser.bgp.notification[.human].timestamp |
                        BgpParser.bgp.notification[.human].peer_protocol |
                        BgpParser.bgp.notification[.human].peer_as |
                        BgpParser.bgp.notification[.human].peer_ip |
                        BgpParser.bgp.notification[.human].error_code |
                        BgpParser.bgp.notification[.human].error_subcode |
                        BgpParser.bgp.notification[.human].data
                       Available:
                        BgpParser.bgp.notification[.human].ALL
                        BgpParser.bgp.notification[.human].NONE
                        BgpParser.bgp.notification[.human].DEFAULT]
  bgp_open          - Select [optionally human-readable] attribute to be included in BgpOpenRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.open[.human].type |
                        BgpParser.bgp.open[.human].timestamp |
                        BgpParser.bgp.open[.human].peer_protocol |
                        BgpParser.bgp.open[.human].peer_as |
                        BgpParser.bgp.open[.human].peer_ip |
                        BgpParser.bgp.open[.human].version |
                        BgpParser.bgp.open[.human].my_as |
                        BgpParser.bgp.open[.human].hold_time |
                        BgpParser.bgp.open[.human].bgp_id |
                        BgpParser.bgp.open[.human].capabilities
                       Available:
                        BgpParser.bgp.open[.human].params_unknown
                        BgpParser.bgp.open[.human].ALL
                        BgpParser.bgp.open[.human].NONE
                        BgpParser.bgp.open[.human].DEFAULT]
  bgp_stats         - Select [optionally human-readable] attribute to be included in BgpStatsRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.stats[.human].type |
                        BgpParser.bgp.stats[.human].parser_lifetime |
                        BgpParser.bgp.stats[.human].parser_runtime |
                        BgpParser.bgp.stats[.human].parser_errors |
                        BgpParser.bgp.stats[.human].lgl_runtime |
                        BgpParser.bgp.stats[.human].lgl_entries |
                        BgpParser.bgp.stats[.human].lgl_errors |
                        BgpParser.bgp.stats[.human].mrt_runtime |
                        BgpParser.bgp.stats[.human].mrt_entries |
                        BgpParser.bgp.stats[.human].mrt_errors |
                        BgpParser.bgp.stats[.human].mrt_fixes |
                        BgpParser.bgp.stats[.human].mrt_bgp_entry_types |
                        BgpParser.bgp.stats[.human].mrt_bgp_message_types |
                        BgpParser.bgp.stats[.human].mrt_bgp_attribute_types |
                        BgpParser.bgp.stats[.human].mrt_bgp_capability_types |
                        BgpParser.bgp.stats[.human].bgp_routes_rib_ipv4 |
                        BgpParser.bgp.stats[.human].bgp_routes_rib_ipv6 |
                        BgpParser.bgp.stats[.human].bgp_routes_announce_ipv4 |
                        BgpParser.bgp.stats[.human].bgp_routes_announce_ipv6 |
                        BgpParser.bgp.stats[.human].bgp_routes_withdraw_ipv4 |
                        BgpParser.bgp.stats[.human].bgp_routes_withdraw_ipv6
                       Available:
                        BgpParser.bgp.stats[.human].ALL
                        BgpParser.bgp.stats[.human].NONE
                        BgpParser.bgp.stats[.human].DEFAULT]
  bgp_error         - Select [optionally human-readable] attribute to be included in BgpErrorRecord entries.
                      Supports bitwise logical operators (OR/AND/NOT) to specify multiple attributes.
                      [Default:
                        BgpParser.bgp.error[.human].type |
                        BgpParser.bgp.error[.human].source |
                        BgpParser.bgp.error[.human].scope |
                        BgpParser.bgp.error[.human].record |
                        BgpParser.bgp.error[.human].reason |
                        BgpParser.bgp.error[.human].message |
                        BgpParser.bgp.error[.human].data |
                        BgpParser.bgp.error[.human].trace
                       Available:
                        BgpParser.bgp.error[.human].ALL
                        BgpParser.bgp.error[.human].NONE
                        BgpParser.bgp.error[.human].DEFAULT]

Returns:
  parse(<filename>) - Parse function that accepts a filename as single positional argument and generates
                      all specified BGP records on invocation. Input files can be provided in .bz2 or .gz
                      format. The parse function may be invoked multiple times within a single context.

Raises:
  FtlError          - Generic parser or runtime error.
  FtlFileError      - Failure during access of input file.
  FtlFormatError    - Unexpected format of input file.
  FtlDataError      - Invalid data entry in input file.

Author

Johann SCHLAMP <schlamp@leitwert.net>

License

Copyright (C) 2014-2024 Leitwert GmbH

This software is distributed under the terms of the MIT license.
It can be found in the LICENSE file or at https://opensource.org/licenses/MIT.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for js9401 from gravatar.com js9401
Meta

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License
  • Tags BGP , MRT , parser
  • Requires: Python >=3.6
Classifiers

Release history Release notifications | RSS feed

1.0.5

This version

1.0.4

1.0.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ftlbgp-1.0.4.tar.gz (61.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ftlbgp-1.0.4-py3-none-any.whl (80.5 kB view details)

Uploaded Python 3

File details

Details for the file ftlbgp-1.0.4.tar.gz.

File metadata

  • Download URL: ftlbgp-1.0.4.tar.gz
  • Upload date:
  • Size: 61.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.12

File hashes

Hashes for ftlbgp-1.0.4.tar.gz
Algorithm Hash digest
SHA256 253d04c08e7dcd856d7ec97d862264e7e242acd0394b4aebf2a241c940b2c7eb
MD5 8144aeff0a017e429ea0378d2a7a8934
BLAKE2b-256 2d80cd5579808e5da26747fc676ad699ed7177db0990963130e84f0060a5266a

See more details on using hashes here.

File details

Details for the file ftlbgp-1.0.4-py3-none-any.whl.

File metadata

  • Download URL: ftlbgp-1.0.4-py3-none-any.whl
  • Upload date:
  • Size: 80.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.12

File hashes

Hashes for ftlbgp-1.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 52f63e72e354f34976206456a0154e79392b30248bc996d8aeefed69835893a2
MD5 2b4a4920dcce8f30ba6b93bf874e1ca4
BLAKE2b-256 380d8f37f69f45feadc2a3e8f4d121959bde4f014d83ef8c55e0eb4041645ca6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page