Skip to main content

A Python tool to automatically build (and test) configurations for BGP route servers.

Project description

ARouteServer

Documentation Build Status Unique test cases PYPI Version Python Versions Requirements Status

A Python tool to automatically build (and test) feature-rich configurations for BGP route servers.

How it works

  1. Two YAML files provide general policies and clients configurations options:

    cfg:
      rs_as: 64496
      router_id: "192.0.2.2"
      add_path: True
      filtering:
        next_hop:
          policy: "same-as"
      blackhole_filtering:
        policy_ipv4: "rewrite-next-hop"
        ...
    clients:
      - asn: 64511
        ip:
        - "192.0.2.11"
        - "2001:db8:1:1::11"
        irrdb:
          as_sets:
            - "RIPE::AS-FOO"
      ...
  2. ARouteServer acquires external information to enrich them: i.e. bgpq3 for IRR data, PeeringDB for max-prefix limit and AS-SETs, …

  3. Jinja2 built-in templates are used to render the final route server’s configuration file.

    Currently, BIRD (1.6.x starting from 1.6.3) and OpenBGPD (OpenBSD 6.1 up to 6.5 and also OpenBGPD Portable 6.5p1) are supported, with almost feature parity between them.

Validation and testing are performed using the built-in live tests framework: Docker instances are used to simulate several scenarios and to validate the behaviour of the route server after configuring it with ARouteServer. More details on the Live tests section.

Features

  • Path hiding mitigation techniques (RFC7947 section 2.3.1).

  • Basic filters (mostly enabled by default):

    • NEXT_HOP enforcement (strict / same AS - RFC7948 section 4.8);

    • minimum and maximum IPv4/IPv6 prefix length;

    • maximum AS_PATH length;

    • reject invalid AS_PATHs (containing private/invalid ASNs);

    • reject AS_PATHs containing transit-free ASNs;

    • reject bogons;

    • max-prefix limit based on global or client-specific values or on PeeringDB data.

  • Prefixes and origin ASNs validation (also in tag-only mode):

    • IRR-based filters (RFC7948 section 4.6.2);

    • AS-SETs configured manually or fetched from PeeringDB;

    • support for IRR sources (RIPE::AS-FOO, RADB::AS-BAR);

    • white lists support;

    • extended dataset for filters generation:

    • RPKI-based filtering (BGP Prefix Origin Validation).

  • Blackhole filtering support:

    • optional NEXT_HOP rewriting;

    • signalling via BGP Communities (BLACKHOLE and custom communities);

    • client-by-client control over propagation.

  • Graceful shutdown support:

    • honor the GRACEFUL_SHUTDOWN BGP community received from clients (draft-ietf-grow-bgp-gshut-11);

    • allow to perform a graceful shutdown of the route server itself.

  • Control and informative communities:

    • prefix/origin ASN present/not present in IRRDBs data;

    • do (not) announce to any / peer / on RTT basis;

    • prepend to any / peer / on RTT basis;

    • add NO_EXPORT / NO_ADVERTISE to any / peer;

    • custom informational BGP communities.

  • Optional session features on a client-by-client basis:

  • Automatic building of clients list:

  • IX-F Member Export JSON files creation.

  • Related tools:

A comprehensive list of features can be found within the comments of the distributed configuration file on GitHub or on the documentation web page.

More feature are already planned: see the Future work section for more details.

Full documentation

Full documentation can be found on ReadTheDocs: https://arouteserver.readthedocs.org/

Presentations

  • RIPE74, 10 May 2017, Connect Working Group: video (9:53), slides (PDF)

  • Salottino MIX, 30 May 2017: slides

Mentions / endorsements:

Who is using ARouteServer?

Are you using it? Do you want to be listed here? Drop me a message!

1: GoBGP configurations are generated using a fork of the project which is still WIP and that hopefully will be merged upstream in the future.

Status

Beta testing, looking for testers and reviewers.

Anyone who wants to share his/her point of view, to review the output configurations or to test them is more than welcome!

Bug? Issues? Support requests?

But also suggestions? New ideas?

Please create an issue on GitHub or drop me a message.

A Slack channel is also available on the network.toCode() community: arouteserver.

Author

Pier Carlo Chiodi - https://pierky.com

Blog: https://blog.pierky.com Twitter: @pierky

Change log

v0.22.1

  • Fix: handle more formats for ROAs exported from the public instances of RIPE and NTT validators.

    A new way of representing ASNs (without the “AS” prefix) and new TA names which were not matched by the default values of rpki_roas.allowed_trust_anchors prevented ROAs from being imported and correctly processed when the default settings were used.

v0.22.0

This is the last release of ARouteServer for which Python 2.7 compatibility is guaranteed. From the next release, any new feature will not be tested against that version of Python.

  • New: OpenBGPD Portable <https://github.com/openbgpd-portable/openbgpd-portable> (release 6.5p1) also supported.

    Release 6.5p1 of OpenBGPD Portable edition passed the integration testing suite.

  • New: add support for OpenBGPD/OpenBSD 6.5 enhancements.

    Support for matching multiple communities at the same time allows to create more readable configurations.

  • Improvement: OpenBGPD, some filters refinement.

    Avoid checking AS0 in AS_PATH since 6.4. No needs to check routes of an address family different than the one used for the session.

As announced with release 0.20.0, OpenBGPD/OpenBSD 6.2 is no longer tested. Also OpenBGPD/OpenBSD 6.3 tests have been decommissioned. Starting with this release, tests will be executed only against the 2 most recent releases of OpenBGPD/OpenBSD and against the last release of the supported major versions of BIRD. The implementation of new features may break compatibility of the configurations built for unsupported releases.

v0.21.1

  • Deprecation: SAVVIS IRR removed from the list of default sources used by bgpq3.

  • Fix (minor): truncate the max length of AS-SET names to 64 characters.

    BIRD supports only names no longer than 64 characters.

    Related: issue #47 on GitHub.

v0.21.0

  • Improvement: when ripe-rpki-validator-cache is set as the source of ROAs, multiple URLs can now be specified to fetch data from.

    URLs will be tried in the same order as they are configured; if the attempt to download ROAs from the first URL fails, the second URL will be tried, an so on.

    By default, the RIPE NCC public instance of the RIPE RPKI Validator will be tried first, then the NTT instance. The list of URLs can be set in the general.yml configuration file, roas.ripe_rpki_validator_url option.

v0.20.0

This is the last release of ARouteServer for which OpenBGPD/OpenBSD 6.1 and 6.2 CI tests are ran. From the next release, any new feature will not be tested against these versions of OpenBGPD. Users are encouraged to move to newer releases.

  • New: add support for OpenBGPD/OpenBSD 6.4 enhancements.

    Use new sets for prefixes, ASNum, and origins (prefix + source-as), and also RPKI ROA sets.

  • Improvement: OpenBGPD, reduce the number of rules by combining some into the same rule.

  • Improvement: route server policies definition files built using the configure command now have RPKI BGP Origin Validation and “use-ROAs-as-route-objects” enabled by default.

As announced with release 0.19.0, OpenBGPD/OpenBSD 6.0 is no longer tested. The implementation of new features may break compatibility of the configurations built for unsupported releases.

Most of this release is based on the work made by Claudio Jeker.

v0.19.1

  • Fix (BIRD configuration only): change bgp_path.last with bgp_path.last_nonaggregated.

    When a route is originated from the aggregation of two different routes using the AS_SET, bgp_path.last always returns 0, so the origin ASN validation against IRR always fails.

    Related: issue #34 on GitHub.

v0.19.0

This is the last release of ARouteServer for which OpenBGPD/OpenBSD 6.0 CI tests are ran. Starting with the next release, any new feature will not be tested against version 6.0 of OpenBGPD. Users are encouraged to move to newer releases.

v0.18.0

  • New: add support for BIRD 1.6.4 and OpenBGPD/OpenBSD 6.3.

    This release breaks backward compatibility (OpenBGPD configs only): the default target version used to build OpenBGPD configurations (when the --target-version argument is not given) is now 6.2; previously it was 6.0. Use the --target-version 6.0 command line argument to build 6.0 compatible configurations.

  • Improvement: transit-free ASNs filters are applied also to sessions toward transit-free peers.

    Related: issue #21 on GitHub.

  • Fix (minor): better handling of user answers in configure and setup commands.

  • Fix: clients-from-peeringdb, list of IXPs retrieved from PeeringDB and no longer from IXFDB.

v0.17.3

  • Fix: clients-from-euroix command, use the configured cache directory.

v0.17.2

  • Fix: configure command, omit extended communities for OpenBGPD configurations.

    This is to avoid the need of using the --ignore-issues extended_communities command line argument.

  • Improvement: environment variables expansion when YAML configuration files are read.

v0.17.1

  • Fix: minor installation issues.

v0.17.0

  • New feature: allow to set the source of IRR objects.

    AS-SETs can be prepended with an optional source: RIPE::AS-FOO, RIPE::AS64496:AS-FOO.

  • New feature: support for RPKI-based Origin Validation added to OpenBGPD configurations.

    RPKI ROAs must be loaded from a RIPE RPKI Validator cache file (local or via HTTP). Mostly inspired by Job Snijders’ tool https://github.com/job/rtrsub

  • Improvement: RPKI ROAs can be loaded from a local file too.

    The file must be in RIPE RPKI Validator cache format.

  • Fix (minor): remove internal communities before accepting blackhole routes tagged with a custom blackhole community.

    This bug did not affect routes tagged with the BLACKHOLE community; anyway, the internal communities were scrubbed before routes were announced to clients.

v0.16.2

  • Fix: avoid empty lists of prefixes when a client’s white_list_pref contains only prefixes for an IP version different from the current one.

v0.16.1

v0.16.0

  • Improvement: OpenBGPD, more flexibility for inbound communities values.

    This allows to use inbound ‘peer_as’ communities which overlap with other inbound communities whose last part is a private ASN.

  • New feature: use ARIN Whois database dump to authorize routes.

    This feature allows to accept those routes whose origin ASN is authorized by a client AS-SET, whose prefix has not a corresponding route object but is covered by an ARIN Whois record for the same origin ASN.

  • Improvement: extend the use of RPKI ROAs as route objects and ARIN Whois database dump to tag_as_set-only mode.

    Before of this, the RPKI ROAs as route objects and ARIN Whois DB dump features were used only when origin AS and prefix enforcing was set. Starting with this release they are used even when enforcing is not configured and only the tag_as_set mode is used.

v0.15.0

  • New feature: configure and show_config support commands.

    • configure: it can be used to quickly generate a route server policy definition file (general.yml) on the basis of best practices and suggestions.

    • show_config: to display current configuration settings and also options that have been left to their default values.

  • New feature: ixf-member-export command, to build IX-F Member Export JSON files from the list of clients.

  • Improvement: cache expiry time values can be set for each external resource type: PeeringDB info, IRR data, …

v0.14.1

  • Fix: BIRD, “Unknown instruction 8574 in same (~)” error when reloading IPv6 configurations.

    A missing case for the !~ operator triggers this bug when neighbors are established and trying to reload bird6 configuration.

    Related: issue #20 on GitHub.

v0.14.0

This release breaks backward compatibility (OpenBGPD configs only): for OpenBGPD configurations, starting with this release the Site of Origin Extended BGP communities in the range 65535:* (soo 65535:*) are reserved for internal reasons.

  • New feature: use RPKI ROAs as if they were route objects.

    This feature allows to accept those routes whose origin ASN is authorized by a client AS-SET, whose prefix is not but it is covered by a RPKI ROA for the same origin ASN.

    Related: issue #19 on GitHub.

  • New feature: automatic checking for new releases.

    This can be disabled by setting check_new_release to False in arouteserver.yml.

  • Improvement: routes accepted solely because of a white_list_route entry are now tagged with the route_validated_via_white_list BGP community.

  • Fix: on OpenBGPD configurations, in case of duplicate definition of a client’s AS-SETs, duplicate BGP informational communities were added after the IRR validation process.

v0.13.0

  • New feature: an option to set RFC1997 well-known communities (NO_EXPORT/NO_ADVERTISE) handling policy: pass-through or strict RFC1997 behaviour.

    This breaks backward compatibility: previously, NO_EXPORT/NO_ADVERTISE communities were treated accordingly to the default implementation of the BGP speaker daemon (BIRD, OpenBGPD). Now, ARouteServer’s default setting is to treat routes tagged with those communities transparently, that is to announce them to other clients and to pass-through the original RFC1997 communities.

  • Improvement: when using PeeringDB records to configure the max-prefix limits, a margin is took into account to accomodate networks that fill the PeeringDB records with their exact route announcement count.

    This breaks backward compatibility: if using max-prefix from PeeringDB, current limits will be raised by the default increment values (+100, +15%): this behaviour can be reverted to the pre-v0.13.0 situation by explicitly setting the max_prefix.peering_db.increment configuration section to 0/0.

    Related: issue #12 on GitHub.

  • New feature: client-level white lists for IRRdb-based filters.

    This allows to manually enter routes that must always be accepted by IRRdb-level checks and prefixes and ASNs that must be treated as if they were included within client’s AS-SETs.

    Related: issue #16 on GitHub.

v0.12.3

  • Improvement: always take the AS*n* macro into account when building IRRdb-based filters.

    Related: issue #15 on GitHub.

v0.12.2

  • Fix: an issue on OpenBGPD builder class was preventing features offered via large BGP communities only from being actually implemented into the final configuration.

    Related: issue #11 on GitHub.

v0.12.1

  • Fix an issue that was impacting templates upgrading under certain circumstances.

    Related: issue #10 on GitHub.

v0.12.0

  • OpenBGPD 6.2 support.

  • New feature: Graceful BGP session shutdown support, to honor GRACEFUL_SHUTDOWN communities received from clients and also to perform graceful shutdown of the route server itself (--perform-graceful-shutdown command line argument).

v0.11.0

  • Python 3.4 support.

  • Improvement: GT registry removed from the sources used to gather info from IRRDB.

    Related: PR #8 on GitHub.

  • Improvement: multiple AS-SETs used for the same client are now grouped together and queried at one time. This allows to leverage bgpq3’s ability and speed to aggregate results in order to have smaller configuration files.

v0.10.0

  • New feature: when IRRDB-based filters are enabled and no AS-SETs are configured for a client, if the cfg.filtering.irrdb.peering_db option is set ARouteServer tries to fetch their values from the client’s ASN record on PeeringDB.

    Related: issue #7 on GitHub.

  • Improvement: config building process performances,

    • reduced memory consumption by moving IRRDB information from memory to temporary files;

    • responses for empty/missing resources are also cached;

    • fix a wrong behaviour that led to multiple PeeringDB requests for the same ASN.

  • Improvement: clients-from-euroix command, the new --merge-from-peeringdb option can be used to integrate missing information into the output clients list by fetching AS-SETs and max-prefix limit from PeeringDB.

v0.9.3

  • Fix: OpenBGPD, an issue was causing values > 65535 to be used in standard BGP communities matching.

v0.9.2

  • Fix: remove quotes from clients description.

  • Fix: OpenBGPD, syntax error for prefix lists with ‘range X - X’ format.

  • Fix: clients-from-euroix command, members with multiple vlan objects with the same vlan_id were not properly listed in the output, only the first object was used.

v0.9.1

  • Improvement: BIRD, new default debug options (states, routes, filters, interfaces, events, was all).

    If needed, they can be overwritten using the header custom .local file.

  • Fix: enrichers errors handling reported a generic message with no further details.

  • Fix: HTTP 404 error handling for “Entity not found” error from PeeringDB.

  • Fix: OpenBGPD, large prefix lists were causing a “string too long” error.

  • Fix: OpenBGPD, clients descriptions longer than 31 characters were not properly truncated.

v0.9.0

  • New feature: RTT-based communities to control propagation of routes on the basis of peers round trip time.

  • Improvement: in conjunction with the “tag” reject policy, the rejected_route_announced_by BGP community can be used to track the ASN of the client that announced an invalid route to the server.

  • Fix: when the “tag” reject policy is used, verify that the reject_cause BGP community is also set.

v0.8.1

  • Fix: default user configuration path not working.

v0.8.0

  • New feature: reject policy configuration option, to control how invalid routes must be treated: immediately discarded or kept for troubleshooting purposes, analysis or statistic reporting.

  • New tool: invalid routes reporter.

  • Fix: the following networks have been removed from the bogons.yml file: 193.239.116.0/22, 80.249.208.0/21, 164.138.24.80/29.

v0.7.0

  • New feature: custom BGP communities can be configured on a client-by-client basis to tag routes entering the route server (for example, for informative purposes).

  • Fix: validation of BGP communities configuration for OpenBGPD.

    Error is given if a peer-AS-specific BGP community overlaps with another community, even if the last part of the latter is a private/reserved ASN.

  • Improvement: the custom !include <filepath> statement can be used now in YAML configuration files to include other files.

    More details here.

  • Improvement: IRRDB-based filters can be configured to allow more specific prefixes (allow_longer_prefixes option).

v0.6.0

  • OpenBGPD 6.1 support: enable large BGP communities support.

  • Improvement: the clients-from-peeringdb command now uses the IX-F database to show a list of IXP and their PeeringDB ID.

  • Improvement: enable NEXT_HOP rewriting for IPv6 blackhole filtering requests on OpenBGPD after OpenBSD 6.1 fixup.

    Related: issue #3.

  • Improvement: BIRD, client-level .local file.

  • Improvement: next-hop checks, the authorized_addresses option allows to authorize IP addresses of non-client routers for NEXT_HOP attribute of routes received from a client.

v0.5.0

  • Fix: avoid the use of standard communities in the range 65535:x.

  • Improvement: option to set max-prefix restart timer for OpenBGPD.

  • Deleted feature: tagging of routes a’ la RPKI-Light has been removed.

    • The reject_invalid flag, that previously was on general scope only, now can be set on a client-by-client basis.

    • The roa_valid, roa_invalid, and roa_unknown communities no longer exist.

    Related: issue #4 on GitHub

    This breaks backward compatibility.

  • New feature: BIRD hooks to add site-specific custom implementations.

  • Improvement: BIRD local files.

    This breaks backward compatibility: previously, *.local, *.local4 and *.local6 files that were found in the same directory where the BIRD configuration was stored were automatically included. Now, only the header([4|6]).local and footer([4|6]).local files are included, depending on the values passed to the --use-local-files command line argument.

  • Improvement: setup command and program’s configuration file.

    The default path of the cache directory (cache_dir option) has changed: it was /var/lib/arouteserver and now it is cache, that is a directory which is relative to the cfg_dir option (by default, the directory where the program’s configuration file is stored).

v0.4.0

  • OpenBGPD support (some limitations apply).

  • Add MD5 password support on clients configuration.

  • The build command used to generate route server configurations has been removed in favor of BGP-speaker-specific sub-commands: bird and openbgpd.

v0.3.0

  • New --test-only flag for builder commands.

  • New --clients-from-euroix command to build the clients.yml file on the basis of records from an Euro-IX member list JSON file.

    This also allows the integration with IXP-Manager.

  • New BGP communities: add NO_EXPORT and/or NO_ADVERTISE to any client or to specific peers.

  • New option (set by default) to automatically add the NO_EXPORT community to blackhole filtering announcements.

v0.2.0

  • setup-templates command to just sync local templates with those distributed within a new release.

  • Multithreading support for tasks that acquire data from external sources (IRRDB info, PeeringDB max-prefix).

    Can be set using the threads option in the arouteserver.yml configuration file.

  • New template-context command, useful to dump the list of context variables and data that can be used inside a template.

  • New empty AS-SETs handling: if an AS-SET is empty, no errors are given but only a warning is logged and the configuration building process goes on.

    Any client with IRRDB enforcing enabled and whose AS-SET is empty will have its routes rejected by the route server.

v0.1.2

  • Fix local files usage among IPv4/IPv6 processes.

    Before of this release, only .local files were included into the route server configuration, for both the IPv4 and IPv6 configurations. After this, .local files continue to be used for both the address families but .local4 and .local6 files can also be used to include IP version specific options, depending on the IP version used to build the configuration. Details here.

To upgrade:

# pull from GitHub master branch or use pip:
pip install --upgrade arouteserver

# install the new template files into local system
arouteserver setup

v0.1.1

  • Add local static files into the route server’s configuration.

v0.1.0

  • First beta version.

v0.1.0a11

  • The filtering.rpsl section of general and clients configuration files has been renamed into filtering.irrdb.

  • The command line argument --template-dir has been renamed into --templates-dir.

  • New options in the program’s configuration file: bgpq3_host and bgpq3_sources, used to set bgpq3 -h and -S arguments when gathering info from IRRDBs.

v0.1.0a10

  • New command to build textual representations of configurations: html.

v0.1.0a9

  • New command to initialize a custom live test scenario: init-scenario.

v0.1.0a8

  • New feature: selective path prepending via BGP communities.

  • The control_communities general option has been removed: it was redundant.

v0.1.0a7

  • Improved communities configuration and handling.

  • Fix issue on standard communities matching against 32-bit ASNs.

  • Fix issue on IPv6 prefix validation.

v0.1.0a6

  • New feature: RPKI-based filtering/tagging.

v0.1.0a5

  • New feature: transit-free ASNs filtering.

  • Program command line: subcommands + clients-from-peeringdb.

  • More logging and some warning.

v0.1.0a4

  • Fix issue with GTSM default value.

  • Add default route to bogons.

  • Better as-sets handling and cache handling.

  • Config syntax change: clients ‘as’ -> ‘asn’.

  • AS-SETs at AS-level.

  • Live tests: path hiding mitigation scenario.

  • Improvements in templates.

v0.1.0a3

  • Fix some cache issues.

v0.1.0a2

  • Packaging.

  • System setup via arouteserver --setup.

v0.1.0a1

First push on GitHub.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

arouteserver-0.22.1.tar.gz (190.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page