A Python tool to automatically build (and test) configurations for BGP route servers.
Project description
ARouteServer
A Python tool to automatically build (and test) feature-rich configurations for BGP route servers.
How it works
Two YAML files provide general policies and clients configurations options:
cfg: rs_as: 64496 router_id: "192.0.2.2" add_path: True filtering: next_hop: policy: "same-as" blackhole_filtering: policy_ipv4: "rewrite-next-hop" ...clients: - asn: 64511 ip: - "192.0.2.11" - "2001:db8:1:1::11" irrdb: as_sets: - "RIPE::AS-FOO" ...ARouteServer acquires external information to enrich them: i.e. bgpq3 for IRR data, PeeringDB for max-prefix limit and AS-SETs, …
Jinja2 built-in templates are used to render the final route server’s configuration file.
Currently, BIRD (1.6.3 and 1.6.4) and OpenBGPD (OpenBSD 6.1 up to 6.4) are supported, with almost feature parity between them.
Validation and testing are performed using the built-in live tests framework: Docker instances are used to simulate several scenarios, and more custom scenarios can be built on the basis of the user’s needs. More details on the Live tests section.
Features
Path hiding mitigation techniques (RFC7947 section 2.3.1).
Basic filters (mostly enabled by default):
NEXT_HOP enforcement (strict / same AS - RFC7948 section 4.8);
minimum and maximum IPv4/IPv6 prefix length;
maximum AS_PATH length;
reject invalid AS_PATHs (containing private/invalid ASNs);
reject AS_PATHs containing transit-free ASNs;
reject bogons;
max-prefix limit based on global or client-specific values or on PeeringDB data.
Prefixes and origin ASNs validation (also in tag-only mode):
IRR-based filters (RFC7948 section 4.6.2);
AS-SETs configured manually or fetched from PeeringDB;
support for IRR sources (RIPE::AS-FOO, RADB::AS-BAR);
white lists support;
extended dataset for filters generation:
RPKI ROAs used as route objects;
Origin AS from ARIN Whois database dump;
NIC.BR Whois data (slide n. 26) from Registro.br;
RPKI-based filtering (BGP Prefix Origin Validation).
Blackhole filtering support:
optional NEXT_HOP rewriting;
signalling via BGP Communities (BLACKHOLE and custom communities);
client-by-client control over propagation.
Graceful shutdown support:
honor the GRACEFUL_SHUTDOWN BGP community received from clients (draft-ietf-grow-bgp-gshut-11);
allow to perform a graceful shutdown of the route server itself.
Control and informative communities:
prefix/origin ASN present/not present in IRRDBs data;
do (not) announce to any / peer / on RTT basis;
prepend to any / peer / on RTT basis;
add NO_EXPORT / NO_ADVERTISE to any / peer;
custom informational BGP communities.
Optional session features on a client-by-client basis:
prepend route server ASN (RFC7947 section 2.2.2.1);
active sessions;
GTSM (Generalized TTL Security Mechanism - RFC5082);
ADD-PATH capability (RFC7911).
Automatic building of clients list:
integration with IXP-Manager;
fetch lists from PeeringDB records and Euro-IX member list JSON files.
IX-F Member Export JSON files creation.
Related tools:
Invalid routes reporter, to log or report invalid routes and their reject reason.
A comprehensive list of features can be found within the comments of the distributed configuration file on GitHub or on the documentation web page.
More feature are already planned: see the Future work section for more details.
Full documentation
Full documentation can be found on ReadTheDocs: https://arouteserver.readthedocs.org/
Presentations
RIPE74, 10 May 2017, Connect Working Group: video (9:53), slides (PDF)
Salottino MIX, 30 May 2017: slides
Mentions / endorsements:
Job Snijders, LINX99, 20 November 2017: slides
Anurag Bhatia, APNIC46, 12 September 2018: video, slides (PDF)
Claudio Jeker, RIPE Labs, 28 November 2018: OpenBGPD - Adding Diversity to the Route Server Landscape.
Who is using ARouteServer?
BharatIX, BIRD.
CATNIX, BIRD.
IX-Denver, BIRD.
MBIX, BIRD.
PIT-IX, BIRD.
SwissIX, OpenBGPD.
Unmetered.Exchange, BIRD.
YXEIX, BIRD.
YYCIX, OpenBGPD.
Are you using it? Do you want to be listed here? Drop me a message!
Status
Beta testing, looking for testers and reviewers.
Anyone who wants to share his/her point of view, to review the output configurations or to test them is more than welcome!
Bug? Issues? Support requests?
But also suggestions? New ideas?
Please create an issue on GitHub or drop me a message.
A Slack channel is also available on the network.toCode() community: arouteserver.
Change log
v0.21.1
Deprecation: SAVVIS IRR removed from the list of default sources used by bgpq3.
Fix (minor): truncate the max length of AS-SET names to 64 characters.
BIRD supports only names no longer than 64 characters.
Related: issue #47 on GitHub.
v0.21.0
Improvement: when ripe-rpki-validator-cache is set as the source of ROAs, multiple URLs can now be specified to fetch data from.
URLs will be tried in the same order as they are configured; if the attempt to download ROAs from the first URL fails, the second URL will be tried, an so on.
By default, the RIPE NCC public instance of the RIPE RPKI Validator will be tried first, then the NTT instance. The list of URLs can be set in the general.yml configuration file, roas.ripe_rpki_validator_url option.
v0.20.0
This is the last release of ARouteServer for which OpenBGPD/OpenBSD 6.1 and 6.2 CI tests are ran. From the next release, any new feature will not be tested against these versions of OpenBGPD. Users are encouraged to move to newer releases.
New: add support for OpenBGPD/OpenBSD 6.4 enhancements.
Use new sets for prefixes, ASNum, and origins (prefix + source-as), and also RPKI ROA sets.
Improvement: OpenBGPD, reduce the number of rules by combining some into the same rule.
Improvement: route server policies definition files built using the configure command now have RPKI BGP Origin Validation and “use-ROAs-as-route-objects” enabled by default.
As announced with release 0.19.0, OpenBGPD/OpenBSD 6.0 is no longer tested. The implementation of new features may break compatibility of the configurations built for unsupported releases.
Most of this release is based on the work made by Claudio Jeker.
v0.19.1
Fix (BIRD configuration only): change bgp_path.last with bgp_path.last_nonaggregated.
When a route is originated from the aggregation of two different routes using the AS_SET, bgp_path.last always returns 0, so the origin ASN validation against IRR always fails.
Related: issue #34 on GitHub.
v0.19.0
This is the last release of ARouteServer for which OpenBGPD/OpenBSD 6.0 CI tests are ran. Starting with the next release, any new feature will not be tested against version 6.0 of OpenBGPD. Users are encouraged to move to newer releases.
New: use NIC.BR Whois data from Registro.br to enrich the dataset used for route validation.
Details: RIPE76, Practical Data Sources For BGP Routing Security.
Related: issue #28 on GitHub.
New: introduce support for OpenBGPD/OpenBSD 6.4.
OpenBSD 6.4 is not released yet, this is just in preparation of it.
Related: issue #31 on GitHub.
Fix (minor): RIPE NCC RPKI Validator v3 expects Accept: text/json as HTTP header.
Related: PR #29 on GitHub.
v0.18.0
New: add support for BIRD 1.6.4 and OpenBGPD/OpenBSD 6.3.
This release breaks backward compatibility (OpenBGPD configs only): the default target version used to build OpenBGPD configurations (when the --target-version argument is not given) is now 6.2; previously it was 6.0. Use the --target-version 6.0 command line argument to build 6.0 compatible configurations.
Improvement: transit-free ASNs filters are applied also to sessions toward transit-free peers.
Related: issue #21 on GitHub.
Fix (minor): better handling of user answers in configure and setup commands.
Fix: clients-from-peeringdb, list of IXPs retrieved from PeeringDB and no longer from IXFDB.
v0.17.3
Fix: clients-from-euroix command, use the configured cache directory.
v0.17.2
Fix: configure command, omit extended communities for OpenBGPD configurations.
This is to avoid the need of using the --ignore-issues extended_communities command line argument.
Improvement: environment variables expansion when YAML configuration files are read.
v0.17.1
Fix: minor installation issues.
v0.17.0
New feature: allow to set the source of IRR objects.
AS-SETs can be prepended with an optional source: RIPE::AS-FOO, RIPE::AS64496:AS-FOO.
New feature: support for RPKI-based Origin Validation added to OpenBGPD configurations.
RPKI ROAs must be loaded from a RIPE RPKI Validator cache file (local or via HTTP). Mostly inspired by Job Snijders’ tool https://github.com/job/rtrsub
Improvement: RPKI ROAs can be loaded from a local file too.
The file must be in RIPE RPKI Validator cache format.
Fix (minor): remove internal communities before accepting blackhole routes tagged with a custom blackhole community.
This bug did not affect routes tagged with the BLACKHOLE community; anyway, the internal communities were scrubbed before routes were announced to clients.
v0.16.2
Fix: avoid empty lists of prefixes when a client’s white_list_pref contains only prefixes for an IP version different from the current one.
v0.16.1
Fix: handle the new version of the JSON schema built by arin-whois-bulk-parser.
v0.16.0
Improvement: OpenBGPD, more flexibility for inbound communities values.
This allows to use inbound ‘peer_as’ communities which overlap with other inbound communities whose last part is a private ASN.
New feature: use ARIN Whois database dump to authorize routes.
This feature allows to accept those routes whose origin ASN is authorized by a client AS-SET, whose prefix has not a corresponding route object but is covered by an ARIN Whois record for the same origin ASN.
Improvement: extend the use of RPKI ROAs as route objects and ARIN Whois database dump to tag_as_set-only mode.
Before of this, the RPKI ROAs as route objects and ARIN Whois DB dump features were used only when origin AS and prefix enforcing was set. Starting with this release they are used even when enforcing is not configured and only the tag_as_set mode is used.
v0.15.0
New feature: configure and show_config support commands.
configure: it can be used to quickly generate a route server policy definition file (general.yml) on the basis of best practices and suggestions.
show_config: to display current configuration settings and also options that have been left to their default values.
New feature: ixf-member-export command, to build IX-F Member Export JSON files from the list of clients.
Improvement: cache expiry time values can be set for each external resource type: PeeringDB info, IRR data, …
v0.14.1
Fix: BIRD, “Unknown instruction 8574 in same (~)” error when reloading IPv6 configurations.
A missing case for the !~ operator triggers this bug when neighbors are established and trying to reload bird6 configuration.
Related: issue #20 on GitHub.
v0.14.0
This release breaks backward compatibility (OpenBGPD configs only): for OpenBGPD configurations, starting with this release the Site of Origin Extended BGP communities in the range 65535:* (soo 65535:*) are reserved for internal reasons.
New feature: use RPKI ROAs as if they were route objects.
This feature allows to accept those routes whose origin ASN is authorized by a client AS-SET, whose prefix is not but it is covered by a RPKI ROA for the same origin ASN.
Related: issue #19 on GitHub.
New feature: automatic checking for new releases.
This can be disabled by setting check_new_release to False in arouteserver.yml.
Improvement: routes accepted solely because of a white_list_route entry are now tagged with the route_validated_via_white_list BGP community.
Fix: on OpenBGPD configurations, in case of duplicate definition of a client’s AS-SETs, duplicate BGP informational communities were added after the IRR validation process.
v0.13.0
New feature: an option to set RFC1997 well-known communities (NO_EXPORT/NO_ADVERTISE) handling policy: pass-through or strict RFC1997 behaviour.
This breaks backward compatibility: previously, NO_EXPORT/NO_ADVERTISE communities were treated accordingly to the default implementation of the BGP speaker daemon (BIRD, OpenBGPD). Now, ARouteServer’s default setting is to treat routes tagged with those communities transparently, that is to announce them to other clients and to pass-through the original RFC1997 communities.
Improvement: when using PeeringDB records to configure the max-prefix limits, a margin is took into account to accomodate networks that fill the PeeringDB records with their exact route announcement count.
This breaks backward compatibility: if using max-prefix from PeeringDB, current limits will be raised by the default increment values (+100, +15%): this behaviour can be reverted to the pre-v0.13.0 situation by explicitly setting the max_prefix.peering_db.increment configuration section to 0/0.
Related: issue #12 on GitHub.
New feature: client-level white lists for IRRdb-based filters.
This allows to manually enter routes that must always be accepted by IRRdb-level checks and prefixes and ASNs that must be treated as if they were included within client’s AS-SETs.
Related: issue #16 on GitHub.
v0.12.3
Improvement: always take the AS*n* macro into account when building IRRdb-based filters.
Related: issue #15 on GitHub.
v0.12.2
Fix: an issue on OpenBGPD builder class was preventing features offered via large BGP communities only from being actually implemented into the final configuration.
Related: issue #11 on GitHub.
v0.12.1
Fix an issue that was impacting templates upgrading under certain circumstances.
Related: issue #10 on GitHub.
v0.12.0
OpenBGPD 6.2 support.
New feature: Graceful BGP session shutdown support, to honor GRACEFUL_SHUTDOWN communities received from clients and also to perform graceful shutdown of the route server itself (--perform-graceful-shutdown command line argument).
v0.11.0
Python 3.4 support.
Improvement: GT registry removed from the sources used to gather info from IRRDB.
Related: PR #8 on GitHub.
Improvement: multiple AS-SETs used for the same client are now grouped together and queried at one time. This allows to leverage bgpq3’s ability and speed to aggregate results in order to have smaller configuration files.
v0.10.0
New feature: when IRRDB-based filters are enabled and no AS-SETs are configured for a client, if the cfg.filtering.irrdb.peering_db option is set ARouteServer tries to fetch their values from the client’s ASN record on PeeringDB.
Related: issue #7 on GitHub.
Improvement: config building process performances,
reduced memory consumption by moving IRRDB information from memory to temporary files;
responses for empty/missing resources are also cached;
fix a wrong behaviour that led to multiple PeeringDB requests for the same ASN.
Improvement: clients-from-euroix command, the new --merge-from-peeringdb option can be used to integrate missing information into the output clients list by fetching AS-SETs and max-prefix limit from PeeringDB.
v0.9.3
Fix: OpenBGPD, an issue was causing values > 65535 to be used in standard BGP communities matching.
v0.9.2
Fix: remove quotes from clients description.
Fix: OpenBGPD, syntax error for prefix lists with ‘range X - X’ format.
Fix: clients-from-euroix command, members with multiple vlan objects with the same vlan_id were not properly listed in the output, only the first object was used.
v0.9.1
Improvement: BIRD, new default debug options (states, routes, filters, interfaces, events, was all).
If needed, they can be overwritten using the header custom .local file.
Fix: enrichers errors handling reported a generic message with no further details.
Fix: HTTP 404 error handling for “Entity not found” error from PeeringDB.
Fix: OpenBGPD, large prefix lists were causing a “string too long” error.
Fix: OpenBGPD, clients descriptions longer than 31 characters were not properly truncated.
v0.9.0
New feature: RTT-based communities to control propagation of routes on the basis of peers round trip time.
Improvement: in conjunction with the “tag” reject policy, the rejected_route_announced_by BGP community can be used to track the ASN of the client that announced an invalid route to the server.
Fix: when the “tag” reject policy is used, verify that the reject_cause BGP community is also set.
v0.8.1
Fix: default user configuration path not working.
v0.8.0
New feature: reject policy configuration option, to control how invalid routes must be treated: immediately discarded or kept for troubleshooting purposes, analysis or statistic reporting.
New tool: invalid routes reporter.
Fix: the following networks have been removed from the bogons.yml file: 193.239.116.0/22, 80.249.208.0/21, 164.138.24.80/29.
v0.7.0
New feature: custom BGP communities can be configured on a client-by-client basis to tag routes entering the route server (for example, for informative purposes).
Fix: validation of BGP communities configuration for OpenBGPD.
Error is given if a peer-AS-specific BGP community overlaps with another community, even if the last part of the latter is a private/reserved ASN.
Improvement: the custom !include <filepath> statement can be used now in YAML configuration files to include other files.
More details here.
Improvement: IRRDB-based filters can be configured to allow more specific prefixes (allow_longer_prefixes option).
v0.6.0
OpenBGPD 6.1 support: enable large BGP communities support.
Improvement: the clients-from-peeringdb command now uses the IX-F database to show a list of IXP and their PeeringDB ID.
Improvement: enable NEXT_HOP rewriting for IPv6 blackhole filtering requests on OpenBGPD after OpenBSD 6.1 fixup.
Related: issue #3.
Improvement: BIRD, client-level .local file.
Improvement: next-hop checks, the authorized_addresses option allows to authorize IP addresses of non-client routers for NEXT_HOP attribute of routes received from a client.
v0.5.0
Fix: avoid the use of standard communities in the range 65535:x.
Improvement: option to set max-prefix restart timer for OpenBGPD.
Deleted feature: tagging of routes a’ la RPKI-Light has been removed.
The reject_invalid flag, that previously was on general scope only, now can be set on a client-by-client basis.
The roa_valid, roa_invalid, and roa_unknown communities no longer exist.
Related: issue #4 on GitHub
This breaks backward compatibility.
New feature: BIRD hooks to add site-specific custom implementations.
Improvement: BIRD local files.
This breaks backward compatibility: previously, *.local, *.local4 and *.local6 files that were found in the same directory where the BIRD configuration was stored were automatically included. Now, only the header([4|6]).local and footer([4|6]).local files are included, depending on the values passed to the --use-local-files command line argument.
Improvement: setup command and program’s configuration file.
The default path of the cache directory (cache_dir option) has changed: it was /var/lib/arouteserver and now it is cache, that is a directory which is relative to the cfg_dir option (by default, the directory where the program’s configuration file is stored).
v0.4.0
OpenBGPD support (some limitations apply).
Add MD5 password support on clients configuration.
The build command used to generate route server configurations has been removed in favor of BGP-speaker-specific sub-commands: bird and openbgpd.
v0.3.0
New --test-only flag for builder commands.
New --clients-from-euroix command to build the clients.yml file on the basis of records from an Euro-IX member list JSON file.
This also allows the integration with IXP-Manager.
New BGP communities: add NO_EXPORT and/or NO_ADVERTISE to any client or to specific peers.
New option (set by default) to automatically add the NO_EXPORT community to blackhole filtering announcements.
v0.2.0
setup-templates command to just sync local templates with those distributed within a new release.
Multithreading support for tasks that acquire data from external sources (IRRDB info, PeeringDB max-prefix).
Can be set using the threads option in the arouteserver.yml configuration file.
New template-context command, useful to dump the list of context variables and data that can be used inside a template.
New empty AS-SETs handling: if an AS-SET is empty, no errors are given but only a warning is logged and the configuration building process goes on.
Any client with IRRDB enforcing enabled and whose AS-SET is empty will have its routes rejected by the route server.
v0.1.2
Fix local files usage among IPv4/IPv6 processes.
Before of this release, only .local files were included into the route server configuration, for both the IPv4 and IPv6 configurations. After this, .local files continue to be used for both the address families but .local4 and .local6 files can also be used to include IP version specific options, depending on the IP version used to build the configuration. Details here.
To upgrade:
# pull from GitHub master branch or use pip:
pip install --upgrade arouteserver
# install the new template files into local system
arouteserver setupv0.1.1
Add local static files into the route server’s configuration.
v0.1.0
First beta version.
v0.1.0a11
The filtering.rpsl section of general and clients configuration files has been renamed into filtering.irrdb.
The command line argument --template-dir has been renamed into --templates-dir.
New options in the program’s configuration file: bgpq3_host and bgpq3_sources, used to set bgpq3 -h and -S arguments when gathering info from IRRDBs.
v0.1.0a10
New command to build textual representations of configurations: html.
v0.1.0a9
New command to initialize a custom live test scenario: init-scenario.
v0.1.0a8
New feature: selective path prepending via BGP communities.
The control_communities general option has been removed: it was redundant.
v0.1.0a7
Improved communities configuration and handling.
Fix issue on standard communities matching against 32-bit ASNs.
Fix issue on IPv6 prefix validation.
v0.1.0a6
New feature: RPKI-based filtering/tagging.
v0.1.0a5
New feature: transit-free ASNs filtering.
Program command line: subcommands + clients-from-peeringdb.
More logging and some warning.
v0.1.0a4
Fix issue with GTSM default value.
Add default route to bogons.
Better as-sets handling and cache handling.
Config syntax change: clients ‘as’ -> ‘asn’.
AS-SETs at AS-level.
Live tests: path hiding mitigation scenario.
Improvements in templates.
v0.1.0a3
Fix some cache issues.
v0.1.0a2
Packaging.
System setup via arouteserver --setup.
v0.1.0a1
First push on GitHub.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
