A Python tool to automatically build (and test) configurations for BGP route servers.
A Python tool to automatically build (and test) feature-rich configurations for BGP route servers.
How it works
Two YAML files provide general policies and clients configurations options:
cfg: rs_as: 64496 router_id: "192.0.2.2" add_path: True filtering: next_hop: policy: "same-as" blackhole_filtering: policy_ipv4: "rewrite-next-hop" ...
clients: - asn: 64511 ip: - "192.0.2.11" - "2001:db8:1:1::11" irrdb: as_sets: - "RIPE::AS-FOO" ...
Jinja2 built-in templates are used to render the final route server’s configuration file.
Currently, BIRD (1.6.3) and OpenBGPD (OpenBSD 6.0, 6.1 and 6.2) are supported, with almost feature parity between them.
Validation and testing are performed using the built-in live tests framework: Docker instances are used to simulate several scenarios, and more custom scenarios can be built on the basis of the user’s needs. More details on the Live tests section.
- Path hiding mitigation techniques (RFC7947 section 2.3.1).
- Basic filters (mostly enabled by default):
- NEXT_HOP enforcement (strict / same AS - RFC7948 section 4.8);
- minimum and maximum IPv4/IPv6 prefix length;
- maximum AS_PATH length;
- reject invalid AS_PATHs (containing private/invalid ASNs);
- reject AS_PATHs containing transit-free ASNs;
- reject bogons;
- max-prefix limit based on global or client-specific values or on PeeringDB data.
- Prefixes and origin ASNs validation (also in tag-only mode):
- IRR-based filters (RFC7948 section 4.6.2);
- AS-SETs configured manually or fetched from PeeringDB;
- support for IRR sources (RIPE::AS-FOO, RADB::AS-BAR);
- white lists support;
- extended dataset for filters generation:
- RPKI ROAs used as route objects;
- Origin AS from ARIN Whois database dump;
- RPKI-based filtering (BGP Prefix Origin Validation).
- Blackhole filtering support:
- optional NEXT_HOP rewriting;
- signalling via BGP Communities (BLACKHOLE and custom communities);
- client-by-client control over propagation.
- Graceful shutdown support:
- honor the GRACEFUL_SHUTDOWN BGP community received from clients (draft-ietf-grow-bgp-gshut-11);
- allow to perform a graceful shutdown of the route server itself.
- Control and informative communities:
- prefix/origin ASN present/not present in IRRDBs data;
- do (not) announce to any / peer / on RTT basis;
- prepend to any / peer / on RTT basis;
- add NO_EXPORT / NO_ADVERTISE to any / peer;
- custom informational BGP communities.
- Optional session features on a client-by-client basis:
- Automatic building of clients list:
- IX-F Member Export JSON files creation.
- Related tools:
- Invalid routes reporter, to log or report invalid routes and their reject reason.
More feature are already planned: see the Future work section for more details.
Full documentation can be found on ReadTheDocs: https://arouteserver.readthedocs.org/
- RIPE74, 10 May 2017, Connect Working Group: video (9:53), slides (PDF)
- Salottino MIX, 30 May 2017: slides
Mentions / endorsements:
- Job Snijders, LINX99, 20 November 2017: slides
Who is using ARouteServer?
Are you using it? Do you want to be listed here? Drop me a message!
Beta testing, looking for testers and reviewers.
Anyone who wants to share his/her point of view, to review the output configurations or to test them is more than welcome!
Upgrade notes: after upgrading, run the arouteserver setup-templates command to sync the local templates with those distributed with the new version. More details on the Upgrading section of the documentation.
- Fix: clients-from-euroix command, use the configured cache directory.
Fix: configure command, omit extended communities for OpenBGPD configurations.
This is to avoid the need of using the --ignore-issues extended_communities command line argument.
Improvement: environment variables expansion when YAML configuration files are read.
- Fix: minor installation issues.
New feature: allow to set the source of IRR objects.
AS-SETs can be prepended with an optional source: RIPE::AS-FOO, RIPE::AS64496:AS-FOO.
New feature: support for RPKI-based Origin Validation added to OpenBGPD configurations.
RPKI ROAs must be loaded from a RIPE RPKI Validator cache file (local or via HTTP). Mostly inspired by Job Snijders’ tool https://github.com/job/rtrsub
Improvement: RPKI ROAs can be loaded from a local file too.
The file must be in RIPE RPKI Validator cache format.
Fix (minor): remove internal communities before accepting blackhole routes tagged with a custom blackhole community.
This bug did not affect routes tagged with the BLACKHOLE community; anyway, the internal communities were scrubbed before routes were announced to clients.
- Fix: avoid empty lists of prefixes when a client’s white_list_pref contains only prefixes for an IP version different from the current one.
- Fix: handle the new version of the JSON schema built by arin-whois-bulk-parser.
Improvement: OpenBGPD, more flexibility for inbound communities values.
This allows to use inbound ‘peer_as’ communities which overlap with other inbound communities whose last part is a private ASN.
New feature: use ARIN Whois database dump to authorize routes.
This feature allows to accept those routes whose origin ASN is authorized by a client AS-SET, whose prefix has not a corresponding route object but is covered by an ARIN Whois record for the same origin ASN.
Improvement: extend the use of RPKI ROAs as route objects and ARIN Whois database dump to tag_as_set-only mode.
Before of this, the RPKI ROAs as route objects and ARIN Whois DB dump features were used only when origin AS and prefix enforcing was set. Starting with this release they are used even when enforcing is not configured and only the tag_as_set mode is used.
- New feature: configure and show_config support commands.
- configure: it can be used to quickly generate a route server policy definition file (general.yml) on the basis of best practices and suggestions.
- show_config: to display current configuration settings and also options that have been left to their default values.
- New feature: ixf-member-export command, to build IX-F Member Export JSON files from the list of clients.
- Improvement: cache expiry time values can be set for each external resource type: PeeringDB info, IRR data, …
This release breaks backward compatibility (OpenBGPD configs only): for OpenBGPD configurations, starting with this release the Site of Origin Extended BGP communities in the range 65535:* (soo 65535:*) are reserved for internal reasons.
New feature: use RPKI ROAs as if they were route objects.
This feature allows to accept those routes whose origin ASN is authorized by a client AS-SET, whose prefix is not but it is covered by a RPKI ROA for the same origin ASN.
Related: issue #19 on GitHub.
New feature: automatic checking for new releases.
This can be disabled by setting check_new_release to False in arouteserver.yml.
Improvement: routes accepted solely because of a white_list_route entry are now tagged with the route_validated_via_white_list BGP community.
Fix: on OpenBGPD configurations, in case of duplicate definition of a client’s AS-SETs, duplicate BGP informational communities were added after the IRR validation process.
New feature: an option to set RFC1997 well-known communities (NO_EXPORT/NO_ADVERTISE) handling policy: pass-through or strict RFC1997 behaviour.
This breaks backward compatibility: previously, NO_EXPORT/NO_ADVERTISE communities were treated accordingly to the default implementation of the BGP speaker daemon (BIRD, OpenBGPD). Now, ARouteServer’s default setting is to treat routes tagged with those communities transparently, that is to announce them to other clients and to pass-through the original RFC1997 communities.
Improvement: when using PeeringDB records to configure the max-prefix limits, a margin is took into account to accomodate networks that fill the PeeringDB records with their exact route announcement count.
This breaks backward compatibility: if using max-prefix from PeeringDB, current limits will be raised by the default increment values (+100, +15%): this behaviour can be reverted to the pre-v0.13.0 situation by explicitly setting the max_prefix.peering_db.increment configuration section to 0/0.
Related: issue #12 on GitHub.
New feature: client-level white lists for IRRdb-based filters.
This allows to manually enter routes that must always be accepted by IRRdb-level checks and prefixes and ASNs that must be treated as if they were included within client’s AS-SETs.
Related: issue #16 on GitHub.
Improvement: always take the AS*n* macro into account when building IRRdb-based filters.
Related: issue #15 on GitHub.
Fix: an issue on OpenBGPD builder class was preventing features offered via large BGP communities only from being actually implemented into the final configuration.
Related: issue #11 on GitHub.
Fix an issue that was impacting templates upgrading under certain circumstances.
Related: issue #10 on GitHub.
Python 3.4 support.
Improvement: GT registry removed from the sources used to gather info from IRRDB.
Related: PR #8 on GitHub.
Improvement: multiple AS-SETs used for the same client are now grouped together and queried at one time. This allows to leverage bgpq3’s ability and speed to aggregate results in order to have smaller configuration files.
New feature: when IRRDB-based filters are enabled and no AS-SETs are configured for a client, if the cfg.filtering.irrdb.peering_db option is set ARouteServer tries to fetch their values from the client’s ASN record on PeeringDB.
Related: issue #7 on GitHub.
Improvement: config building process performances,
- reduced memory consumption by moving IRRDB information from memory to temporary files;
- responses for empty/missing resources are also cached;
- fix a wrong behaviour that led to multiple PeeringDB requests for the same ASN.
Improvement: clients-from-euroix command, the new --merge-from-peeringdb option can be used to integrate missing information into the output clients list by fetching AS-SETs and max-prefix limit from PeeringDB.
- Fix: OpenBGPD, an issue was causing values > 65535 to be used in standard BGP communities matching.
- Fix: remove quotes from clients description.
- Fix: OpenBGPD, syntax error for prefix lists with ‘range X - X’ format.
- Fix: clients-from-euroix command, members with multiple vlan objects with the same vlan_id were not properly listed in the output, only the first object was used.
Improvement: BIRD, new default debug options (states, routes, filters, interfaces, events, was all).
If needed, they can be overwritten using the header custom .local file.
Fix: enrichers errors handling reported a generic message with no further details.
Fix: HTTP 404 error handling for “Entity not found” error from PeeringDB.
Fix: OpenBGPD, large prefix lists were causing a “string too long” error.
Fix: OpenBGPD, clients descriptions longer than 31 characters were not properly truncated.
- New feature: RTT-based communities to control propagation of routes on the basis of peers round trip time.
- Improvement: in conjunction with the “tag” reject policy, the rejected_route_announced_by BGP community can be used to track the ASN of the client that announced an invalid route to the server.
- Fix: when the “tag” reject policy is used, verify that the reject_cause BGP community is also set.
- Fix: default user configuration path not working.
- New feature: reject policy configuration option, to control how invalid routes must be treated: immediately discarded or kept for troubleshooting purposes, analysis or statistic reporting.
- New tool: invalid routes reporter.
- Fix: the following networks have been removed from the bogons.yml file: 184.108.40.206/22, 220.127.116.11/21, 18.104.22.168/29.
New feature: custom BGP communities can be configured on a client-by-client basis to tag routes entering the route server (for example, for informative purposes).
Fix: validation of BGP communities configuration for OpenBGPD.
Error is given if a peer-AS-specific BGP community overlaps with another community, even if the last part of the latter is a private/reserved ASN.
Improvement: the custom !include <filepath> statement can be used now in YAML configuration files to include other files.
More details here.
Improvement: IRRDB-based filters can be configured to allow more specific prefixes (allow_longer_prefixes option).
OpenBGPD 6.1 support: enable large BGP communities support.
Improvement: the clients-from-peeringdb command now uses the IX-F database to show a list of IXP and their PeeringDB ID.
Improvement: enable NEXT_HOP rewriting for IPv6 blackhole filtering requests on OpenBGPD after OpenBSD 6.1 fixup.
Related: issue #3.
Improvement: BIRD, client-level .local file.
Improvement: next-hop checks, the authorized_addresses option allows to authorize IP addresses of non-client routers for NEXT_HOP attribute of routes received from a client.
Fix: avoid the use of standard communities in the range 65535:x.
Improvement: option to set max-prefix restart timer for OpenBGPD.
Deleted feature: tagging of routes a’ la RPKI-Light has been removed.
- The reject_invalid flag, that previously was on general scope only, now can be set on a client-by-client basis.
- The roa_valid, roa_invalid, and roa_unknown communities no longer exist.
Related: issue #4 on GitHub
This breaks backward compatibility.
New feature: BIRD hooks to add site-specific custom implementations.
Improvement: BIRD local files.
This breaks backward compatibility: previously, *.local, *.local4 and *.local6 files that were found in the same directory where the BIRD configuration was stored were automatically included. Now, only the header([4|6]).local and footer([4|6]).local files are included, depending on the values passed to the --use-local-files command line argument.
Improvement: setup command and program’s configuration file.
The default path of the cache directory (cache_dir option) has changed: it was /var/lib/arouteserver and now it is cache, that is a directory which is relative to the cfg_dir option (by default, the directory where the program’s configuration file is stored).
- OpenBGPD support (some limitations apply).
- Add MD5 password support on clients configuration.
- The build command used to generate route server configurations has been removed in favor of BGP-speaker-specific sub-commands: bird and openbgpd.
New --test-only flag for builder commands.
New BGP communities: add NO_EXPORT and/or NO_ADVERTISE to any client or to specific peers.
New option (set by default) to automatically add the NO_EXPORT community to blackhole filtering announcements.
setup-templates command to just sync local templates with those distributed within a new release.
Multithreading support for tasks that acquire data from external sources (IRRDB info, PeeringDB max-prefix).
Can be set using the threads option in the arouteserver.yml configuration file.
New template-context command, useful to dump the list of context variables and data that can be used inside a template.
New empty AS-SETs handling: if an AS-SET is empty, no errors are given but only a warning is logged and the configuration building process goes on.
Any client with IRRDB enforcing enabled and whose AS-SET is empty will have its routes rejected by the route server.
Fix local files usage among IPv4/IPv6 processes.
Before of this release, only .local files were included into the route server configuration, for both the IPv4 and IPv6 configurations. After this, .local files continue to be used for both the address families but .local4 and .local6 files can also be used to include IP version specific options, depending on the IP version used to build the configuration. Details here.
# pull from GitHub master branch or use pip: pip install --upgrade arouteserver # install the new template files into local system arouteserver setup
- Add local static files into the route server’s configuration.
- First beta version.
- The filtering.rpsl section of general and clients configuration files has been renamed into filtering.irrdb.
- The command line argument --template-dir has been renamed into --templates-dir.
- New options in the program’s configuration file: bgpq3_host and bgpq3_sources, used to set bgpq3 -h and -S arguments when gathering info from IRRDBs.
- New command to build textual representations of configurations: html.
- New command to initialize a custom live test scenario: init-scenario.
- New feature: selective path prepending via BGP communities.
- The control_communities general option has been removed: it was redundant.
- Improved communities configuration and handling.
- Fix issue on standard communities matching against 32-bit ASNs.
- Fix issue on IPv6 prefix validation.
- New feature: RPKI-based filtering/tagging.
- New feature: transit-free ASNs filtering.
- Program command line: subcommands + clients-from-peeringdb.
- More logging and some warning.
- Fix issue with GTSM default value.
- Add default route to bogons.
- Better as-sets handling and cache handling.
- Config syntax change: clients ‘as’ -> ‘asn’.
- AS-SETs at AS-level.
- Live tests: path hiding mitigation scenario.
- Improvements in templates.
- Fix some cache issues.
- System setup via arouteserver --setup.
First push on GitHub.