Flattens SPF records and updates Cloudflare zones
Project description
Cloudflare SPF Flattener
Finding sender-policy-flattener (spflat)
A few weeks ago one of our SaaS vendors added a couple of additional DNS entries to their SPF record (without telling us...).
We started to get reports of email deliver failures. Looking into it we discovered the number of DNS entries in our SPF record was the problem. The SaaS vendors change put us over the maximum of 10.
We've decided future SaaS vendors and partners will send from a subdomain, but for existing ones there are too many actors - both internal and external to accomplish this change any time soon.
Hence we started researching SPF flattening as the best solution, but were concerned about the vendors making changes to the sender records.
Looking for a solution to that probkem we stumbled upon the sender-policy-flattener project. This tool - spflat
- solved many problems and has some excellent features and characteristics.
- The project is opensource
- It will query your list of approved sendors and flatten these into a list of appropriately formatted and sized spf records composed of
ip4
andip6
spf entries. - Provide these entries as a chain of spf
include
entries. - Should any of your the senders dns entries change, it sends email indicating the changes and the the corrections required to your spf records.
spflat
works reliably and accurately- Can run entirely from a JSON configuration file.
- Written in Python, and easy to install and configure.
sendier-policy-flattner
did 90% of what we desired for flattening and managing our SPF records.
Requirements that led to this project
After using splat
for a bit we come up with a wish list - the other 10% of what we needed.
Here is the wishlist of requirements we came up with:
-
The ability to run
spflat
in the command line, without the email being sent. (a--no-email
switch) -
Our public DNS zones are in Cloudflare. We wanted
spflat
to AUTOMATICALLY update the necessary resource records in Cloudflare when a change is detected. (a--update
switch). -
Ability to have
spflat
re-create the SPF zone records in Cloudflare DNS for zone repair (and initial configuration.) (a--force-update
switch) -
Have the output status file (sums file) update ONLY if the zones are updated.
- original
spflat
replaces the sums file each time it ran, so in monitoring mode a change was reported only once.
- original
-
sender-policy-flattener
is a fine tool, so we did not want to reinvent the wheel:-
Utilized
spflat
by supplementing its functionality. -
A supplement, not a replacement.
-
-
Keep the changes opensource:
-
Recognize the debt to
sender-policy-flattener
-
Make the changes avialable under the same licenses.
-
To limit confusion, change the command from
spflat
tocfspflat
-
Hence cfspflat
- CloudFlare Sender Policy Flattener was created.
cfspflat
- Cloudflare Sender Policy Flattener
Quick overview:
% cfspflat -h
usage: cfspflat [-h] [-c CONFIG] [-o OUTPUT] [--update-records] [--force-update]
[--no-email]
A script that crawls and compacts SPF records into IP networks. This helps to
avoid exceeding the DNS lookup limit of the Sender Policy Framework (SPF)
https://tools.ietf.org/html/rfc7208#section-4.6.4
optional arguments:
-h, --help show this help message and exit
-c CONFIG, --config CONFIG
Name/path of JSON configuration file (default:
spfs.json)
-o OUTPUT, --output OUTPUT
Name/path of output file (default spf_sums.json)
--update-records Update SPF records in CloudFlare
--force-update Force an update of SPF records in Cloudflare
--no-email don't send the email
- The
sender-policy-flattener
module is installed as part ofcfspflat
- The existing core of
spflat
is kept mostly intact, so the basic features are maintained bycfspflat
. - The changes to accomodate
cfspflat
were in the parameter handling and adding the Cloudflare updates to the processing look. - The
cloudflare
library, with some abstraction classes, is used to make the zone updates. cfspflat
eliminates many of the command arguments of spflat in favor of using the json config file.- Cloudflare TXT records are automatically generated and updated when the configuration changes.
- With
cfspflat
you can completely automate your SPF flattening using cfspflat with the--update
switch in a cron job, even silently with the--no-email
switch
Installing and Configuring cfspflat
1. pip install the cfspflat
% pip install cfspflat
- But it's advisable to do this in its own venv:
% python3 -m venv spfenv
% source spfenv/bin/activate
% pip install cfspflat
- pip will install the prerequisites, including the
sender-policy-flattner
(spflat),dnspython
,netaddr
, andcloudflare
python modules. - The executable is installed in bin of the venv as
cfspflat
2. Create an anchor SPF record for the zone apex in Cloudflare
Create the TXT SPF record on zone apex used (e.g. example.com), At the end of this anchor record include the first SPF record that slpat will write - spf0.example.com
- we also include our own
ip4
andip6
entries in this anchor record.
example.com TXT "v=spf1 mx include:spf0.example.com -all"
- This anchor record is never changed by
cfspflat
. It's purpose is to link to the first SPF record in the chain thatcfspflat
manages.
2. Edit the cfspflat configuration file
Create a spfs.json file. Add all the entries required:
cfspflat
uses the same configuration and sums file formats as the originalspflat
.- If you already use spflat you can use thos files as is with cfspflat.
- There is one extension - the "update_subject" entry containing the subject of the email sent when cfspflat has updated your SPF records. This message will contain the same detail spflat provides.
spfs.json
is the default name of the config file, but it can be specified with the--config
switch.- Here is an example config file:
Example spfs.json configuration file:
{
"sending domains": {
"example.edu": {
"amazonses.com": "txt",
"goodwebsolutions.net": "txt",
.... more sender spf's here ....
"delivery.xyz.com": "txt",
"spf.protection.outlook.com": "txt"
}
},
"resolvers": [
"1.1.1.1", "8.8.8.8"
],
"email": {
"to": "dnsadmins@example.com",
"from": "spf_monitor@example.com",
"subject": "[WARNING] SPF Records for {zone} have changed and should be updated.",
"update_subject" : "[NOTICE] SPF Records for {zone} have been updated.",
"server": "smtp.example.com"
},
"output": "monitor_sums.json"
}
Config file details
- The
sending domains
section is required and contains sending domain entries: this is your sender domain (e.g. example.com for j.smith@example.com, noreply.example.com for deals@noreply.example.com ) There can be multiple sending domains in the config file. - Each sending domain contains dns spf records for the dns
include
records of your approved senders.These dns names are resolved and flattened:- These entries are in the key-value pairs of : .
- Record type can be "txt" (other SPF records), "A" and "AAAA" records (for specific hosts).
- The
resolvers
section is optional (using the system default DNS resolvers if none are supplied) - The
email
stanza is required and is global (i.e. for allsending domains
). This section includes:subject
(optional) is the email subject if a change was detected but no updates were made. The default message is the one shown in the example.update_subject
(optional) is the email subject if a change was detected and the dns records were updated. The default message is shown in the example.to
- is required - this is the destination for emails sent bycfspflat
from
- is required - the source email of the messagescfspflat
sendsserver
- is required - your mail relay.
output
is the file that maintains the existing state and checksum of sender records. If this is not specifiedspfs_sum.json
is used.
Output file details
- The
output
file is a JSON file only updated if it is new (empty) or the records have been updated. - Likewise the default output file is
spf_sums.json
but can be changed in the config file or by the--output
switch. - This contains the list of flattened spf records and a checksum used to assess changes in senders records.
- Because you recieve emails of detected changes or updates, there is little reason to care about the output file.
3. Create a credentials file for Cloudflare
There are a couple of locations and formats for the API credentials Cloudfare requires.
- Consult cloudflare documentation and the dashboard for creating API credentials
- Consult the python-cloudflare site for documentation on passing credentials.
- For simplicity you can use the
.cloudflare.cfg
file in either your home directory or in the directory you will run cfspflat.
[CloudFlare]
email = dnsadmin@example.org
token = 1234567890abc...abc
- It should go without saying - protect the API keys and this file.
- It's also possible to pass the credentials as environment variables.
4. Run cfspflat
to build the sums file and SPF entries
- Run cfspflat twice:
% cfspflat --no-email
% cfspflat --force
- The first time constructs the base records and the second time forces the dns updates.
- With force update the DNS records are created even if a change hasn't been detected.
- A list of the records will be sent to your email.
5. Automate cfspflat
to the level you are comfortable with
- You are up and running:
- You can run
cfspflat
in advisory mode (likespflat
) sending you emails notifying of changes - Or you can run it with the
--update-records
switch and update your records automatically whenever they change (still giving you notifications of the changes made.)
- You can run
Example email format
- Example from
sender-policy-flattener
README:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file cfspflat-22.2.17.tar.gz
.
File metadata
- Download URL: cfspflat-22.2.17.tar.gz
- Upload date:
- Size: 13.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.7.1 importlib_metadata/4.6.3 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8e4ed52cadec645239d8db7f502754527a03490ac6e1e16b05ea57898dc2d5af |
|
MD5 | c0313951bcb6621eb6f55673d61bec2f |
|
BLAKE2b-256 | 0d0ca47e30b7bd828ad82bde9df6233bd4447d36ba66e44e34a037baed4cfe61 |
File details
Details for the file cfspflat-22.2.17-py3-none-any.whl
.
File metadata
- Download URL: cfspflat-22.2.17-py3-none-any.whl
- Upload date:
- Size: 10.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.7.1 importlib_metadata/4.6.3 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.0 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a76baf16aae4016132f92b1d8b3240a7e215b5432ce5b3ba567ff065be95027 |
|
MD5 | 11920393a09415b4334a8d07d7f337d2 |
|
BLAKE2b-256 | 2da26e68d91c6a817de8d3ad8b193fac79fed0293cf2304a3620057679218629 |