Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

Compare directories, efficiently sync changes to AWS, and invalidate CDNs.

Project description

https://travis-ci.org/gebn/wood.svg?branch=master https://coveralls.io/repos/github/gebn/wood/badge.svg?branch=master

Wood is a toolbox for representing directory trees, calculating comparisons between them, and effecting those changes in S3, CloudFront and Cloudflare. It started as a set of deploy scripts for my personal website, but grew into something that I was too chuffed with to keep hidden away in the bowls of another website repository. It’s not as refined as, say, requests, but should be useful to anyone looking to efficiently deploy a website to AWS, and/or using Cloudflare as a CDN.

Notably, unlike the AWS CLI tools or boto3, wood does not rely on file timestamps, instead using MD5 checksums to make the minimum number of changes possible in S3. It also has an algorithm for aggregating invalidation paths to make the CloudFront free tier go as far as possible. Syncers and invalidators are implemented in a generic way, allowing easy extension to additional services as needed.

Features

  • Tree representation
  • Tree comparison
    • Action that comparison in S3
    • Invalidate paths as necessary in CloudFront and Cloudflare

Demo

import wood

# low-level comparison of two local directories
comparison = wood.compare('~/dir', '~/.snapshot/hourly.1/dir')
comparison.new()       # files added since the snapshot
comparison.modified()  # files modified since the snapshot
comparison.deleted()   # files deleted since the snapshot


import pathlib
import boto3

local_base = pathlib.Path('/path/to/web/root')
bucket = boto3.resource('s3').Bucket('example.com')

# create representations of the local and remote trees
to_deploy = wood.root(local_base)
deployed = wood.s3.objects_to_root(bucket.objects.all())

# compare the two as if they were local directories
comparison = wood.compare(to_deploy, deployed)

# write all changes (additions, modifications, deletions) to the S3 bucket
syncer = wood.s3.Syncer(local_base, bucket)
syncer.sync(comparison)

# invalidate the minimum amount in CloudFront to ensure the changes are
# visible, using prefix grouping where possible
cloudfront = boto3.client('cloudfront')
invalidator = wood.cloudfront.Invalidator(cloudfront, '{distribution}',
                                          '{reference}')
invalidator.invalidate(comparison)

# do the same for Cloudflare in the case of a second CDN
invalidator = wood.cloudflare.Invalidator(sess, email, key, zone,
                                         'https://example.com/')
invalidator.invalidate(comparison)

Why “wood”?

Because it started as a tree comparison tool, and grew out from there. It’s also a little rough around the edges.

Project details


Release history Release notifications

This version
History Node

1.0.3

History Node

1.0.2

History Node

1.0.1

History Node

1.0.0

History Node

1.0.0rc2

History Node

1.0.0rc1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
wood-1.0.3-py2.py3-none-any.whl (20.5 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Sep 23, 2017
wood-1.0.3.tar.gz (15.6 kB) Copy SHA256 hash SHA256 Source None Sep 23, 2017

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page