Tools for working with AWS Organizations
Project description
A python library for managing resources across all accounts in an AWS Organization.
OrgCrawler package exports two primary classes:
- orgcrawler.orgs.Org
provides a data model and methods for querying AWS Organizations resources:
accounts
organizational units
service control policies
- orgcrawler.crawlers.Crawler
provides a framework for executing user defined python code in all accounts and regions or a subset thereof.
OrgCrawler also contains two console scripts: orgquery and orgcrawler. These attempt to provide a generic interface for running organization queries and custom crawler functions from the commandline.
See full documentation as https://orgcrawler.readthedocs.io/en/latest/
Currently orgcrawler is tested in python 3.6, 3.7.
Installation
pip install orgcrawler
The Org object
Usage
Create an orgcrawler.orgs.Org instance:
master_account_id = orgcrawler.utils.get_master_account_id('myOrgMasterRole') my_org = orgcrawler.orgs.Org(master_account_id, 'myOrgMasterRole')
Load your organization’s account and organizational unit resources into your instance:
my_org.load()
Query your organization’s account and organizational unit resources:
all_accounts = my_org.list_accounts_by_name() all_org_units = my_org.list_org_units() test_accounts = my_org.list_accounts_in_ou('testing')
Details
A newly initialized Org object only contains master account information.
The load() method makes boto3 client calls to the master account to populate the Org object’s Account and OrganizationalUnit resource data.
The loaded Org object is cached to local disk to improve the performance of subsequent load() calls. After an hour, the cached Org object is timed out. It contains no AWS account credentials.
Org object query methods can return listings of accounts based on OrganizationalUnit membership. See the API docs for a full listing of query methods
The Crawler object
The orgcrawler.crawlers.Crawler object provides a framework for running python functions in all accounts and regions in your organization or a subset thereof.
Create an orgcrawler.crawlers.Crawler instance:
my_crawler = orgcrawler.crawlers.Crawler( my_org, access_role='myAccountAdminRole', accounts=test_accounts, regions=['us-west-1', 'us-east-1'], )
Load AWS credentials for each account into your Crawler instance:
my_crawler.load_account_credentials()
Shortcut Crawler Setup
You can generate an Crawler object and associated Org object with a single utility function:
from orgcrawler.cli.utils import setup_crawler my_crawler = crawler_setup('MyMasterAccountRole')
or:
crawler_params = dict( access_role='myAccountAdminRole', accounts=test_accounts, regions=['us-west-1', 'us-east-1'], ) my_crawler = crawler_setup('MyMasterAccountRole', **crawler_params)
Running Crawler Payload Functions
Run your custom python code in all account/regions configured in your Crawler by calling the execute() method of your Crawler instance. Supply your payload function and any function arguments as parameters:
import my_payloads all_buckets = my_crawler.execute(my_payloads.list_s3_buckets) my_crawler.execute(my_payloads.create_bucket, 'my_bucket')
Requirments for Payload Functions
The Crawler.execute method calls your custom function with the following parameters: region, account, *args, where region is a string, account is orgcrawler.orgs.OrgAccount instance, and args is a list of positional parameters to pass to your function. Your function must create its own boto3 client for whatever services it will use.
Examples:
def list_s3_buckets(region, account): client = boto3.client('s3', region_name=region, **account.credentials) response = client.list_buckets() def create_bucket(region, account, bucket_prefix): client = boto3.client('s3', region_name=region, **account.credentials) bucket_name = '-'.join([bucket_prefix, account.id, region]) bucket_attributes = {'Bucket': bucket_name} if not region == 'us-east-1': bucket_attributes['CreateBucketConfiguration'] = {'LocationConstraint': region} response = client.create_bucket(**bucket_attributes)
Running payloads which call global AWS services
When calling a payload function which accesses a global service, set the regions attribute of your Crawler instance either to us-east-1 or to the keywork GLOBAL. Otherwise, Crawler.execute() will travers every AWS region and redandantly run your payload:
my_crawler = crawler_setup('MyMasterAccountRole') my_crawler.update_regions('GLOBAL') my_crawler.execute(iam_list_users)
OrgCrawler CLI Scripts
This package contains two console scripts: orgquery and orgcrawler. These attempt to provide a generic interface for running organization queries and custom crawler functions. They provide concrete examples for how to build orgcrawler applications.
See orgcrawler/cli/{orgquery|orgcrawler}.py for code.
Run with the --help option for usage.
CLI Examples
orgquery
orgquery -h orgquery -r OrgMasterRole list_accounts_by_name orgquery -r OrgMasterRole -f yaml get_account_id_by_name webapps orgquery -r OrgMasterRole get_targets_for_policy LimitAWSRegions | jq -re .[].Name
orgcrawler
orgcrawler -h orgcrawler -r OrgMasterRole orgcrawler.payloads.list_buckets orgcrawler -r OrgMasterRole --account-role S3Admin orgcrawler.payloads.create_bucket orgcrawler-testbucket orgcrawler -r OrgMasterRole --service codecommit --payload-file ~/my_payloads.py list_cc_repositories orgcrawler -r OrgMasterRole --accounts app-test,app-prod --regions us-east-1,us-west-2 orgcrawler.payloads.config_describe_rules
When running a payload function which calls a global AWS service such as IAM or Route53, use the --service option. This will set the regions attibute of your crawler object to the default region us-east-1:
orgcrawler -r OrgMasterRole --service iam orgcrawler.payloads.iam_list_users orgcrawler -r OrgMasterRole --service route53 orgcrawler.payloads.list_hosted_zones
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for orgcrawler-1.0.0a2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f68a161261113929dc86e37acf66647f45c89d3cb4fb150b8cbc548bbc4024b |
|
MD5 | ee2a76cd2a20a40f5c0c8dfb641cf07d |
|
BLAKE2b-256 | e011b23f1d4d0a62d4f41f8c0197541d598fdd9aefb8a6eb1c5cb92dcbeceed7 |