cloudrecon

Cloud platform data storage bucket and blob finder and crawler

Check out the cloudrecon docs

Installation

NOTE: cloudrecon requires python version >=3.6

$ pip install cloudrecon

Usage

usage: cloudrecon [-h] [-o file] [-d] [-p] [-t seconds] [-v] [-c num] [-ct CLOUDTYPE]
                  word_list [word_list ...]

positional arguments:
  word_list                        read words from one or more <word-list> files

optional arguments:
  -h, --help                       show this help message and exit
  -o file, --output file           Write output to <file>
  -d, --db                         Write output to database
  -p, --public                     Only include 'public' buckets in the output
  -t seconds, --timeout seconds    HTTP request timeout in <seconds> (default: 30)
  -v, --version                    show program's version number and exit
  -c num, --concurrency num        maximum <num> of concurrent requests (default: 8)
  -ct CLOUDTYPE, --cloudtype CLOUDTYPE
                                   Input which cloud platform to query: "AWS", "GCP", "Azure", or
                                   "Alibaba"

Example 1: Output to a json file

1. Download a word-list.

The SecLists repository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.

$ curl -sSfL -o "word-list.txt" "https://raw.githubusercontent.com/3of3/cloudrecon/master/data/words.txt"

2. Run cloudrecon.

Execute cloudrecon using the word-list.txt file and output the public buckets/blobs to a json file named results.json.

$ cloudrecon "word-list.txt" -o "results.json" --public

- PRIVATE https://s3.sa-east-1.amazonaws.com/test-lyft
- PRIVATE https://s3.ap-south-1.amazonaws.com/test.amazon
+ PUBLIC https://walmart-dev.s3.us-east-1.amazonaws.com
- PRIVATE https://s3.ap-southeast-1.amazonaws.com/apple-prod
- PRIVATE https://walmart.s3.ap-southeast-1.amazonaws.com
...

3. Inspect the results.

Check the results.json output file to view the buckets/blobs you have discovered!

$ cat "results.json"

{
    "public": {
        "total": 12,
        "hits": [
            "https://walmart-dev.s3.us-east-1.amazonaws.com",
            "https://apple-production.s3.ap-southeast-1.amazonaws.com",
            ...
        ]
    }
}

Note: to include private buckets/blobs in the results omit the -p, --public flag from the command.

4. Crawl the results.

Enumerate the static files located in each bucket/blob and record the findings.

Coming soon!

Example 2: Output to a MongoDB database

1. Download a word-list.

The SecLists repository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.

$ curl -sSfL -o "word-list.txt" "https://raw.githubusercontent.com/3of3/cloudrecon/master/data/words.txt"

2. Start an instance of MongoDB

$ docker run --name "mongo" -p 27017:27017 -v "mongodb_data:/data/db" -v "mongodb_config:/data/configdb" -d mongo

3. Run cloudrecon.

Execute cloudrecon using the word-list.txt file and output to MongoDB instance.

$ cloudrecon "word-list.txt" --db

- PRIVATE https://s3.sa-east-1.amazonaws.com/test-lyft
- PRIVATE https://s3.ap-south-1.amazonaws.com/test.amazon
+ PUBLIC https://walmart-dev.s3.us-east-1.amazonaws.com
- PRIVATE https://s3.ap-southeast-1.amazonaws.com/apple-prod
- PRIVATE https://walmart.s3.ap-southeast-1.amazonaws.com
...

3. Inspect the results.

Check the MongoDB database: cloudrecon collection: hits to view the buckets/blobs you have discovered!

$ mongo "cloudrecon" --quiet --eval 'db.hits.find({}, {"url": 1, "access": 1, "_id": 0}).limit(5)'

{ "url" : "https://s3.us-east-2.amazonaws.com/apple", "access" : "private" }
{ "url" : "https://s3.us-west-1.amazonaws.com/microsoft-dev", "access" : "private" }
{ "url" : "https://s3.us-west-1.amazonaws.com/dev-microsoft", "access" : "private" }
{ "url" : "https://s3.us-east-2.amazonaws.com/amazon", "access" : "private" }
{ "url" : "https://s3.us-east-1.amazonaws.com/dev-amazon", "access" : "private" }

4. Crawl the results.

Enumerate the static files located in each bucket and record the findings.

Coming soon!

FAQ

Q: How do I configure this utility?

A:

cloudrecon can be configure using a yaml configuration file located in either the current working directory (e.g. ./cloudrecon.yml) or your home diretory (e.g. ~/cloudrecon.yml).

The following is the list of configurable values:

# cloudrecon.yml

database: { host: "0.0.0.0", port: 27017 }

separators: ["-", ".", ""]

environments: ["", "0", "1", ... "asset"]

aws-regions: ["ap-northeast-1", "ap-northeast-2", ...]

alibaba-regions: ["cn-hangzhou", "cn-shanghai", ...]

To see the full list of configurable values (and their defaults) please refer to the cloudrecon.yml file in this repository.

Q: How can I customize the AWS or Alibaba regions?

A:

The AWS and Alibaba regions can be altered by setting the regions array in your cloudrecon.yml configuration file.

# cloudrecon.yml

aws-regions: ["ap-northeast-1", "ap-northeast-2", ...]

alibaba-regions: ["cn-hangzhou", "cn-shanghai", ...]

Q: How do I customize the environment values used in the recon?

A:

The environments are modifiers permuted with each item of the word-list (and the separator) to construct the bucket value in request. The value can be altered by setting the environments array in your cloudrecon.yml configuration file.

For example, to only search lines from the word-list verbatim (i.e. without modification) you can set this value to an empty array.

FYI, AWS only allows for the non-alphanumeric characters "-" and "." to be used within a bucketname. More research is required to determine what characters are allowed within Alibaba, GCP, and Azure.

# cloudrecon.yml

environments: []

Q: How do I customize the MongoDB host and port?

A:

The database host and port can be configured by altering the database map in your cloudrecon.yml configuration file.

For example, host and port can be set directly inside the database map

# cloudrecon.yml

database: {
  host: "0.0.0.0",
  port: 27017
}

Q: How do I use a database other than MongoDB?

A:

Sorry, at the moment only MongoDB is supported.

Going Forward

• Integrate my own s3content script into this script for an all in one capability
• Write this tool in GoLang!! Make it faster!

Disclaimer

This tools is distributed for educational and security purposes. I take no responsibility and assume no liability for the manner in which this tool is used.

License

MIT © Nathaniel "Q" Quist