Manipulate McAfee ATD appliance
Project description
Robust ATD CLI tools
================
[![GitHub release](https://img.shields.io/github/release/shadowbq/robust-atd.svg?style=for-the-badge)](https://github.com/shadowbq/robust-atd/releases)
[![license](https://img.shields.io/github/license/shadowbq/robust-atd.svg?style=for-the-badge)](/LICENSE)
[![GitHub Release Date](https://img.shields.io/github/release-date/shadowbq/robust-atd.svg?style=for-the-badge)](https://github.com/shadowbq/robust-atd/releases)
[![Code](https://img.shields.io/badge/Language-Python--2.7-ff69b4.svg?style=for-the-badge)](/README.md)
"Robust" is a set of tools to leverage the HTTPS REST API of the [McAfee Advanced Threat Detection](http://www.mcafee.com/us/products/advanced-threat-defense.aspx) 3.8 - 4.x appliance.
## Tools Overview
* `robust` : basic cli submission of a single piece of malware to a MATD server.
* `robust-profiles` : list the available MATD profiles
* `robust-search` : search MATD via MD5 for a report.
* `robust-watchdog` : monitor a directory for files and submit `multithreaded` to MATD
* `robust-convict` : submit `multithreaded` a directory filled with samples and sort into malicious, clean, error, etc.
* `robust-reporter` : parse offline the json files returned during large batch submissions.
* `robust-version-checker` : Check the MATD Server Version
## Important
This is *not a supported or official application of McAfee*. This work is based off of publicly available published documentation for integrating with the McAfee ATD REST API 3.6.x to 4.x
Official API Documentation is available here:
* https://support.mcafee.com/ServicePortal/faces/knowledgecenter?q=api&v=&p=Advanced+Threat+Defense
## McAfee ATD - Advanced Threat defense
McAfee ATD is a commercial grade enterprise security sandbox analysis appliance. It main function is to provide advanced detection for stealthy, zero-day malware. McAfee Advanced Threat Defense is available as an on-premises appliance or a virtual form factor, with support for both private and public cloud with availability in the Azure Marketplace.
* https://www.mcafee.com/us/products/advanced-threat-defense.aspx
* https://www.mcafee.com/us/resources/data-sheets/ds-advanced-threat-defense.pdf
## Install
Req: Python 2.7.x.
Bug #5: https://github.com/shadowbq/robust-atd/issues/5 - ~~`pip install robust-atd`~~
Note: Python 3.x is not supported.
### PKG Download & Manual Install Alternative
Note: `python setup.py install` will attempt to install dependencies from the internet via `pip`.
For offline runtime installation, please download the pip packages listed in the `requirements.txt`.
### Virutalenv
It is recommended to install virtualenv & virtualenvwrapper via `Virtualenv Burrito`.
See: [README_PYTHON_UP.md](/README_PYTHON_UP.md)
```
$> mkvirtualenv robust
$> workon robust
$(robust)> wget https://github.com/shadowbq/robust-atd/archive/master.zip
$(robust)> unzip master.zip
$(robust)> cd master
$(robust)> python setup.py install
```
-or-
```
$> mkvirtualenv --python=python2.7 robust
$> workon robust
$(robust)> pip install robust-atd
```
### Robust (DOT) Configuration file
Robust will use the `~\.robust` configuration file to load defaults into the scripts.
The configuration file is broken into multiple sections. If you use a section you must define all the settings in that section.
* [auth]
* [connection]
* [convict]
It is recommended to set the file to `read-write` only for the current user, and remove all world `(-)rwx` permissions.
Authentication Section `[auth]` :
```shell
$(robust)> cat ~/.robust
[auth]
user: admin
password: password.
```
Connection Detail Section `[connection]` :
```shell
$(robust)> cat ~/.robust
[connection]
ip: atd.localhost.localdomain
skipssl: true
maxthreads: 15
```
Data Storage Section `[storage]`:
Note: Datastorage locations will be created if they do not exist.
```shell
$(robust)> cat ~/.robust
[storage]
severity: 3
cleandir: ~/robust/clean
dirtydir: ~/robust/dirty
reportdir: ~/robust/reports
errordir: ~/robust/errors
```
This file is expanded via the `os` module is compliant with windows user directories.
## Robust:
Using `robust` for submitting samples.
```
usage: robust.py [-h] [-u USER] [-p PASSWORD] [-i ATD IP] [-n] -s
FILE_TO_UPLOAD -a ANALYZER_PROFILE [-v] [--version]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output verbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: admin)
-p PASSWORD (p)assword for username
(default: password.)
-i ATD IP (i)p or hostname address of ATD
(default: atd.localhost.localdomain)
-n do (n)ot verify the SSL certificate for the communications
(default: False)
Sample parameters:
-s FILE_TO_UPLOAD (s)ample or file to be analyzed
(default: None)
-a ANALYZER_PROFILE (a)nalyzer profile id to be used during analysis
(default: None)
Examples:
robust.py -u admin -p password. -i atd.localhost.localdomain -s /usr/local/bin/file_to_scan -a 1
```
### Submitting a Sample
A sample can be submitted via cli with full flags, `.robust` configuration file, or interrupt passwords.
```shell
$(robust)> robust.py -u admin -p password. -i atd.localhost.localdomain -s /home/malware/non-malicious-container/putty_upx_7.exe
```
Using interrupt (interactive) passwords:
```shell
$(robust)> robust-profiles.py -n -l
Password: <input password>
ATD profiles: 1
Profile id: 26
Name: Win XP Down Select (Online)
OS: winXPsp3
Run all down selects?: Off
******************
```
### Managing Outputs
Using System Return codes with `-q` Quiet output flag. When the quiet flag is
used for submitting samples or searching reports the *severity* of the application
is returned as a system exit/return code. Negative return codes indicate *faults*
or failure during submission.
```
(robust)>$ robust.py -n -a 26 -s ./.samples/Sample.exe -q
(robust)>$ echo $?
2
```
Common Fault codes:
```
-1 ---> Error connecting to the ATD Server
-2 ---> Error uploading file to the ATD Server
-3 ---> Analysis failed
-4 ---> Error getting report
-5 ---> Error Obtaining vmprofilelist
```
Malware ranking:
(If the severity level of the sample is 3 and above it is generally regarded a threat)
```
N/A -> Sample did not run
-1 --> Sample is white listed
0 ---> No malicious activity detected (None)
1 ---> Sample is slightly suspicious (Low)
2 ---> Sample is somewhat/probably is suspicious
3 ---> Sample is malicious (Medium)
4 ---> Sample is malicious
5 ---> Sample is malicious (Very High)
```
## robust-version-checker
You can quickly test your connection settings in the CLI.
```
$(robust)> robust-version-checker.py -u robust -p password. -i atd.example.com -n
Connection successful...
Session Value: g7aenj99pfp0gbrogfbqsd9085
User ID: 57
ATD ver: 4.2.2.16
ATD Box heartbeat: 1519939175
```
## robust-profiles
A tool designed to pull the *Analyzer Profile* policy list available to a specific user.
Pulling the Policy List - In order to submit a sample using `robust` you must identify the Analyzer Profile ID. `robust-profiles` assists in identifying the available profiles your user can submit samples to.
```
$(robust)> robust-profiles.py -n -l
ATD profiles: 10
Profile id: 1
Name: Android
OS: android
Run All Selected?: Off
******************
Profile id: 26
Name: Win XP Down Select (Online)
OS: winXPsp3
Run All Selected?: Off
******************
Profile id: 25
Name: Windows XP Full Run (Offline)
OS: winXPsp3
Run All Selected?: On
******************
Profile id: 24
Name: Windows XP Full Run (Online)
OS: winXPsp3
Run All Selected?: On
******************
```
## robust-search
A tool designed to search and return reports for a specific md5 hash.
```shell
(robust)$> $ robust-search.py -m 2F7568342339CDB8321B52FF7BEBE661 -n
(Sample.exe:2F7568342339CDB8321B52FF7BEBE661) = 2: "Sample probably is suspicious"
```
### Help details
`robust-search` has the options `-w` and `-t` to collect the proper report on the submission.
```
usage: robust-search.py [-h] [-u USER] [-p PASSWORD] [-i ATD IP] [-n] -m MD5
[-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}]
[-w FILENAME] [--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: admin)
-p PASSWORD (p)assword for username
(default: password.)
-i ATD IP (i)p or hostname address of ATD
(default: atd.localhost.localdomain)
-n do (n)ot verify the SSL certificate for the communications
(default: False)
Search parameters:
-m MD5 (m)d5 32bit hash of the sample to search
(default: None)
Reporting parameters:
-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}
(t)ype of report requested
(default: None)
-w FILENAME (w)rite filename for saving the requested report
(default: None)
```
## robust-watchdog
A tool that watches a directory recursively for any new files to submit.
Example CLI
```
usage: robust-watchdog.py [-h] -u USER [-p PASSWORD] -i ATD IP [-n] -a
ANALYZER_PROFILE -d DIRECTORY [-e] [-j MAXTHREADS]
[--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: None)
-p PASSWORD (p)assword for username
(default: None)
-i ATD IP (i)p or hostname address of ATD
(default: None)
-n do (n)ot verify the SSL certificate for the communications
(default: False)
Watch parameters:
-f (f)ollow and watch the directory for new files to submit
(default: True)
-a ANALYZER_PROFILE (a)nalyzer profile id to be used during analysis
(default: None)
-d DIRECTORY (d)irectory to watch for events
(default: None)
-e (e)xisting files in directory will be submitted
(default: False)
-j MAXTHREADS (j) max number of threads
(default: 1)
```
Let it run in a shell and open another one or the file browser to create files in the /path/to/directory. Since the handler is printing the results, the output will reflect the flags chosen similar to `robust.py`:
The `-e` flag can be passed to cause all existing files in the directory (recurisively) to be submitted upon start.
```shell
(robust)$> robust-watchdog.py -a 26 -d ./ -n -e
.
...
.
.....
```
## robust-convict
`robust-convict` is a tool designed like `robust-watchdog` but its purpose is to help sort large directories of malware samples into directories, while downloading their corresponding reports.
Example Usage
```
robust-convict.py -n -a 26 -c ./tmp/clean/ -x ./tmp/dirty/ -r ./tmp/reports/ -z ./tmp/errors/ -d ./tmp/preprocess -j 10 -t zip -q
```
Options
```
usage: robust-convict.py [-h] [-u USER] [-p PASSWORD] [-i ATD IP] [-n] -a
ANALYZER_PROFILE -d DIRECTORY [-e] [-y SEVERITY]
[-c CLEANDIR] [-x DIRTYDIR] [-r REPORTDIR]
[-z ERRORDIR]
[-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}]
[-j MAXTHREADS] [--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: robust)
-p PASSWORD (p)assword for username
(default: ****<.robust>*****)
-i ATD IP (i)p or hostname address of ATD
(default: atd.localhost.localdomain)
-n do (n)ot verify the SSL certificate for the communications
(default: True)
Watch parameters:
-f (f)ollow and watch the directory for new files to submit
(default: False)
-a ANALYZER_PROFILE (a)nalyzer profile id to be used during analysis
(default: None)
-d DIRECTORY (d)irectory to watch for events
(default: None)
-e (e)xisting files in directory will be submitted
(default: False)
-j MAXTHREADS (j) max number of threads
(default: 1)
Convict parameters:
-y SEVERITY (y) treat sample as dirty with this severity [0-5] or higher
(default: 3)
-c CLEANDIR (c) move clean files to this directory
(default: ~/robust/clean/)
-x DIRTYDIR (x) move processed dirty files to this directory
(default: ~/robust/malware/)
-r REPORTDIR (r) save reports to this directory
(default: ~/robust/reports/)
-z ERRORDIR (z) move error or skip files to this directory
(default: ~/robust/errors/)
-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}
(t)ype of report requested
(default: None)
```
## robust-reporter
`robust-reporter` is a tool designed to quickly summarize the downloaded `*.json` files in your 'reports' directory.
Options
```
usage: robust-reporter.py [-h] [-r REPORTDIR] [--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Reporter parameters:
-r REPORTDIR (r) reports are processed or stored using this directory
(default: ~/robust/reports/)
```
Sample Run
```
$ robust-reporter.py
82344C9864B0F1D120C0D1AB7F7C54C3 (---) : Somewhat/probably is suspicious(Low-Medium) - 24sec
D012492123E4CF0CFB3A017A2E92C077 (Malware.Dynamic) : Malicious(High) - 194sec
DB273A97C54E3E23F411EA7C9B5A82DA (Malware.Dynamic) : Malicious (Medium) - 53sec
165A36C02B3FAAF4DE38F93A3DCB821B (---) : Somewhat/probably is suspicious(Low-Medium) - 36sec
D10195670651A40C46C22972CD839E89 (Artemis!D10195670651) : Malicious (Very High) - 32sec
8271093E0E78574428BBDDDA6F34A980 (Malware.Dynamic) : Malicious(High) - 192sec
86DAFA0262BF217F5344A3B057C0DB06 (Malware.Dynamic) : Malicious(High) - 193sec
8DA4CDC3E2EE16021F237EA7A043DA8E (Malware.Dynamic) : Malicious(High) - 191sec
```
## Tunning for Linux File Watchers
### iNotify Tuning Parameters
The inotify(7) subsystem has three important tunings that impact robust's directory watching.
```
/proc/sys/fs/inotify/max_user_instances impacts how many different root dirs you can watch.
/proc/sys/fs/inotify/max_user_watches impacts how many dirs you can watch across all watched roots.
/proc/sys/fs/inotify/max_queued_events impacts how likely it is that your system will experience a notification overflow.
```
You obviously need to ensure that `max_user_instances` and `max_user_watches` are set so that the system is capable of keeping track of your files.
`max_queued_events` is important to size correctly; if it is too small, the kernel will drop events and robust won't be able to report on them. Making this value bigger reduces the risk of this happening.
# Developers
## Install Development
```
$(robust)> pip install -r devel-requirements.txt
```
## Development Tasks
```shell
(robust)$> invoke -l
Available tasks:
build Build the setup.py
clean Clean up docs, bytecode, and extras
codestats Run flake8 PeP8 tests for code stats
release ``version`` should be a string like '0.4' or '1.0'.
smell Run flake8 PeP8 tests
test Run Unit tests
```
### Running the Test Suite
Nose is run via `invoke test`
```
Clearing rm -rf build
Clearing rm -rf dist
Clearing rm -rf *.egg-info
Clearing rm -rf pyclient.log
Clearing rm -rf **/**/*.pyc
Clearing rm -rf **/*.pyc
Clearing rm -rf ./*.pyc
...................
-----------------------------------------------------------------------------
19 tests run in 0.3 seconds (19 tests passed)
117 E501 line too long (97 > 79 characters)
7 F401 'ratd' imported but unused
1 F841 local variable 'rb_rtnv' is assigned to but never used
1 N802 function name should be lowercase
5 W601 .has_key() is deprecated, use 'in'
```
### Additional LICENSE information
A modified Fork of `atdcli.py` (Carlos Munoz - 2014).
https://pypi.python.org/pypi/atd
## VX Workshop Appliance Option
There is a fully operational Xubuntu 14.04 liveCD that includes:
* robust - https://github.com/shadowbq/robust-atd
* maltrieve - https://github.com/shadowbq/maltrieve
* vxcage - https://github.com/shadowbq/vxcage
It also includes
* hexeditors
* static analysis tools
* google chrome
* vmtools
* etc..
xubuntu-14.04.4-desktop-x86_64-VX-Workshop-0.4.iso (~ 1.2 GB)
Available to download with READMEs here: https://goo.gl/flcvew
================
[![GitHub release](https://img.shields.io/github/release/shadowbq/robust-atd.svg?style=for-the-badge)](https://github.com/shadowbq/robust-atd/releases)
[![license](https://img.shields.io/github/license/shadowbq/robust-atd.svg?style=for-the-badge)](/LICENSE)
[![GitHub Release Date](https://img.shields.io/github/release-date/shadowbq/robust-atd.svg?style=for-the-badge)](https://github.com/shadowbq/robust-atd/releases)
[![Code](https://img.shields.io/badge/Language-Python--2.7-ff69b4.svg?style=for-the-badge)](/README.md)
"Robust" is a set of tools to leverage the HTTPS REST API of the [McAfee Advanced Threat Detection](http://www.mcafee.com/us/products/advanced-threat-defense.aspx) 3.8 - 4.x appliance.
## Tools Overview
* `robust` : basic cli submission of a single piece of malware to a MATD server.
* `robust-profiles` : list the available MATD profiles
* `robust-search` : search MATD via MD5 for a report.
* `robust-watchdog` : monitor a directory for files and submit `multithreaded` to MATD
* `robust-convict` : submit `multithreaded` a directory filled with samples and sort into malicious, clean, error, etc.
* `robust-reporter` : parse offline the json files returned during large batch submissions.
* `robust-version-checker` : Check the MATD Server Version
## Important
This is *not a supported or official application of McAfee*. This work is based off of publicly available published documentation for integrating with the McAfee ATD REST API 3.6.x to 4.x
Official API Documentation is available here:
* https://support.mcafee.com/ServicePortal/faces/knowledgecenter?q=api&v=&p=Advanced+Threat+Defense
## McAfee ATD - Advanced Threat defense
McAfee ATD is a commercial grade enterprise security sandbox analysis appliance. It main function is to provide advanced detection for stealthy, zero-day malware. McAfee Advanced Threat Defense is available as an on-premises appliance or a virtual form factor, with support for both private and public cloud with availability in the Azure Marketplace.
* https://www.mcafee.com/us/products/advanced-threat-defense.aspx
* https://www.mcafee.com/us/resources/data-sheets/ds-advanced-threat-defense.pdf
## Install
Req: Python 2.7.x.
Bug #5: https://github.com/shadowbq/robust-atd/issues/5 - ~~`pip install robust-atd`~~
Note: Python 3.x is not supported.
### PKG Download & Manual Install Alternative
Note: `python setup.py install` will attempt to install dependencies from the internet via `pip`.
For offline runtime installation, please download the pip packages listed in the `requirements.txt`.
### Virutalenv
It is recommended to install virtualenv & virtualenvwrapper via `Virtualenv Burrito`.
See: [README_PYTHON_UP.md](/README_PYTHON_UP.md)
```
$> mkvirtualenv robust
$> workon robust
$(robust)> wget https://github.com/shadowbq/robust-atd/archive/master.zip
$(robust)> unzip master.zip
$(robust)> cd master
$(robust)> python setup.py install
```
-or-
```
$> mkvirtualenv --python=python2.7 robust
$> workon robust
$(robust)> pip install robust-atd
```
### Robust (DOT) Configuration file
Robust will use the `~\.robust` configuration file to load defaults into the scripts.
The configuration file is broken into multiple sections. If you use a section you must define all the settings in that section.
* [auth]
* [connection]
* [convict]
It is recommended to set the file to `read-write` only for the current user, and remove all world `(-)rwx` permissions.
Authentication Section `[auth]` :
```shell
$(robust)> cat ~/.robust
[auth]
user: admin
password: password.
```
Connection Detail Section `[connection]` :
```shell
$(robust)> cat ~/.robust
[connection]
ip: atd.localhost.localdomain
skipssl: true
maxthreads: 15
```
Data Storage Section `[storage]`:
Note: Datastorage locations will be created if they do not exist.
```shell
$(robust)> cat ~/.robust
[storage]
severity: 3
cleandir: ~/robust/clean
dirtydir: ~/robust/dirty
reportdir: ~/robust/reports
errordir: ~/robust/errors
```
This file is expanded via the `os` module is compliant with windows user directories.
## Robust:
Using `robust` for submitting samples.
```
usage: robust.py [-h] [-u USER] [-p PASSWORD] [-i ATD IP] [-n] -s
FILE_TO_UPLOAD -a ANALYZER_PROFILE [-v] [--version]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output verbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: admin)
-p PASSWORD (p)assword for username
(default: password.)
-i ATD IP (i)p or hostname address of ATD
(default: atd.localhost.localdomain)
-n do (n)ot verify the SSL certificate for the communications
(default: False)
Sample parameters:
-s FILE_TO_UPLOAD (s)ample or file to be analyzed
(default: None)
-a ANALYZER_PROFILE (a)nalyzer profile id to be used during analysis
(default: None)
Examples:
robust.py -u admin -p password. -i atd.localhost.localdomain -s /usr/local/bin/file_to_scan -a 1
```
### Submitting a Sample
A sample can be submitted via cli with full flags, `.robust` configuration file, or interrupt passwords.
```shell
$(robust)> robust.py -u admin -p password. -i atd.localhost.localdomain -s /home/malware/non-malicious-container/putty_upx_7.exe
```
Using interrupt (interactive) passwords:
```shell
$(robust)> robust-profiles.py -n -l
Password: <input password>
ATD profiles: 1
Profile id: 26
Name: Win XP Down Select (Online)
OS: winXPsp3
Run all down selects?: Off
******************
```
### Managing Outputs
Using System Return codes with `-q` Quiet output flag. When the quiet flag is
used for submitting samples or searching reports the *severity* of the application
is returned as a system exit/return code. Negative return codes indicate *faults*
or failure during submission.
```
(robust)>$ robust.py -n -a 26 -s ./.samples/Sample.exe -q
(robust)>$ echo $?
2
```
Common Fault codes:
```
-1 ---> Error connecting to the ATD Server
-2 ---> Error uploading file to the ATD Server
-3 ---> Analysis failed
-4 ---> Error getting report
-5 ---> Error Obtaining vmprofilelist
```
Malware ranking:
(If the severity level of the sample is 3 and above it is generally regarded a threat)
```
N/A -> Sample did not run
-1 --> Sample is white listed
0 ---> No malicious activity detected (None)
1 ---> Sample is slightly suspicious (Low)
2 ---> Sample is somewhat/probably is suspicious
3 ---> Sample is malicious (Medium)
4 ---> Sample is malicious
5 ---> Sample is malicious (Very High)
```
## robust-version-checker
You can quickly test your connection settings in the CLI.
```
$(robust)> robust-version-checker.py -u robust -p password. -i atd.example.com -n
Connection successful...
Session Value: g7aenj99pfp0gbrogfbqsd9085
User ID: 57
ATD ver: 4.2.2.16
ATD Box heartbeat: 1519939175
```
## robust-profiles
A tool designed to pull the *Analyzer Profile* policy list available to a specific user.
Pulling the Policy List - In order to submit a sample using `robust` you must identify the Analyzer Profile ID. `robust-profiles` assists in identifying the available profiles your user can submit samples to.
```
$(robust)> robust-profiles.py -n -l
ATD profiles: 10
Profile id: 1
Name: Android
OS: android
Run All Selected?: Off
******************
Profile id: 26
Name: Win XP Down Select (Online)
OS: winXPsp3
Run All Selected?: Off
******************
Profile id: 25
Name: Windows XP Full Run (Offline)
OS: winXPsp3
Run All Selected?: On
******************
Profile id: 24
Name: Windows XP Full Run (Online)
OS: winXPsp3
Run All Selected?: On
******************
```
## robust-search
A tool designed to search and return reports for a specific md5 hash.
```shell
(robust)$> $ robust-search.py -m 2F7568342339CDB8321B52FF7BEBE661 -n
(Sample.exe:2F7568342339CDB8321B52FF7BEBE661) = 2: "Sample probably is suspicious"
```
### Help details
`robust-search` has the options `-w` and `-t` to collect the proper report on the submission.
```
usage: robust-search.py [-h] [-u USER] [-p PASSWORD] [-i ATD IP] [-n] -m MD5
[-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}]
[-w FILENAME] [--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: admin)
-p PASSWORD (p)assword for username
(default: password.)
-i ATD IP (i)p or hostname address of ATD
(default: atd.localhost.localdomain)
-n do (n)ot verify the SSL certificate for the communications
(default: False)
Search parameters:
-m MD5 (m)d5 32bit hash of the sample to search
(default: None)
Reporting parameters:
-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}
(t)ype of report requested
(default: None)
-w FILENAME (w)rite filename for saving the requested report
(default: None)
```
## robust-watchdog
A tool that watches a directory recursively for any new files to submit.
Example CLI
```
usage: robust-watchdog.py [-h] -u USER [-p PASSWORD] -i ATD IP [-n] -a
ANALYZER_PROFILE -d DIRECTORY [-e] [-j MAXTHREADS]
[--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: None)
-p PASSWORD (p)assword for username
(default: None)
-i ATD IP (i)p or hostname address of ATD
(default: None)
-n do (n)ot verify the SSL certificate for the communications
(default: False)
Watch parameters:
-f (f)ollow and watch the directory for new files to submit
(default: True)
-a ANALYZER_PROFILE (a)nalyzer profile id to be used during analysis
(default: None)
-d DIRECTORY (d)irectory to watch for events
(default: None)
-e (e)xisting files in directory will be submitted
(default: False)
-j MAXTHREADS (j) max number of threads
(default: 1)
```
Let it run in a shell and open another one or the file browser to create files in the /path/to/directory. Since the handler is printing the results, the output will reflect the flags chosen similar to `robust.py`:
The `-e` flag can be passed to cause all existing files in the directory (recurisively) to be submitted upon start.
```shell
(robust)$> robust-watchdog.py -a 26 -d ./ -n -e
.
...
.
.....
```
## robust-convict
`robust-convict` is a tool designed like `robust-watchdog` but its purpose is to help sort large directories of malware samples into directories, while downloading their corresponding reports.
Example Usage
```
robust-convict.py -n -a 26 -c ./tmp/clean/ -x ./tmp/dirty/ -r ./tmp/reports/ -z ./tmp/errors/ -d ./tmp/preprocess -j 10 -t zip -q
```
Options
```
usage: robust-convict.py [-h] [-u USER] [-p PASSWORD] [-i ATD IP] [-n] -a
ANALYZER_PROFILE -d DIRECTORY [-e] [-y SEVERITY]
[-c CLEANDIR] [-x DIRTYDIR] [-r REPORTDIR]
[-z ERRORDIR]
[-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}]
[-j MAXTHREADS] [--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Authentication parameters:
-u USER (u)sername for the API of the ATD
(default: robust)
-p PASSWORD (p)assword for username
(default: ****<.robust>*****)
-i ATD IP (i)p or hostname address of ATD
(default: atd.localhost.localdomain)
-n do (n)ot verify the SSL certificate for the communications
(default: True)
Watch parameters:
-f (f)ollow and watch the directory for new files to submit
(default: False)
-a ANALYZER_PROFILE (a)nalyzer profile id to be used during analysis
(default: None)
-d DIRECTORY (d)irectory to watch for events
(default: None)
-e (e)xisting files in directory will be submitted
(default: False)
-j MAXTHREADS (j) max number of threads
(default: 1)
Convict parameters:
-y SEVERITY (y) treat sample as dirty with this severity [0-5] or higher
(default: 3)
-c CLEANDIR (c) move clean files to this directory
(default: ~/robust/clean/)
-x DIRTYDIR (x) move processed dirty files to this directory
(default: ~/robust/malware/)
-r REPORTDIR (r) save reports to this directory
(default: ~/robust/reports/)
-z ERRORDIR (z) move error or skip files to this directory
(default: ~/robust/errors/)
-t {html,txt,xml,zip,json,ioc,stix,pdf,sample}
(t)ype of report requested
(default: None)
```
## robust-reporter
`robust-reporter` is a tool designed to quickly summarize the downloaded `*.json` files in your 'reports' directory.
Options
```
usage: robust-reporter.py [-h] [-r REPORTDIR] [--version] [-v | -q]
Robust McAfee ATD Python CLI tool
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbosity increase output (v)erbosity
(default: None)
-q, --quiet (q)uiet all output
(default: False)
Reporter parameters:
-r REPORTDIR (r) reports are processed or stored using this directory
(default: ~/robust/reports/)
```
Sample Run
```
$ robust-reporter.py
82344C9864B0F1D120C0D1AB7F7C54C3 (---) : Somewhat/probably is suspicious(Low-Medium) - 24sec
D012492123E4CF0CFB3A017A2E92C077 (Malware.Dynamic) : Malicious(High) - 194sec
DB273A97C54E3E23F411EA7C9B5A82DA (Malware.Dynamic) : Malicious (Medium) - 53sec
165A36C02B3FAAF4DE38F93A3DCB821B (---) : Somewhat/probably is suspicious(Low-Medium) - 36sec
D10195670651A40C46C22972CD839E89 (Artemis!D10195670651) : Malicious (Very High) - 32sec
8271093E0E78574428BBDDDA6F34A980 (Malware.Dynamic) : Malicious(High) - 192sec
86DAFA0262BF217F5344A3B057C0DB06 (Malware.Dynamic) : Malicious(High) - 193sec
8DA4CDC3E2EE16021F237EA7A043DA8E (Malware.Dynamic) : Malicious(High) - 191sec
```
## Tunning for Linux File Watchers
### iNotify Tuning Parameters
The inotify(7) subsystem has three important tunings that impact robust's directory watching.
```
/proc/sys/fs/inotify/max_user_instances impacts how many different root dirs you can watch.
/proc/sys/fs/inotify/max_user_watches impacts how many dirs you can watch across all watched roots.
/proc/sys/fs/inotify/max_queued_events impacts how likely it is that your system will experience a notification overflow.
```
You obviously need to ensure that `max_user_instances` and `max_user_watches` are set so that the system is capable of keeping track of your files.
`max_queued_events` is important to size correctly; if it is too small, the kernel will drop events and robust won't be able to report on them. Making this value bigger reduces the risk of this happening.
# Developers
## Install Development
```
$(robust)> pip install -r devel-requirements.txt
```
## Development Tasks
```shell
(robust)$> invoke -l
Available tasks:
build Build the setup.py
clean Clean up docs, bytecode, and extras
codestats Run flake8 PeP8 tests for code stats
release ``version`` should be a string like '0.4' or '1.0'.
smell Run flake8 PeP8 tests
test Run Unit tests
```
### Running the Test Suite
Nose is run via `invoke test`
```
Clearing rm -rf build
Clearing rm -rf dist
Clearing rm -rf *.egg-info
Clearing rm -rf pyclient.log
Clearing rm -rf **/**/*.pyc
Clearing rm -rf **/*.pyc
Clearing rm -rf ./*.pyc
...................
-----------------------------------------------------------------------------
19 tests run in 0.3 seconds (19 tests passed)
117 E501 line too long (97 > 79 characters)
7 F401 'ratd' imported but unused
1 F841 local variable 'rb_rtnv' is assigned to but never used
1 N802 function name should be lowercase
5 W601 .has_key() is deprecated, use 'in'
```
### Additional LICENSE information
A modified Fork of `atdcli.py` (Carlos Munoz - 2014).
https://pypi.python.org/pypi/atd
## VX Workshop Appliance Option
There is a fully operational Xubuntu 14.04 liveCD that includes:
* robust - https://github.com/shadowbq/robust-atd
* maltrieve - https://github.com/shadowbq/maltrieve
* vxcage - https://github.com/shadowbq/vxcage
It also includes
* hexeditors
* static analysis tools
* google chrome
* vmtools
* etc..
xubuntu-14.04.4-desktop-x86_64-VX-Workshop-0.4.iso (~ 1.2 GB)
Available to download with READMEs here: https://goo.gl/flcvew
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
robust-atd-0.5.2.tar.gz
(26.3 kB
view hashes)
Built Distribution
robust_atd-0.5.2-py2-none-any.whl
(32.0 kB
view hashes)
Close
Hashes for robust_atd-0.5.2-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8a732716f82413ade5ebf0d44efc238d43c8eb4461346478cde178ebef10658e |
|
MD5 | c471c9b53663d8c6c4670d8ee5333c22 |
|
BLAKE2b-256 | 272813974c3a496e1ec77a6aa3cb4134a4b5e6dc4d06f5e80bac15e161bb6d62 |