ACT workers
Project description
ACT Workers
Introduction
This repository contains workers for the ACT platform.
The source code the workers are available on github.
Setup
To use the workers, install from PyPi:
sudo pip3 install act-workers
This will install scripts for all workers:
- act-argus-case
- act-attack
- act-country-regions
- act-ip-filter
- act-misp-feeds
- act-mnemonic-pdns
- act-origin
- act-scio
- act-search-graph
- act-shadowserver-asn
- act-uploader
- act-url-shorter-unpack
- act-veris
- act-vt
Origins
All workers support the optional arguments --origin-name and --origin-id. If they are not specified, facts will be added with the origin of the user performing the upload to the platform.
For managing origins, use the tool act-origin
which can list, add and delete origins.
Add origin
$ act-origin --act-baseurl <BASEURL> --user-id <USERID> --add
Origin name: myorigin
Origin description: My Test Origin
Origin trust (float 0.0-1.0. Default=0.8):
Origin organization (UUID):
[2019-11-11 10:46:22] app=origin-client level=INFO msg=Created origin: myorigin
Origin added:
Origin(name='myorigin', id='e5a9792e-78c7-4190-9275-27616be47ca8', organization=Organization(), description='My Test Origin', trust=0.8)
List origins
$ act-origin --act-baseurl <BASEURL> --user-id <USERID> --list
Origin(name='Test origin', id='5da8b157-5129-4f2f-9b90-6d624d62eebe', organization=Organization(), description='My test origin', trust=0.5)
Delete origin
$ act-origin --act-baseurl <BASEURL> --user-id <USERID> --delete --origin-id e5a9792e-78c7-4190-9275-27616be47ca8
Origin deleted: e5a9792e-78c7-4190-9275-27616be47ca8
Access mode
By default, all facts will have access-mode "RoleBased", which means that the user needs access to the organization specified when creating the facts.
The access mode can be explicit set with --access-mode
, e.g. like this, to set all facts to Public access mode:
--access-mode Public
Organization
All workers support the optional arguments --organization
If they are not specified, facts will be added with the organization of the origin or the user performing the upload to the platform (if not set by the origin.
Worker usage
To print facts to stdout:
$ act-country-regions
{"type": "memberOf", "value": "", "accessMode": "Public", "sourceObject": {"type": "country", "value": "Afghanistan"}, "destinationObject": {"type": "subRegion", "value": "Southern Asia"}, "bidirectionalBinding": false}
{"type": "memberOf", "value": "", "accessMode": "Public", "sourceObject": {"type": "subRegion", "value": "Southern Asia"}, "destinationObject": {"type": "region", "value": "Asia"}, "bidirectionalBinding": false}
(...)
Or print facts as text representation:
$ act-country-regions --output-format str
(country/Afghanistan) -[memberOf]-> (subRegion/Southern Asia)
(subRegion/Southern Asia) -[memberOf]-> (region/Asia)
(...)
To add facts directly to the platform, include the act-baseurl and user-id options:
$ act-country-regions --act-baseurl http://localhost:8888 --user-id 1
Configuration
All workers support options specified as command line arguments, environment variables and in a configuration file.
A utility to show and start with a default ini file is also included:
act-worker-config --help
usage: ACT worker config [-h] {show,user,system}
positional arguments:
{show,user,system}
optional arguments:
-h, --help show this help message and exit
show - Print default config
user - Copy default config to /home/fredrikb/.config/actworkers/actworkers.ini
system - Copy default config to /etc/actworkers.ini
You can see the default options in act/workers/etc/actworkers.ini.
The configuration presedence are (from lowest to highest):
- Defaults (shown in --help for each worker)
- INI file
- Environment variable
- Command line argument
INI-file
Arguments are parsed in two phases. First, it will look for the argument --config argument which can be used to specify an alternative location for the ini file. If not --config argument is given it will look for an ini file in the following locations:
/etc/<CONFIG_FILE_NAME>
~/.config/<CONFIG_ID>/<CONFIG_FILE_NAME> (or directory specified by $XDG_CONFIG_HOME)
The ini file contains a "[DEFAULT]" section that will be used for all workers. In addition there are separate sections for each worker which you can use to configure worker-specific options, and override default options.
Environment variables
The configuration step will also look for environment variables in uppercase and with "-" replaced with "_". For the example for the option "cert-file" it will look for the enviornment variable "$CERT_FILE".
Requirements
All workers requires python version >= 3.5 and the act-api library:
In addition some of the libraries might have additional requirements. See requirements.txt for a full list of all requirements.
Proxy configuration
Workers will honor the proxy-string
option on the command line when connecting to external APIs. However, if you need to
use the proxy to connect to the ACT platform (--act-baseurl), you will need to add the "--proxy-platform" switch:
echo -n www.mnemonic.no | act-vt --proxy-string <PROXY> --user-id <USER-ID> --act-baseurl <ACT-HOST> --proxy-platform
Local development
Use pip to install in local development mode. act-workers (and act-api) uses namespacing, so it is not compatible with using setup.py install
or setup.py develop
.
In repository, run:
pip3 install --user -e .
search
A worker to run graph queries is also included. A sample search config is inscluded in etc/searc_jobs.ini
:
act-search-graph etc/search_jobs.ini
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.