A tool for downloading the Greenbone Community Feed
pip install greenbone-feed-sync
greenbone-feed-sync
Tool for downloading the Greenbone Community Feed.
greenbone-feed-sync assumes you are using the latest feed release and up to
date components of the Greenbone Community Edition by default. It is highly
configurable and can be adjusted easily for downloading different feed releases
and types.
greenbone-feed-sync requires the rsync tool being installed and available
within the PATH.
On Debian based Distributions like Ubuntu and Kali rsync can be installed via
sudoaptinstallrsync
Install using pipx
You can install the latest release of greenbone-feed-sync from the Python
Package Index (pypi) using pipx
python3-mpipxinstallgreenbone-feed-sync
On Debian based Distributions like Ubuntu and Kali pipx itself can be
installed via
sudoaptinstallpipx
Install using pip
NOTE: The pip install command does no longer work out-of-the-box in newer
distributions like Ubuntu 23.04 or Debian 12 because of PEP 668.
Please use the installation via pipx instead.
You can install the latest release of greenbone-feed-sync from the
Python Package Index (pypi) using pip
python3-mpipinstallgreenbone-feed-sync
Usage
Most of the time you should just run the script without any arguments to
download the new data for all necessary feed types
To get verbose progress output during the data download you might increase the
verbosity
sudogreenbone-feed-sync-vvv
If the script is run in a cron job the output can be turned off via
sudogreenbone-feed-sync--quiet
To download only a specific feed content the --type argument can be used
sudogreenbone-feed-sync--typenvt
Run --help to get information about all possible types and additional argument
options
greenbone-feed-sync--help
Usage on Kali Linux
When running greenbone-feed-sync as root user, for example via sudo, the
actual user and group of the process are changed to the gvm user and group via
seteuid. This is done to
ensure that gvmd and
openvas-scanner can read the
downloaded file contents.
When using the Greenbone Community Edition installed via packages on Kali Linux
a different user and group are used. They are both named _gvm instead.
Therefore the group and user settings need to be adjusted.
This can be done by using a config file.
sudomkdir/etc/gvm
sudochmod+r/etc/gvm
cat<<EOF | sudo tee /etc/gvm/greenbone-feed-sync.toml[greenbone-feed-sync]user="_gvm"group="_gvm"EOF
sudochmod+r/etc/gvm/greenbone-feed-sync.toml
Command Completion
greenbone-feed-sync comes with support for command line completion in bash
and zsh.
The greenbone-feed-sync script is adjustable for all kind of purposes and very
flexible which content gets downloaded. Most likely you will never need to
adjust the settings because the defaults will suffice. Changing the settings
is only required for experts and testing purposes.
verbose
Name
Value
CLI Argument
--verbose, -v
Config Variable
verbose
Environment Variable
GREENBONE_FEED_SYNC_VERBOSE
Default Value
2
Description
Log verbosity. -vvv for maximum verbosity.
quiet
Name
Value
CLI Argument
--quiet
Config Variable
Environment Variable
Default Value
Description
Disable all log output. Same as setting verbose or GREENBONE_FEED_SYNC_VERBOSE to 0
config
Name
Value
CLI Argument
--config, -c
Config Variable
Environment Variable
Default Value
~/.config/greenbone-feed-sync.toml and /etc/gvm/greenbone-feed-sync.toml
Description
TOML config file to load settings from.
private-directory
Name
Value
CLI Argument
--private-directory
Config Variable
private-directory
Environment Variable
GREENBONE_FEED_SYNC_PRIVATE_DIRECTORY
Default Value
Description
(Sub-)Directory to exclude from the sync which will never get deleted automatically.
compression-level
Name
Value
CLI Argument
--compression-level
Config Variable
compression-level
Environment Variable
GREENBONE_FEED_SYNC_COMPRESSION_LEVEL
Default Value
9
Description
rsync compression level 0-9. (0 - no compression, 9 - high compression)
type
Name
Value
CLI Argument
--type
Config Variable
Environment Variable
Default Value
all
Description
Specifies which feed data should be downloaded. Possible values are all, all-enterprise, nvt/nvts, gvmd-data, scap, cert, notus, nasl, skiron, agent, report-format/report-formats, scan-config/scan-configs, port-list/port-lists or vt-tech-info.
feed-url
Name
Value
CLI Argument
Config Variable
feed-url
Environment Variable
GREENBONE_FEED_SYNC_URL
Default Value
rsync://feed.community.greenbone.net/community
Description
URL to download the feed data from. Other URLs will be relative to this URL by default. For example using rsync://example.com as feed url the notus url will be rsync://example.com/vulnerability-feed/$FEED_VERSION/vt-data/notus/.
feed-release
Name
CLI Argument
--feed-release
Config Variable
feed-release
Environment Variable
GREENBONE_FEED_SYNC_FEED_RELEASE
Default Value
25.0
Description
Release series of the feed to be downloaded. Download destinations and URLs will use this variable.
destination-prefix
Name
Value
CLI Argument
--destination-prefix
Config Variable
destination-prefix
Environment Variable
GREENBONE_FEED_SYNC_DESTINATION_PREFIX
Default Value
/var/lib/
Description
Directory prefix to use for default feed data download destinations. Other download destinations will be relative to this path by default. For example using /opt/lib as destination prefix will change the default of the notus destination to /opt/lib/notus.
File to use for locking the feed synchronization for data loaded by the gvmd daemon. Used to avoid that more then one process accesses the feed data at the same time.
openvas-lock-file
Name
Value
CLI Argument
--openvas-lock-file
Config Variable
openvas-lock-file
Environment Variable
GREENBONE_FEED_SYNC_OPENVAS_LOCK_FILE
Default Value
$DESTINATION_PREFIX/openvas/feed-update.lock
Description
File to use for locking the feed synchronization for data loaded by the openvas scanner. Used to avoid that more then one process accesses the feed data at the same time.
fail-fast
Name
Value
CLI Argument
--fail-fast, --failfast
Config Variable
fail-fast
Environment Variable
GREENBONE_FEED_SYNC_FAIL_FAST
Default Value
false
Description
Stop after a first error has occurred. Otherwise the script tries to download additional data if specified.
no-wait
Name
Value
CLI Argument
--no-wait
Config Variable
no-wait
Environment Variable
GREENBONE_FEED_SYNC_NO_WAIT
Default Value
false
Description
Fail directly if the lock file can't be acquired.
wait-interval
Name
Value
CLI Argument
--wait-interval
Config Variable
wait-interval
Environment Variable
GREENBONE_FEED_SYNC_LOCK_WAIT_INTERVAL
Default Value
5
Description
Time to wait in seconds after failed lock attempt before re-trying to lock the file.
rsync-timeout
Name
Value
CLI Argument
--rsync-timeout
Config Variable
rsync-timeout
Environment Variable
GREENBONE_FEED_SYNC_RSYNC_TIMEOUT
Default Value
Description
Maximum I/O timeout in seconds used for rsync. If no data is transferred for the specified time then rsync will exit. By default no timeout is set and the rsync default will be used.
no-permission-change
Name
Value
CLI Argument
--no-permission-change
Config Variable
no-permission-change
Environment Variable
GREENBONE_FEED_SYNC_NO_PERMISSION_CHANGE
Default Value
false
Description
Do not preserve permissions and skip normalizing the feed tree via rsync's --perms and --chmod options. By default permissions are preserved. Enable this to pass --no-perms to rsync, for storage that allows writing files but rejects changing their modes, for example some bind mounts, network filesystems or container volumes where rsync would otherwise fail with a permission error.
group
Name
Value
CLI Argument
--group
Config Variable
group
Environment Variable
GREENBONE_FEED_SYNC_GROUP
Default Value
gvm
Description
If the greenbone-feed-sync script is run as root, the effective group is changed to this group name or ID.
user
Name
Value
CLI Argument
--user
Config Variable
user
Environment Variable
GREENBONE_FEED_SYNC_USER
Default Value
gvm
Description
If the greenbone-feed-sync script is run as root, the effective user is changed to this user name or ID.
greenbone-enterprise-feed-key
Name
Value
CLI Argument
--greenbone-enterprise-feed-key
Config Variable
greenbone-enterprise-feed-key
Environment Variable
GREENBONE_FEED_SYNC_ENTERPRISE_FEED_KEY
Default Value
/etc/gvm/greenbone-enterprise-feed-key
Description
File to read the Greenbone Enterprise Feed key from. The key gives access to additional vulnerability tests for enterprise software among other advantages. See Greenbone Enterprise Feed and Greenbone Community Feed in Comparison for more details. The default URLs are adjusted according to the data in the key. If the key file does not exist it is ignored.
Config
It is possible to use a config file for loading the settings of the
greenbone-feed-sync script. The config file uses the TOML format. Without
explicitly passing a config file, greenbone-feed-sync tries to load
~/.config/greenbone-feed-sync.toml and if that file doesn't exist afterwards
/etc/gvm/greenbone-feed-sync.toml.
Your contributions are highly appreciated. Please
create a pull request
on GitHub. Bigger changes need to be discussed with the development team via the
issues section at GitHub
first.