ultimate-hosts-blacklist-whitelist 3.10.2

The whitelisting tool of the Ultimate Hosts Blacklist project.

This is the whitelisting tool provided by the Ultimate Hosts Blacklist project. It is mainly used to whitelist subject (domains, IP, URL) into our infrastructure but can also be used outside our infrastructure.

Indeed, this tool allows you:

• to get rid of our whitelist.

• to use your own whitelist.

• to use your own whitelist along with our whitelist.

• to get rid of one of the rule mentionned in our whitelist.

• to have a whitelist tool ready to use as a Python module.

Installation

$ pip3 install --user ultimate-hosts-blacklist-whitelist

Complementary whitelist

Our tool allow us to link one or more file(s) to the system which will be used in complementary of our whitelist list.

Special markers

If you already used a whitelist list you already know that we generaly only list all domains we want to whitelist one by one.

It’s also possible to do that with our whitelisting system but we can do more.

ALL

The ALL marker will tell the system to escape and regex check againt what follows.

INVALID characters

• $

  • As we automatically append $ to the end, you should not use this character.

• \\

  • As we automatically escape the given expression, you should not explicitly escape your regular expression when declaring an ALL marker.

REG

The REG marker will tell the system to explicitly check for the given regex which follows the marker.

RZD

The RZD marker will tell the system to explicitly check for the given string plus all possible TDL.

Anti whitelist

Don’t like one of our rule ? Our tool allows you to specify a file which contain a list of rule you don’t want to apply.

Simply use the --anti-whitelist flag to tell us one or more anti whitelist files and we will apply!

Understanding what we actually do

If we have the following secondary whitelist list:

facebook.com
ALL .gov
REG face
RZD example

our system will actually :

• Remove every line which match facebook.com and www.facebook.com

• Remove everyline which match example.*

• In complementary convert all lines with ALL ` or :code:`REG to the right format.

• Check every line again the regular expression.

• Print or save on screen the results.

The generated regular expression will be in this example:

(\.gov$)|(face)|(example(.*))

NOTE: The :code:`example` group is much longer as we construct the list of TDL based on the Root Zone Database of the IANA and the Public Suffix List project.

Which actually means that we whitelist:

• all elements/lines which ends with .gov

• all elements/lines which contain the word face

Contributors

• Daniel - @dnmTX

Usage of the tool

The sript can be called as uhb-whitelist, uhb_whitelist and ultimate-hosts-blacklist-whitelist.

usage: uhb_whitelist [-h] [-a ANTI_WHITELIST [ANTI_WHITELIST ...]] [-d]
                    [-f FILE] [-o OUTPUT] [-m] [-p PROCESSES] [-v]
                    [-w WHITELIST [WHITELIST ...]] [-wc]

The tool to clean a list or a hosts file with the Ultimate Hosts Blacklist
whitelist list or your own.

optional arguments:
    -h, --help            show this help message and exit
    -a ANTI_WHITELIST [ANTI_WHITELIST ...], --anti-whitelist ANTI_WHITELIST [ANTI_WHITELIST ...]
                            Read the given file and remove the rules (its data)
                            from the whitelist list we are going to use.
    -d, --debug           Activate the debug mode. This mode will write the
                            whole processes to stdout.
    -f FILE, --file FILE  Read the given file and remove all element to
                            whitelist.
    -o OUTPUT, --output OUTPUT
                            Save the result to the given filename or path.
    -m, --multiprocessing
                            Activate the usage of multiple processes.
    -p PROCESSES, --processes PROCESSES
                            The number of (maximal) processes to use.
    -v, --version         Show the version end exist.
    -w WHITELIST [WHITELIST ...], --whitelist WHITELIST [WHITELIST ...]
                            Read the given file and append its data to the our
                            whitelist list.
    -wc, --without-core   Disable the usage of the Ultimate Hosts Blacklist
                            whitelist list.

Crafted with ♥ by Nissar Chababy (Funilrys)