thredds_security_test 0.1.0

THREDDS Data Server security configuration test utilities

=====================
THREDDS Security Test
=====================

Description
===========
This is a set of tools for testing the security configuration of THREDDS data servers. There are
three tools:

thredds_test_url_access::
- Determines whether access is available to a single resource specified by URL. An option is
provided to download the resource.

thredds_test_catalog_access::
- Determines whether access is available to the resources listed in a catalog specified by URL.
Optionally, it will recurse through catalog references. Numbers of resources to which access is
allowed and/or denied can be specified and a check is made that these match the results.

Each service listed in the catalog as applicable to a dataset is tried. For services that have
multiple extensions used to access different types of data or metadata, each extension is tried
(by default, this following extensions are tried for OPENDAP services: .html, .dds, .das, .asc,
.ascii and .dods).

If there is more than one service or service/extension combination available for accessing a
dataset, and access is found to be possible by some of these but not others, the security
configuration will be reported as inconsistent.

thredds_test_file_access::
- Determines whether access is available to a set of resources specified by URL listed in a file.
This is intended to be used to check that files that exist in the THREDDS content directories,
but which are not listed in catalogs, are not accessible.

Prerequisites
=============
This has been developed and tested for Python 2.6 compiled with HTTPS support. Earlier Python
versions do not handle HTTPS proxies.

Installation
============
Installation can be performed using easy_install, e.g.::
easy_install thredds_security_test-0.1.0-py2.6.egg

Running
=======
Each tool has a -h or --help option, which shows the usage and options. Common options are::

-h, --help Show help message and exit.
-k FILE, --private-key=FILE
Private key file - defaults to $HOME/.esg/credentials.pem
-c FILE, --certificate=FILE
Certificate file - defaults to $HOME/.esg/credentials.pem
-d, --debug Print debug information - this may be useful in solving problems with HTTP
or HTTPS access to a server.
-q, --quiet Produce minimal output.

Specific arguments and options::

thredds_test_url_access [options] url
url The URL of the resource the accessibility of which is to be tested
-f FILE, --fetch=FILE
Specifies a file into which to fetch the resource contents if accessible.

thredds_test_catalog_access [options] catalog_url
catalog_url The URL of the catalog to test
-l, --list-only Only list datasets in the catalog(s) without checking access.
-r, --recurse Recurse into referenced catalogs. References of the following types are
followed:
o Service with service type 'Catalog' or 'Resolver
o Catalog references contained within a catalog
-a NUMBER, --expect-allowed=NUMBER
Number of datasets for which access is expected to be allowed. After the
access checks are made, this number is compared with the number found; if
they are not the same an error message printed and the tool returns status
value 1.
-x NUMBER, --expect-denied=NUMBER
Number of datasets for which access is expected to be denied. After the
access checks are made, this number is compared with the number found; if
they are not the same an error message printed and the tool returns status
value 1.
-e SERVICE_EXTENSIONS, --service-extensions=SERVICE_EXTENSIONS
A list of service types and and for each a list of extensions to be tried.
The format is as in the following:
OPENDAP:.html,.dds;OtherSvc:.dat,.asc
Defaults to: OPENDAP:.html,.dds,.das,.asc,.ascii,.dods
-p PUBLIC_SERVICE_EXTENSIONS, --public-service-extensions=PUBLIC_SERVICE_EXTENSIONS
Service types and extensions for which public access
is expected, e.g., OPENDAP:.html,.dds
Defaults to: OPENDAP:.html
This option is used to prevent access to a dataset being classed as
inconsistent if, e.g., the .html resource is publicly accessible but other
resources are not accessible using the account used for testing.
-f FORBIDDEN_SERVICE_EXTENSIONS, --forbidden-service-extensions=FORBIDDEN_SERVICE_EXTENSIONS
Service types and extensions for which no access is
expected, e.g., OPENDAP:.asc
This is of use, e.g., if a security configuration allows access to the .asc
resources but not the equivalent .ascii resources.

thredds_test_file_access [options] list_file_path
list_file_path Path of a file containing a list of URLs, of which the accessibility of each
is to be tested
-a NUMBER, --expect-allowed=NUMBER
Number of datasets for which access is expected to be allowed. After the
access checks are made, this number is compared with the number found; if
they are not the same an error message printed and the tool returns status
value 1.
-x NUMBER, --expect-denied=NUMBER
Number of datasets for which access is expected to be denied. After the
access checks are made, this number is compared with the number found; if
they are not the same an error message printed and the tool returns status
value 1.

Return statuses::
thredds_test_url_access
0 - if the resource is accessible
1 -if the resource is not accessible

thredds_test_catalog_access
1 - if the expect-allowed and/or expect-denied options were specified and the result did not match
the option value.
2 - if the user interrupted test
0 - otherwise

thredds_test_file_access
1 - if the expect-allowed and/or expect-denied options were specified and the result did not match
the option value.
2 - if the user interrupted test
0 - otherwise

Scripted Testing
================
In addition to the command line tools described above, Python test scripts may also be written. The
tests should call any of the functions::
thredds_security_test.lib.catalog_access.check_catalog
thredds_security_test.lib.file_access.check_files
thredds_security_test.lib.httpget.check_url

See thredds_security_test.test.test_access.py for an example of usage.