Skip to main content

HTTP Observatory: a set of tests and tools to scan your website for basic web hygeine.

Project description

Mozilla HTTP Observatory - Build Status Requirements Status

The Mozilla HTTP Observatory is a set of tools to analyze your website and inform you if you are utilizing the many available methods to secure it.

It is split into three projects:

Scanning sites with the HTTP Observatory

Sites can be scanned using:

Contributing

Prerequisites

  • Python 3
  • Git
  • pip3

Notes

These instructions assume that you have a working Python3 development environment with pip3 installed and capable of building requirements, which may require installing an additional python OS package (-dev, -devel).

If this is not appropriate for your environment, you may install the appropriate requirements using your OS package manager (or other means) and skip the pip3 -r requirements command.

Running a scan from the local codebase, without DB, for continuous integration

# Install the HTTP Observatory
$ git clone https://github.com/mozilla/http-observatory.git
$ cd http-observatory
$ pip3 install --upgrade .
$ pip3 install --upgrade -r requirements.txt

Using the local scanner function calls

>>> from httpobs.scanner.local import scan
>>> scan('observatory.mozilla.org')  # a scan with default options
>>> scan('observatory.mozilla.org',  # all the custom options
         http_port=8080,             # http server runs on port 8080
         https_port=8443,            # https server runs on port 8443
         path='/foo/bar',            # don't scan /, instead scan /foo/bar
         cookies={'foo': 'bar'},     # set the "foo" cookie to "bar"
         headers={'X-Foo': 'bar'},   # send an X-Foo: bar HTTP header
         verify=False)               # treat self-signed certs as valid for tests like HSTS/HPKP

The same, but with the local CLI

$ httpobs-local-scan --http-port 8080 --https-port 8443 --path '/foo/bar' \
    --cookies '{"foo": "bar"}' --headers '{"X-Foo": "bar"}' --no-verify mozilla.org

Running a local scanner with Docker

# Install the HTTP Observatory client and requests library
$ git clone https://github.com/mozilla/http-observatory.git
$ cd http-observatory
$ pip3 install .
$ pip3 install --upgrade requests

# Create docker machine
$ docker-machine create --driver virtualbox --virtualbox-disk-size "40000" http-observatory

# Save the URL to the API in your .profile, .bash_profile, or whatever
$ echo export HTTPOBS_API_URL=http://$(docker-machine ip http-observatory):57001/api/v1 >> ~/.profile
$ . ~/.profile

# Start up the docker instance and install all the pieces
$ eval $(docker-machine env http-observatory)
$ docker-compose up -d

Creating a local installation (tested on Ubuntu 15)

# Install git, postgresql, and redis
# sudo -s
# apt-get install -y git libpq-dev postgresql redis-server

# Clone the repo
# cd /opt
# git clone https://github.com/mozilla/http-observatory.git
# cd http-observatory

# Install the observatory and scanner
# pip install .
# pip3 install -r requirements.txt

# Install the database
# su - postgres
$ createdb http_observatory
$ psql http_observatory < httpobs/database/schema.sql
$ psql http_observatory
http_observatory=# \password httpobsapi
http_observatory=# \password httpobsscanner
# vi /etc/postgresql/9.4/main/postgresql.conf (set max_connections = 512, shared_buffers = 256MB)
# service postgresql restart

# Create the httpobs user, and log/pid directories
# useradd -m httpobs
# install -m 750 -o httpobs -g httpobs -d /var/run/httpobs /var/log/httpobs

# Update the environmental variables
# su - httpobs
$ echo export HTTPOBS_API_URL="http://localhost:57001/api/v1" >> ~/.profile

# Start the scanner
$ cd /opt/http-observatory
$ HTTPOBS_DATABASE_USER="httpobsscanner" HTTPOBS_DATABASE_PASS="....." \
    /opt/http-observatory/httpobs/scripts/httpobs-scan-worker

# Start the API (in another terminal)
# HTTPOBS_DATABASE_USER="httpobsapi" HTTPOBS_DATABASE_PASS="....." \
    uwsgi --http :57001 --wsgi-file httpobs/website/main.py --processes 8 --callable app --master

Authors

  • April King

License

  • Mozilla Public License Version 2.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for httpobs-alt, version 0.9.2.post4
Filename, size File type Python version Upload date Hashes
Filename, size httpobs_alt-0.9.2.post4-py3-none-any.whl (574.2 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size httpobs-alt-0.9.2.post4.tar.gz (544.3 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page