sloth-ci.ext.robots_txt 1.0.3

Robots.txt for the Sloth CI server

Robots.txt for the Sloth CI server.

`Robots.txt <https://en.wikipedia.org/wiki/Robots_exclusion_standard>`__ is a file you put on your server to protect certain URLs from being accessed by crawler bots.

By default,tThis extension serves *robots.txt* on the server root, e.g. http://example.com:8080/robots.txt. However, you can specify your own file to serve and the URL to serve it on.


Installation
------------

.. code-block:: bash

$ pip install sloth-ci.ext.robots_txt


Usage
-----

.. code-block:: yaml
:caption: sloth.yml

extensions:
robots_txt:
# Use the sloth_ci.ext.robots_txt module.
module: robots_txt

# Absolute path to the custom robots.txt file.
# If not given, the bundled one is used (disallows everything to everyone).
# file: ~/robots.txt

# URL path to robots.txt.
# By default the file is available in the root: *http://example.com:8080/robots.txt*.
# path: /static/robots.txt