Robots.txt for the Sloth CI server
Project description
Robots.txt for the Sloth CI server.
`Robots.txt <https://en.wikipedia.org/wiki/Robots_exclusion_standard>`__ is a file you put on your server to protect certain URLs from being accessed by crawler bots.
By default,tThis extension serves *robots.txt* on the server root, e.g. http://example.com:8080/robots.txt. However, you can specify your own file to serve and the URL to serve it on.
Installation
------------
.. code-block:: bash
$ pip install sloth-ci.ext.robots_txt
Usage
-----
.. code-block:: yaml
:caption: sloth.yml
extensions:
robots_txt:
# Use the sloth_ci.ext.robots_txt module.
module: robots_txt
# Absolute path to the custom robots.txt file.
# If not given, the bundled one is used (disallows everything to everyone).
# file: ~/robots.txt
# URL path to robots.txt.
# By default the file is available in the root: *http://example.com:8080/robots.txt*.
# path: /static/robots.txt
`Robots.txt <https://en.wikipedia.org/wiki/Robots_exclusion_standard>`__ is a file you put on your server to protect certain URLs from being accessed by crawler bots.
By default,tThis extension serves *robots.txt* on the server root, e.g. http://example.com:8080/robots.txt. However, you can specify your own file to serve and the URL to serve it on.
Installation
------------
.. code-block:: bash
$ pip install sloth-ci.ext.robots_txt
Usage
-----
.. code-block:: yaml
:caption: sloth.yml
extensions:
robots_txt:
# Use the sloth_ci.ext.robots_txt module.
module: robots_txt
# Absolute path to the custom robots.txt file.
# If not given, the bundled one is used (disallows everything to everyone).
# file: ~/robots.txt
# URL path to robots.txt.
# By default the file is available in the root: *http://example.com:8080/robots.txt*.
# path: /static/robots.txt
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Close
Hashes for sloth-ci.ext.robots_txt-1.0.3.zip
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2bb199b762fafc428eec2066d54891d908a30d9d534436c42e98d749b56660a1 |
|
MD5 | e346ef61f05ac59e4e8a27e3675fdfd6 |
|
BLAKE2b-256 | 1932f91eee2d7069c4dead1dd6255bcf07e3f4042cdbe89cfe4994d85014f721 |
Close
Hashes for sloth_ci.ext.robots_txt-1.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 91bf48ab81791eb558d031f0a59e2f0c7cb6ff109de66f985c6736262274138a |
|
MD5 | 5c630cd75c39543559aa479450eea375 |
|
BLAKE2b-256 | 2de00e51185d5b554edbe7dc6cda6c906fbe2ad4a29b7dea7aefb444a92523c3 |
Close
Hashes for sloth-ci.ext.robots_txt-1.0.3.win-amd64.exe
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8a6fb6b4621650e9f34810fde9aea222c59bacca864ae108cfa81eedc588f77 |
|
MD5 | 142df3de2060132e07024d0cefe20457 |
|
BLAKE2b-256 | 9846322ec8f94b4ceb4e2086a16533bdfe4a5a53e4d82a58334e67f1a95d346c |