Datasette plugin that blocks all robots using robots.txt
Project description
datasette-block-robots
Datasette plugin that blocks all robots using robots.txt
Installation
Install this plugin in the same environment as Datasette.
$ pip install datasette-block-robots
Usage
Having installed the plugin, /robots.txt
on your Datasette instance will return the following:
User-agent: *
Disallow: /
Here's a demo of the plugin in action: https://sqlite-generate-demo.datasette.io/robots.txt
Development
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
cd datasette-block-robots
python3 -mvenv venv
source venv/bin/activate
Or if you are using pipenv
:
pipenv shell
Now install the dependencies and tests:
pip install -e '.[test]'
To run the tests:
pytest
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for datasette-block-robots-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ba58f2fe600568ab24cd789e078adb90e9d2682533cbd9ca7bdc70f281d953e5 |
|
MD5 | 148397c06fdb77d574235de8ac8a7b08 |
|
BLAKE2b-256 | d9525107fa8d77249ff91cbed8f2e858b492b68b10bc7ba01bb80785f311a321 |
Close
Hashes for datasette_block_robots-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d278bcccfedb0160a0b6b68b9d325cf1d58582f15202a6bafb85f8f0b1cdc0a |
|
MD5 | f907da7ce7f07235258cfe9da1a8c263 |
|
BLAKE2b-256 | 5f66f443fb9fb1904fcb31ff52d196ea05f3d3efb0cd7d97f32919c976bfa22d |