Django robots.txt generator
Project description
Fork of dimka665 https://github.com/dimka665/django-url-robots
Django robots.txt generator. Based on using decorated django.conf.urls.url. It gets urlpatterns and replaces ambiguous parts by *.
Installation & Usage
The recommended way to install django2-url-robots is with pip
Install from PyPI with easy_install or pip:
pip install django2-url-robots
Add 'django2_url_robots' to your INSTALLED_APPS:
INSTALLED_APPS = ( ... 'django2_url_robots', ... )
Add django2_url_robots view to your root URLconf:
urlpatterns += [ url(r'^robots\.txt$', django2_url_robots.views.robots_txt), ]
Describe rules by boolean keyword argument robots_allow using for it django2_url_robots.utils.url instead django.conf.urls.url:
from django2_url_robots.utils import url urlpatterns += [ url('^profile/private$', views.some_view, robots_allow=False), ]
django2-url-robots tested with Django-2.0.5+. Encodes unicode characters by percent-encoding.
Settings
In this moment there are only one option to define template of robots.txt file:
urlpatterns += [ url(r'^robots\.txt$', django2_url_robots.views.robots_txt, {'template': 'my_awesome_robots_template.txt'}), ]
Example
robots_template.txt:
User-agent: * Disallow: /* # disallow all {{ rules|safe }}
urls.py:
from django.conf.urls import include urlpatterns = [ url(r'^profile', include('django2_url_robots.tests.urls_profile')), ]
urls_profile.py:
from django2_url_robots.utils import url urlpatterns = [ url(r'^s$', views.some_view, name='profiles', robots_allow=True), url(r'^/(?P<nick>\w+)$', views.some_view), url(r'^/(?P<nick>\w+)/private', views.some_view, name='profile_private', robots_allow=False), url(r'^/(?P<nick>\w+)/public', views.some_view, name='profile_public', robots_allow=True), ]
Resulting robots.txt:
User-agent: * Disallow: /* # disallow all Allow: /profiles$ Disallow: /profile/*/private* Allow: /profile/*/public*
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for django2_url_robots-3.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7f8735e8324c89366c864a6c1d24d4bf25999cea41cebc0315a9ae639ca02067 |
|
MD5 | a96661c37ae0ab19686e4c61e708cb0c |
|
BLAKE2b-256 | 8beb363ce333810897a41a26a680140098881df047ff0f4907281678a9352716 |