Django robots.txt generator
Project description
Fork of dimka665 https://github.com/dimka665/django-url-robots
Django robots.txt generator. Based on using decorated django.conf.urls.url. It gets urlpatterns and replaces ambiguous parts by *.
Installation & Usage
The recommended way to install django2-url-robots is with pip
Install from PyPI with easy_install or pip:
pip install django2-url-robots
Add 'django2_url_robots' to your INSTALLED_APPS:
INSTALLED_APPS = ( ... 'django2_url_robots', ... )
Add django2_url_robots view to your root URLconf:
urlpatterns += [ url(r'^robots\.txt$', django2_url_robots.views.robots_txt), ]
Describe rules by boolean keyword argument robots_allow using for it django2_url_robots.utils.url instead django.conf.urls.url:
from django2_url_robots.utils import url urlpatterns += [ url('^profile/private$', views.some_view, robots_allow=False), ]
django2-url-robots tested with Django-2.0.5+. Encodes unicode characters by percent-encoding.
Settings
In this moment there are only one option to define template of robots.txt file:
urlpatterns += [ url(r'^robots\.txt$', django2_url_robots.views.robots_txt, {'template': 'my_awesome_robots_template.txt'}), ]
Example
robots_template.txt:
User-agent: * Disallow: /* # disallow all {{ rules|safe }}
urls.py:
from django.conf.urls import include urlpatterns = [ url(r'^profile', include('django2_url_robots.tests.urls_profile')), ]
urls_profile.py:
from django2_url_robots.utils import url urlpatterns = [ url(r'^s$', views.some_view, name='profiles', robots_allow=True), url(r'^/(?P<nick>\w+)$', views.some_view), url(r'^/(?P<nick>\w+)/private', views.some_view, name='profile_private', robots_allow=False), url(r'^/(?P<nick>\w+)/public', views.some_view, name='profile_public', robots_allow=True), ]
Resulting robots.txt:
User-agent: * Disallow: /* # disallow all Allow: /profiles$ Disallow: /profile/*/private* Allow: /profile/*/public*
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file django2-url-robots-3.2.2.tar.gz
.
File metadata
- Download URL: django2-url-robots-3.2.2.tar.gz
- Upload date:
- Size: 4.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/51.0.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.7.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
fe8028f242325e3de6ec59c9982b0e04b294f81f145dbe17038be9e7346f0639
|
|
MD5 |
dd4a045c5ab31b83b0abd9290f7f5985
|
|
BLAKE2b-256 |
8dbd9f307f3ac66a6955041fedb2f8af399dad0be35e521cba8e14c03548c2f8
|
File details
Details for the file django2_url_robots-3.2.2-py3-none-any.whl
.
File metadata
- Download URL: django2_url_robots-3.2.2-py3-none-any.whl
- Upload date:
- Size: 6.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.25.0 setuptools/51.0.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.7.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
f93f3ca88ac9963dfab1d694ea89564f2e9abbf25e8453c61cb2942b722cb3b3
|
|
MD5 |
655da0d2bb2df980532929100b0f1e7a
|
|
BLAKE2b-256 |
ed2cef130863c305510646e560933aa9b3e48deabda6228437ddf4ed0f1de1a3
|