django-tools 0.30.1

miscellaneous tools for django

Miscellaneous tools for django.

	travis-ci.org/jedie/django-tools
	coveralls.io/r/jedie/django-tools
	landscape.io/github/jedie/django-tools/master

existing stuff

Filemanager library

Library for building django application like filemanager, gallery etc.

more info, read ./filemanager/README.creole

per-site cache middleware

Similar to django UpdateCacheMiddleware and FetchFromCacheMiddleware, but has some enhancements: ‘per site cache’ in ./cache/README.creole

smooth cache backends

Same as django cache backends, but adds cache.smooth_update() to clears the cache smoothly depend on the current system load. more info in: ‘smooth cache backends’ in ./cache/README.creole

local sync cache

Keep a local dict in a multi-threaded environment up-to-date. Usefull for cache dicts. More info, read DocString in ./local_sync_cache/local_sync_cache.py.

threadlocals middleware

For getting request object anywhere, use ./middlewares/ThreadLocal.py

Dynamic SITE_ID middleware

Set settings.SITE_ID dynamically with a middleware base on the current request domain name. Domain name alias can be specify as a simple string or as a regular expression.

more info, read ./dynamic_site/README.creole.

StackInfoStorage

Message storage like LegacyFallbackStorage, except, every message would have a stack info, witch is helpful, for debugging. Stack info would only be added, if settings DEBUG or MESSAGE_DEBUG is on. To use it, put this into your settings:

MESSAGE_STORAGE = "django_tools.utils.messages.StackInfoStorage"

More info, read DocString in ./utils/messages.py.

limit to usergroups

Limit something with only one field, by selecting:

• anonymous users

• staff users

• superusers

• ..all existing user groups..

More info, read DocString in ./limit_to_usergroups.py

form/model fields

• Directory field - check if exist and if in a defined base path

• language code field with validator

• Media Path field browse existign path to select and validate input

• sign seperated form/model field e.g. comma seperated field

• static path field

• url field A flexible version of the original django form URLField

unittests helpers

BaseUnittestCase

django_tools.unittest_utils.unittest_base.BaseUnittestCase contains some low-level assert methods:

• assertEqual_dedent()

• assert_is_dir(), assert_not_is_dir()

• assert_is_file(), assert_not_is_File()

django_tools.unittest_utils.tempdir contains TempDir, a Context Manager Class:

with TempDir(prefix="foo_") as tempfolder:
    # create a file:
    open(os.path.join(tempfolder, "bar"), "w").close()

# the created temp folder was deleted with shutil.rmtree()

usage/tests:

• /tests/test_unittest_utils.py

DOM compare in unittests

The Problem: You can’t easy check if e.g. some form input fields are in the response, because the form rendering use a dict for storing all html attributes. So, the ordering of form field attributes are not sorted and varied.

The Solution: You need to parse the response content into a DOM tree and compare nodes.

We add the great work of Gregor Müllegger at his GSoC 2011 form-rendering branch. You will have the following assert methods inherit from: django_tools.unittest_utils.unittest_base.BaseTestCase

• self.assertHTMLEqual() – for compare two HTML DOM trees

• self.assertDOM() – for check if nodes in response or not.

• self.assertContains() – Check if ond node occurs ‘count’ times in response

More info and examples in ./tests/test_dom_asserts.py

Speedup tests

Speedup test run start by disable migrations, e.g.:

from django_tools.unittest_utils.disable_migrations import DisableMigrations
MIGRATION_MODULES = DisableMigrations()

small tools

debug_csrf_failure()

Display the normal debug page and not the minimal csrf debug page. More info in DocString here: django_tools/views/csrf.py

import lib helper

additional helper to the existing importlib more info in the sourcecode: ./utils/importlib.py

http utils

Pimped HttpRequest to get some more information about a request. More info in DocString here: django_tools/utils/http.py

upgrade virtualenv

A simple commandline script that calls pip install —-upgrade XY for every package thats installed in a virtualenv. Simply copy/symlink it into the root directory of your virtualenv and start it.

Note: Seems that this solution can’t observe editables right.

To use it, without installing django-tools:

~/$ cd goto/your_env
.../your_env/$ wget https://github.com/jedie/django-tools/raw/master/django_tools/upgrade_virtualenv.py
.../your_env/$ chmod +x upgrade_virtualenv.py
.../your_env/$ ./upgrade_virtualenv.py

This script will be obsolete, if pip has a own upgrade command.

SignedCookieStorage

Store information in signed Cookies, use django.core.signing. So the cookie data can’t be manipulated from the client. Sources/examples:

• /django_tools/utils/client_storage.py

• /tests/test_signed_cookie.py

Print SQL Queries

Print the used SQL queries via context manager.

usage e.g.:

from django_tools.unittest_utils.print_sql import PrintQueries

# e.g. use in unittests:
class MyTests(TestCase):
    def test_foobar(self):
        with PrintQueries("Create object"):
            FooBar.objects.create("name"=foo)

# e.g. use in views:
def my_view(request):
    with PrintQueries("Create object"):
        FooBar.objects.create("name"=foo)

the output is like:

_______________________________________________________________________________
 *** Create object ***
1 - INSERT INTO "foobar" ("name")
    VALUES (foo)
-------------------------------------------------------------------------------

SetRequestDebugMiddleware

middleware to add debug bool attribute to request object. More info: ./debug/README.creole

InternalIps() - Unix shell-style wildcards in INTERNAL_IPS

settings.py e.g.:

from django_tools.settings_utils import InternalIps

INTERNAL_IPS = InternalIps(["127.0.0.1", "::1", "192.168.*.*", "10.0.*.*"])

StdoutStderrBuffer()

redirect stdout + stderr to a string buffer. e.g.:

from django_tools.unittest_utils.stdout_redirect import StdoutStderrBuffer

with StdoutStderrBuffer() as buffer:
    print("foo")
output = buffer.get_output() # contains "foo\n"

..all others…

There exist many miscellaneous stuff. Look in the source, luke!

running django-tools unittests

There exist different ways to run unittests, e.g.:

.../django-tools $ ./setup.py test
.../django-tools $ ./manage.py test
.../django-tools $ ./runtests.sh

Run specific tests, e.g.:

.../django-tools $ ./setup.py test tests.test_local_sync_cache
.../django-tools $ ./manage.py test tests.test_local_sync_cache
.../django-tools $ ./runtests.sh tests.test_local_sync_cache

Backwards-incompatible changes

v0.29

ClientCookieStorage was renamed to SignedCookieStorage import e.g.:

from django_tools.utils.client_storage import SignedCookieStorage

v0.25.0

SmoothCacheBackends API changed: The cache.clear() method will really clear the cache, as the origin backend API. You must call cache.smooth_update() to set the “last change” timestamp.

v0.24.10

AutoUpdateFileBasedCache is deprecated, use new SmoothCacheBackends.

v0.9

Language code field and SelectMediaPath are renamed.

change: from django_tools.fields import LanguageCodeFormField to: from django_tools.fields.language_code import LanguageCodeFormField

change and rename: from django_tools.fields import LanguageCodeField to: from django_tools.fields.language_code import LanguageCodeModelField

change and rename: from django_tools.widgets import SelectMediaPath to: from django_tools.fields.media_path import MediaPathWidget

Django compatibility

django-tools	django version	python
>= v0.30.1	1.8, 1.9, 1.10	2.7, 3.4, 3.5
v0.30	1.8, 1.9	2.7, 3.4
v0.29	1.6 - 1.8	2.7, 3.4
v0.26	<=1.6
v0.25	<=1.4

(since v0.29 the given version combination will be tested via travis-ci)

history

• v0.30.1 - 26.08.2016 - compare v0.30.0…v0.30.1

  • add: django_tools.unittest_utils.disable_migrations.DisableMigrations (see above)

  • run tests also with django v1.10 and Python 3.5

  • use tox

• v0.30.0 - 27.04.2016 - compare v0.29.5…v0.30.0

  • Django 1.9 and Python 3 support contributed by naegelyd

• v0.29.4 and v0.29.5 - 10.08.2015 - compare v0.29.3…v0.29.5

  • Some bugfixes for django 1.6 support

• v0.29.3 - 10.08.2015 - compare v0.29.2…v0.29.3

  • Clear ThreadLocal request atttribute after response is processed (contributed by Lucas Wiman)

• v0.29.2 - 19.06.2015 - compare v0.29.1…v0.29.2

  • Bugfix in unittest_utils.selenium_utils.selenium2fakes_response

  • assertResponse used assertContains from django

  • Add QueryLogMiddleware (TODO: add tests)

• v0.29.1 - 17.06.2015 - compare v0.29.0…v0.29.1

  • Bugfixes for Py2 and Py3

  • add StdoutStderrBuffer()

• v0.29.0 - 09.06.2015 - compare v0.26.0…v0.29.0

  • WIP: Refactor unittests (DocTests must be updated for Py3 and more unittests must be written to cover all)

  • catch more directory traversal attacks in BaseFilesystemBrowser (and merge code parts)

  • Bugfix for “django.core.exceptions.AppRegistryNotReady: Models aren’t loaded yet.” if using UpdateInfoBaseModel

  • Bugfixes in dynamic_site for django 1.7

  • add: django_tools.settings_utils.InternalIps

• v0.28.0 - 12.02.2015 - compare v0.26.0…v0.28.0

  • Work-a-round for import loops

  • (new Version number, because of PyPi stress)

• v0.26.0 - 11.02.2015 - compare v0.25.1…v0.26.0

  • Updates for Django 1.6 and Python 3

• v0.25.1 - 18.11.2013

  • Bugfix: Fall back to “UTF-8” if server send no encoding info

• v0.25.0 - 28.08.2012

  • Rename cache.clear() in SmoothCacheBackends to cache.smooth_update(), so that reset timestamp is independ from clear the cache.

• v0.24.10 - 24.08.2012

  • Add SmoothCacheBackends: ./cache/README.creole

• v0.24.9 - 24.08.2012

  • Bugfix in per-site cache middleware: set inital count values to None, if counting is disabled.

• v0.24.8 - 24.08.2012

  • Enhanced per-site cache middleware: ./cache/README.creole

  • Add SetRequestDebugMiddleware: ./debug/README.creole

• v0.24.7 - 21.08.2012

  • Add the per-site cache middleware (see above)

  • Add import lib helper: ./utils/importlib.py

• v0.24.6 - 21.08.2012

  • Add the filemanager library (see above)

• v0.24.5 - 06.08.2012

  • Add Print SQL Queries context manager. (see above)

• v0.24.4 - 26.07.2012

  • remove date from version string, cause of side-effects e.g.: user clone the repo and has the filter not installed

• v0.24.3 - 25.07.2012

  • “Hardcode” the version string date attachment via gitattribute filter script to fix a reported issues with pip requirements file bug.

• v0.24.2 - 10.07.2012

  • Split UpdateInfoBaseModel(): So you can only set “createtime”, “lastupdatetime” or “createby”, “lastupdateby” or both types (This is backwards compatible)

• v0.24.1 - 12.06.2012

  • Bugfix: UsergroupsModelField() and add unittests for it

  • Add “normal users” in UsergroupsModelField()

  • New: Add create_user() and create_testusers() to BaseTestCase

  • Add a test project for the unittests. TODO: use this for all tests

• v0.24.0 - 04.06.2012

  • Don’t use auto_now_add and auto_now in UpdateInfoBaseModel

  • Bugfix in UsergroupsModelField

• v0.23.1

  • Dynamic Site would be only initialised if settings.USE_DYNAMIC_SITE_MIDDLEWARE = True

• v0.23.0

  • Use cryptographic signing tools from django 1.4 in django_tools.utils.client_storage

• v0.22.0

  • Add static_path.py thats used settings.STATIC_ROOT.

  • The old media_path.py which used settings.MEDIA_ROOT is deprecated and will be removed in the future.

  • auto_add_check_unique_together() can use settings.DATABASES[“default”][“ENGINE”], too.

• v0.21.1

  • Bugfixes in Dynamic Site.

• v0.21.0beta

  • New: site alias function

  • refractory ‘DynamicSiteMiddleware’ to a own app (Backwards-incompatible change: change your settings if you use the old DynamicSiteMiddleware.)

• v0.20.1

  • New: debug_csrf_failure() to display the normal debug page and not the minimal csrf debug page.

• v0.20.0

  • Add experimental DynamicSiteMiddleware, please test it and give feedback.

• v0.19.6

  • Add some south introspection rules for LanguageCodeModelField and jQueryTagModelField

  • fallback if message for anonymous user can’t created, because django.contrib.messages middleware not used.

  • Bugfix in django_tools.utils.messages.StackInfoStorage

• v0.19.5

  • Add http://bugs.python.org/file22767/hp_fix.diff for https://github.com/gregmuellegger/django/issues/1

• v0.19.4

  • Bugfix for PyPy in local_sync_cache get_cache_information(): sys.getsizeof() not implemented on PyPy

  • Bugfix in template.filters.chmod_symbol()

  • Nicer solution for template.filters.human_duration()

• v0.19.3

  • Add support for https in utils/http.py

• v0.19.2

  • Bugfix in utils/http.py timeout work-a-round

• v0.19.1

  • utils/http.py changes:

    • Use a better solution, see:

    • Add timeout and add a work-a-round for Python < 2.6

• v0.19.0

  • NEW: Add utils/http.py with helpers to get a webpage via http GET in unicode

  • Change README from textile to creole ;)

• v0.18.2

  • Bugfix: Add missing template in pypi package

• v0.18.0

  • NEW: Add DOM compare from Gregor Müllegger GSoC work into unittest utils.

• v0.17.1

  • Bugfix in “limit_to_usergroups”: Make choices “lazy”: Don’t access the database in init

• v0.17

  • Add the script “upgrade_virtualenv.py”

  • Add “limit_to_usergroups”

  • Add “local sync cache”

  • Add models.UpdateInfoBaseModel

  • Update decorators.render_to

  • render_to pass keyword arguments to render_to_response() (e.g.: mimetype=“text/plain”)

  • new argument “skip_fail” in get_filtered_apps(): If True: raise excaption if app is not importable

• v0.16.4

  • Bugfix: get_db_prep_save() got an unexpected keyword argument 'connection’ when save a SignSeparatedModelField()

• v0.16.3

  • Update BrowserDebug: Use response.templates instead of response.template and make output nicer

• v0.16.2

  • Merge stack info code and display better stack info on browser debug page

• v0.16.1

  • Update django_tools.utils.messages.StackInfoStorage for django code changes.

• v0.16.0

  • NEW: path model field (check if direcotry exist)

• v0.15.0

  • NEW: Add a flexible URL field (own validator, model- and form-field)

• v0.14.1

  • Bugfix: make path in MediaPathModelField relativ (remove slashes)

• v0.14

  • NEW: django-tagging addon: Display existing tags under a tag field

• v0.13

  • Bugfix UnicodeEncodeError in Browser debug

• v0.12

  • NEW: django_tools.utils.messages.failsafe_message

• v0.11

  • NEW: Store data in a secure cookie, see: utils/client_storage.py

• v0.10.1

  • New: Display used templates in unittest BrowserDebug

  • Bugfix: catch if last usermessages exist

• v0.10.0

  • NEW: utils around django messages, see: /django_tools/utils/messages.py

• v0.9.1

  • Bugfix: database column was not created: don’t overwrite get_internal_type()

• v0.9

  • New: stuff in /django_tools/fields/

  • see also backwards-incompatible changes, above!

• v0.8.2

  • New: widgets.SelectMediaPath(): Select a sub directory in settings.MEDIA_ROOT

  • New: fields.SignSeparatedField()

• v0.8.1

  • Add “no_args” keyword argument to installed_apps_utils.get_filtered_apps()

• v0.8.0

  • Add model LanguageCode field and form LanguageCode field in Accept-Language header format (RFC 2616)

• v0.7.0

  • Add decorators.py

• v0.6.0

  • Add forms_utils.LimitManyToManyFields, crosspost: http://www.djangosnippets.org/snippets/1691/

• v0.5.0

  • Add template/filters.py from PyLucid v0.8.x

• v0.4.0

  • Add experimental “warn_invalid_template_vars”

• v0.3.1

  • Bugfix: Exclude the instance if it was saved in the past.

• v0.3.0

  • Add utils.installed_apps_utils

• v0.2.0

  • Add models_utils, see: http://www.jensdiemer.de/_command/118/blog/detail/67/ (de)

• v0.1.0

  • first version cut out from PyLucid CMS – http://www.pylucid.org

pip upgrade

To do a pip upgrade in a virtual environment, run this:

~$ cd /YourVirtualEnv/bin
~/YourVirtualEnv/bin$ source activate
(YourVirtualEnv)~/YourVirtualEnv/bin$ pip install --upgrade --verbose --editable=git+git://github.com/jedie/django-tools.git#egg=django-tools

The example used git readonly clone url. If you use subversion do this:

(YourVirtualEnv)~/YourVirtualEnv/bin$ pip install --upgrade --verbose --editable=http://svn.github.com/jedie/django-tools.git#egg=django-tools

If you have git write access, use this:

(YourVirtualEnv)~/YourVirtualEnv/bin$ pip install --upgrade --verbose --editable=[e-mail]:jedie/django-tools.git#egg=django-tools

…or just use our limit_to_usergroups.py script (see above)

fast repo update

To made a fast repository update, you can run this simple shell script:

~$ cd /path/to/django-tools
/path/to/django-tools$ ./update.sh

the update script runs “git pull origin master” or “svn update”.

links

Homepage	http://github.com/jedie/django-tools
PyPi	http://pypi.python.org/pypi/django-tools/

contact

Come into the conversation, besides the github communication features:

Forum	official ‘django-tools’ Forum
IRC	#pylucid on freenode.net (Yes, the PyLucid channel…)
webchat	http://webchat.freenode.net/?channels=pylucid

donation

• paypal.me/JensDiemer

• Flattr This!

• Send Bitcoins to 1823RZ5Md1Q2X5aSXRC5LRPcYdveCiVX6F