django-compat-patcher 0.11

A monkey-patcher to ease the transition of project to latest Django versions.

django-compat-patcher

Compatibility Matters

DCP is a companion application which adds backward/forward compatibility patches to Django, so that your app ecosystem doesn’t get broken by trivial changes made to the core of the framework. You can thus mix bleeding-edge applications with others that are still stuck at much older Django versions.

To know more about the whole concept of Compat Patchers, see the documentation of the underlying Compat Patcher Core.

Note that DCP is aimed at project maintainers. If you are developing a reusable Django application, you can’t expect all your users to integrate DCP as well. In this case, to support a wide range of Django versions, you should rather use a toolkit like Django-compat. You may think of DCP as a “runtime 2to3 and 3to2 for the Django core framework’, whereas Django-Compat is rather a “six module for Django”. If you only seek to upgrade your own codebase to newer Django versions, look at Django Codemod too (which is like a “static 2to3 for Django-dependent code repositories”).

Feel free to ask for (or contribute) new fixers, for backwards or forwards compatibility, depending on the compatibility troubles you encounter on your own projects. See docs/django_deprecation_timeline_notes.rst for a list of breaking changes in Django history, and their current status in DCP.

How to setup

Django-compat-patcher is currently tested on python 3.6/3.7/3.8/3.9/3.10, with Django versions 1.8/1.9/1.10/1.11/2.0/2.1/2.2/3.0/3.1/3.2/4.0, where these combinations make sense (e.g. Django2+ dropped support for Python2).

First add django-compat-patcher to your project requirements, or install it directly with pip install django-compat-patcher.

The Django settings of your project are not altered by compatibility shims, so they should be kept up-to-date with your installed Django version (eg. now use TEMPLATES, MIDDLEWARE, and not deprecated equivalents). In particular, always put real package names in your INSTALLED_APPS, not their potential “import aliases”.

Despite DCP patching, you might encounter errors raised by the Django check framework, like the following. Use the SILENCED_SYSTEM_CHECKS setting to bypass such blocking checks.

(fields.E900) IPAddressField has been removed except for support in historical migrations. HINT: Use GenericIPAddressField instead.
(fields.E160) The options auto_now, auto_now_add, and default are mutually exclusive. Only one of these options may be present.
(fields.E903) NullBooleanField is removed except for support in historical migrations.

Activation method 1 - with code

You can activate patching with:

import django_compat_patcher
django_compat_patcher.patch()

This code should be placed before any use of Django (eg. in your manage.py or your wsgi.py script), but after the DJANGO_SETTINGS_MODULE environment variable has been set.

In particular, some fixers only work if they are applied before the loading of INSTALLED_APPS (so before django.setup() gets called).

Note for Pytest-Django users

Pytest-Django triggers django.setup() early during test suite execution, so to place your django_compat_patcher.patch() before, you might have to use a pytest plugin as explained here. Use plugin pytest-pythonpath if your plugin is not at repository root.

Activation method 2 - with python wrapper

You may force the patching of Django at python startup using https://pypi.org/project/main-wrapper/:

pip install main-wrapper
export DJANGO_SETTINGS_MODULE=<your-settings-module>  # Mandatory
export DCP_INCLUDE_FIXER_FAMILIES='["django3.0"]'  # Optional DCP setting tweaks
python-main-wrapper django_compat_patcher:patch <your-normal-command-line>

This unintrusive method is especially useful to repeatedly launch the unit-tests of a library, with different settings, and thus determine how many fixers it needs to function properly under latest Django version:

python-main-wrapper django_compat_patcher:patch manage.py test
python-main-wrapper django_compat_patcher:patch pytest
python-main-wrapper django_compat_patcher:patch -m <some-module>

See python-main-wrapper -h for more details on this launcher.

Tweaking DCP settings

By default, DCP emits logs and warnings when patching the code, and applies all “relevant” fixers, i.e all that support your currently installed django version, and are not deemed unsafe.

Unsafe fixers are the few ones which might conflict with modern third-party libraries , e.g. if these add their own workarounds for Django incompatibilites (see DCP_EXCLUDE_FIXER_IDS default below).

This behaviour can be customized via the Django settings documented below.

Note however, that some fixers depend on other fixers, so it’s advised to be consistent and always include contiguous series of fixers around your current version (ex. if you use Django1.11, apply fixers from Django1.8 up to Django1.11, or up to Django2.X if you want some forward compatibility as well). DCP filters out, by himself, fixers which are useless for your Django version.

You might also provide a settings dictionary directly to patch(), in which case the DCP django settings of your project will be entirely ignored (only DCP library defaults will be used as fallbacks):

django_compat_patcher.patch(settings=dict(DCP_INCLUDE_FIXER_IDS=["my_fixer_id"]))

It is also possible to override only one or more of these settings by using environment variables with the same name (e.g. “DCP_INCLUDE_FIXER_IDS”), in JSON format (so a string must be passed as ‘”*”’ for example, or a boolean as ‘true’; beware of single quotes, forbidden for JSON strings).

Note that exclusion filters have precedence over inclusion ones.

DCP_INCLUDE_FIXER_IDS

List of fixer identifiers to include. If "*" is used, then all fixers are included.

In rare case of name conflicts when using several registries at once, you may use qualified qualified fixer names like “fixer_family|fixer_id”.

Default: '*'

Type: List of strings, or "*"

Example: DCP_INCLUDE_FIXER_IDS = ['fix_deletion_templatetags_future_url']

DCP_INCLUDE_FIXER_FAMILIES

List of fixer families to include. If "*" is used, then all families are included.

Note: If you want to include only specific families, remember to replace the value "*" from :code:`DCP_INCLUDE_FIXER_IDS by, for example, an empty list.

Default: []

Type: List of strings, or "*"

Choices: ("djangoX.Y") where X and Y are respectively the major and minor versions

Example: DCP_INCLUDE_FIXER_FAMILIES = ["django1.9"]

DCP_EXCLUDE_FIXER_IDS

List of fixer identifiers to exclude. If "*" is used, then all fixers are excluded.

In rare case of name conflicts when using several registries at once, you may use qualified qualified fixer names like “fixer_family|fixer_id”.

Note: The “EXCLUDE” filters are applied AFTER the “INCLUDE” ones, and so take precedence.

Default: ['fix_behaviour_core_management_parser_optparse', 'fix_deletion_contrib_postgres_forms_jsonb_InvalidJSONInput_JSONString', 'fix_deletion_contrib_postgres_fields_jsonb_JsonAdapter', 'fix_deletion_contrib_postgres_forms_jsonb', 'fix_deletion_contrib_postgres_fields_jsonb']

Type: List of strings, or "*"

Example: DCP_EXCLUDE_FIXER_IDS = ['fix_deletion_templatetags_future_url']

DCP_EXCLUDE_FIXER_FAMILIES

List of fixer families to exclude. If "*" is used, then all families are excluded.

Note: The “EXCLUDE” filters are applied AFTER the “INCLUDE” ones, and so take precedence.

Default: []

Type: List of strings, or "*"

Choices: ("djangoX.Y") where X and Y are respectively the major and minor versions

Example: DCP_EXCLUDE_FIXER_FAMILIES = ["django1.6", "django1.9"]

DCP_PATCH_INJECTED_OBJECTS

By default, the patcher sets an attribute (with value True) on injected objects (callables, classes, modules, attributes…) when possible, with this attribute name, to differentiate them from original objects. Set this setting to True to automatically choose the attribute name, or False to disable the feature.

Default: '__dcp_injected__'

Type: Str (or Boolean)

Example: DCP_PATCH_INJECTED_OBJECTS = False

DCP_ENABLE_WARNINGS

If True, compatibility shims emit python warnings (warnings.warn(...)) when they are imported/used, to help detect deprecated code. These warnings are mostly subclasses of DeprecationWarning (ex. RemovedInDjango19Warning).

Once emitted, the handling of warnings depends on your setup (python command line flags, logging config…), see the official doc on warnings for more information.

Default: True

Type: Boolean

Example: DCP_ENABLE_WARNINGS = False

DCP_LOGGING_LEVEL

The patch() system of DCP can output to STDERR which fixers are getting applied, and provide debug information (ex. for which reason a specific fixer was discarded).

This setting sets the logging level of that information stream, which is typically only viewed at django startup. A value None disables DCP logging entirely.

Note that DCP does NOT actually use stdlib loggers, because it mostly performs operations before Django logging has been setup (ex. using the LOGGING setting), so log entries would most probably get discarded.

Default: 'INFO'

Type: Logging level string, or None

Example: DCP_LOGGING_LEVEL = "DEBUG"

Table of fixers

There are currently 83 available fixers.

Fixer and its ID	Fixer family	Min version	Max version
Preserve the request.raw_post_data alias for request.body. (fix_deletion_http_request_HttpRequest_raw_post_data)	django1.6	1.6
Keep ‘django.contrib.comments’ as an import alias for the now external package ‘django_comments’ (django-contrib-comments on pypi) ; the latter must be installed separately. (fix_outsourcing_contrib_comments)	django1.8	1.8
Preserve the get_formsets method of ModelAdmin (fix_deletion_contrib_admin_ModelAdmin_get_formsets)	django1.9	1.9
Preserve contrib.sites.models.RequestSite alias. (fix_deletion_contrib_sites_models_RequestSite)	django1.9	1.9
Preserve contrib.sites.models.get_current_site alias. (fix_deletion_contrib_sites_models_get_current_site)	django1.9	1.9
Preserve django.core.cache.get_cache() utility, superseded by django.core.cache.caches (fix_deletion_core_cache_get_cache)	django1.9	1.9
Preserve the `request.REQUEST` attribute, merging parameters from GET (fix_deletion_core_handlers_wsgi_WSGIRequest_REQUEST)	django1.9	1.9
Preserve the fallback to AppCommand.handle_app() method in django management commands. (fix_deletion_core_management_base_AppCommand_handle_app)	django1.9	1.9
Preserve the IPAddressField form field, now superseded by GenericIPAddressField (fix_deletion_forms_fields_IPAddressField)	django1.9	1.9
Preserve the `ssi` tag in the `future` templatetags library. (fix_deletion_templatetags_future_ssi)	django1.9	1.9
Preserve the `url` tag in the `future` templatetags library. (fix_deletion_templatetags_future_url)	django1.9	1.9
Preserve the MergeDict util datastructure (fix_deletion_utils_datastructures_MergeDict)	django1.9	1.9
Preserve the SortedDict util datastructure (fix_deletion_utils_datastructures_SortedDict)	django1.9	1.9
Preserve the dictconfig util file (fix_deletion_utils_dictconfig)	django1.9	1.9
Preserve utils.functional.memoize() utility (fix_deletion_utils_functional_memoize)	django1.9	1.9
Preserve the importlib util file (fix_deletion_utils_importlib)	django1.9	1.9
Preserve the tzinfo util file (fix_deletion_utils_tzinfo)	django1.9	1.9
Preserve the unittest util file (fix_deletion_utils_unittest)	django1.9	1.9
Support passing views to url() as dotted strings instead of view objects. (fix_behaviour_conf_urls_url)	django1.10	1.10
[UNSAFE] Preserve the support for old optparse instead of argparse parser, in management commands. Beware, Bash shell autocompletion might fail if some management commands use Optparse! (fix_behaviour_core_management_parser_optparse)	django1.10	1.10
Preserve the ability to call urlresolver on dotted string view, instead of explicit view name. (fix_behaviour_core_urlresolvers_reverse_with_prefix)	django1.10	1.10
Preserve support for a single ‘=’ sign in {% if %} tag. (fix_behaviour_template_smartif_OPERATORS_equals)	django1.10	1.10
Restore support for dotted-string view parameter in RegexURLPattern, instead passing a view object. (fix_behaviour_urls_resolvers_RegexURLPattern)	django1.10	1.10
Preserve the patterns() builder for django urls. (fix_deletion_conf_urls_patterns)	django1.10	1.10
Preserve the “ssi” default template tag. (fix_deletion_template_defaulttags_ssi)	django1.10	1.10
Preserve the “future” templatetags library, with its improved `firstof` and `cycle` tags. (fix_deletion_templatetags_future)	django1.10	1.10
Put a forward compatibility import path for django.urls, which replaces django.core.urlresolvers (fix_incoming_urls_submodule)	django1.10		1.10
Preserve compatibility with the old signature of Widget.build_attrs(): extra_attrs=None, **kwargs. (fix_behaviour_widget_build_attrs)	django1.11	1.11
Set a forward compatibility wrapper for setup_test_environment() which takes a “debug” argument later. (fix_incoming_test_utils_setup_test_environment_signature_change)	django1.11		1.11
Keep accepting a 3-tuple (urlconf_module, app_name, namespace) as first argument of include(), instead of providing namespace argument directly to include() (fix_behaviour_conf_urls_include_3tuples)	django2.0	2.0
Make user.is_anonymous and user.is_authenticated behave both as properties and methods, by preserving their callability like in earlier Django version. (fix_behaviour_contrib_auth_user_is_anonymous_is_authenticated_callability)	django2.0	2.0
Let “on_delete” parameter of ForeignKey and OneToOneField be optional, defaulting to CASCADE. (fix_behaviour_db_models_fields_related_ForeignKey_OneToOneField)	django2.0	2.0
Restore support for direct assignment to the reverse side of a related set, in many-to-one and many-to-many relationships. (fix_behaviour_db_models_fields_related_descriptors_ReverseManyToOneDescriptor_setter)	django2.0	2.0
Preserve django.core.urlresolvers module, now replaced by django.urls. (fix_deletion_core_urlresolvers)	django2.0	2.0
Preserve the Context.has_key() utility, replaced by “in” operator use. (fix_deletion_template_context_Context_has_key)	django2.0	2.0
Preserve the assignment_tag() helper, superseded by simple_tag(). (fix_deletion_template_library_assignment_tag)	django2.0	2.0
Preserve RegexURLPattern and RegexURLResolver in django.urls, which disappeared due to DEP 0201. (fix_deletion_urls_RegexURLPattern_RegexURLResolver)	django2.0	2.0
Preserve the allow_lazy() utility, superseded by keep_lazy(). (fix_deletion_utils_functional_allow_lazy)	django2.0	2.0
Preserve the javascript_catalog() and json_catalog() i18n views, superseded by class-based views. (fix_deletion_views_i18n_javascript_and_json_catalog)	django2.0	2.0
Restore the behaviour where the “renderer” parameter of Widget.render() may not be supported by subclasses. (fix_behaviour_widget_render_forced_renderer)	django2.1	2.1
Preserve django.utils.translation.string_concat(), superseded by django.utils.text.format_lazy(). (fix_deletion_utils_translation_string_concat)	django2.1	2.1
Preserve the field_name keyword argument to QuerySet.earliest() and latest() (fix_behaviour_db_models_query_QuerySet_earliest_latest)	django3.0	3.0
Preserve staticfiles and admin_static template tag libraries. (fix_deletion_contrib_staticfiles_templatetags_and_admin_static)	django3.0	3.0
Preserve HttpRequest.xreadlines(), replaced by iteration on request object. (fix_deletion_http_request_HttpRequest_xreadlines)	django3.0	3.0
Preserve django.shortcuts.render_to_response(), superseded by render(). (fix_deletion_shortcuts_render_to_response)	django3.0	3.0
Preserve django.test.utils.patch_logger() context manager. (fix_deletion_test_utils_patch_logger)	django3.0	3.0
Preserve django.test.utils.str_prefix class. (fix_deletion_test_utils_str_prefix)	django3.0	3.0
Preserve django.utils.decorators.ContextDecorator, alias of contextlib.ContextDecorator. (fix_deletion_utils_decorators_ContextDecorator)	django3.0	3.0
Preserve django.utils.decorators.available_attrs, which just returns functools.WRAPPER_ASSIGNMENTS. (fix_deletion_utils_decorators_available_attrs)	django3.0	3.0
Preserve django.utils.encoding.python_2_unicode_compatible() class decorator. (fix_deletion_utils_encoding_python_2_unicode_compatible)	django3.0	3.0
Preserve django.utils.functional.curry()function. (fix_deletion_utils_functional_curry)	django3.0	3.0
Preserve django.utils.http.cookie_date(), superseded by http_date(). (fix_deletion_utils_http_cookie_date)	django3.0	3.0
Preserve django.utils.lru_cache.lru_cache(), alias of functools.lru_cache(), and its containing module. (fix_deletion_utils_lru_cache_lru_cache)	django3.0	3.0
Preserve django.utils.safestring.SafeBytes class. (fix_deletion_utils_safestring_SafeBytes)	django3.0	3.0
Preserve the vendored copy of “six” compatibility utility, in django.utils, as well as the `six` import in django.utils.encoding (fix_deletion_utils_six)	django3.0	3.0
Preserve python2 path normalization functions. (fix_deletion_utils_upath_npath_abspathu)	django3.0	3.0
Preserve import of ACTION_CHECKBOX_NAME in django.contrib.admin (fix_deletion_contrib_admin_ACTION_CHECKBOX_NAME)	django3.1	3.1
[UNSAFE] Preserve undocumented JsonAdapter class in django.contrib.postgres.fields.jsonb Requires psycopg2 to be installed. (fix_deletion_contrib_postgres_fields_jsonb_JsonAdapter)	django3.1	3.1
[UNSAFE] Preserve undocumented InvalidJSONInput and JSONString classes in django.contrib.postgres.forms.jsonb Requires psycopg2 to be installed. (fix_deletion_contrib_postgres_forms_jsonb_InvalidJSONInput_JSONString)	django3.1	3.1
Preserve the compatibility alias django.core.management.commands.runserver.BaseRunserverCommand (fix_deletion_core_management_commands_runserver)	django3.1	3.1
Preserve compatibility import of django.core.exceptions.FieldDoesNotExist in django.db.models.fields (fix_deletion_db_models_fields_FieldDoesNotExist)	django3.1	3.1
Preserve compatibility imports of django.core.exceptions.EmptyResultSet in django.db.models.query, django.db.models.sql, and django.db.models.sql.datastructures (fix_deletion_db_models_submodules_EmptyResultSet)	django3.1	3.1
Preserve the compatibility import of django.core.validators.EMPTY_VALUES in django.forms.fields (fix_deletion_forms_fields_EMPTY_VALUES)	django3.1	3.1
Preserve the compatibility imports of django.forms.utils.pretty_name() and django.forms.boundfield.BoundField in django.forms.forms (fix_deletion_forms_forms_pretty_name_BoundField)	django3.1	3.1
Preserve the compatibility imports django.template.Context, django.template.RequestContext and django.template.ContextPopException (fix_deletion_template_base_Context_classes)	django3.1	3.1
Preserve django.utils.decorators.classproperty as alias of new django.utils.functional.classproperty (fix_deletion_utils_decorators_classproperty)	django3.1	3.1
Preserve undocumented ExceptionReporterFilter class (fix_deletion_views_debug_ExceptionReporterFilter)	django3.1	3.1
Preserve HttpResponseBase._headers as an alias to the new HttpResponseBase.headers (fix_deletion_http_response_HttpResponseBase_private_headers)	django3.2	3.2
Keep accepting the `providing_args` init argument of Signal instances. (fix_behaviour_dispatch_dispatcher_Signal_providing_args)	django4.0	4.0
Keep `get_response` argument optional and nullable in middleware classes (fix_behaviour_middleware_get_response_parameter_nullability)	django4.0	4.0
Allow get_random_string() call without length argument (defaults to length=12) (fix_behaviour_utils_crypto_get_random_string_length)	django4.0	4.0
Preserve django.conf.urls.url() as an alias to django.urls.re_path() (fix_deletion_conf_urls_url)	django4.0	4.0
[UNSAFE] Preserve django.contrib.postgres.fields.jsonb.KeyTransform/KeyTextTransform as aliases to django.db.models.fields.json objects (fix_deletion_contrib_postgres_fields_jsonb)	django4.0	4.0
[UNSAFE] Preserve django.contrib.postgres.forms.JSONField and its jsonb source module (fix_deletion_contrib_postgres_forms_jsonb)	django4.0	4.0
Preserve the django.db.models.query_utils.InvalidQuery exception class (fix_deletion_db_models_query_utils_InvalidQuery)	django4.0	4.0
Preserve “list” error message for ModelMultipleChoiceField, replaced by “invalid_list” (fix_deletion_forms_models_ModelMultipleChoiceField_error_messages_list_entry)	django4.0	4.0
Preserve the HttpRequest.is_ajax() method (fix_deletion_http_request_HttpRequest_is_ajax)	django4.0	4.0
Preserve {% ifequal %} and {% ifnotequal %} builtin template tags (fix_deletion_template_defaulttags_ifequal_ifnotequal)	django4.0	4.0
Preserve django.utils.encoding.force_text() and smart_text() as aliases for force_str() and smart_str() (fix_deletion_utils_encoding_smart_force_text)	django4.0	4.0
Preserve django.utils.http.is_safe_url() as an alias to url_has_allowed_host_and_scheme() (fix_deletion_utils_http_is_safe_url)	django4.0	4.0
Preserve aliases of urlib methods (quote, quote_plus, unquote, unquote_plus) in django.utils.http (fix_deletion_utils_http_quote_utilities)	django4.0	4.0
Preserve django.utils.text.unescape_entities() as an alias of html.unescape() (fix_deletion_utils_text_unescape_entities)	django4.0	4.0
Preserve ugettext(), ugettext_lazy(), ugettext_noop(), ungettext(), and ungettext_lazy() as aliases of gettext methods (fix_deletion_utils_translation_ugettext_utilities)	django4.0	4.0