django-smartstaticfiles 0.3.2

Provides enhanced static files storage backend for Django 1.11

django-smartstaticfiles enhances the functionalities of collectstatic management command of Django 1.11.x, allows for finer-grained control over serving static files in production.

Under the hood, it provides a file storage backend for use with STATICFILES_STORAGE setting, which inherits and improves Django’s ManifestStaticFilesStorage storage backend.

Features

• Deletes unhashed files and intermediate files by default.

• Optionally ignores hashing of specific files.

• Optionally minifies JavaScript and CSS files.

• Optionally replace JavaScript asset URLs with hashed versions using loud comments markup. (New in v0.2.0)

• Optimizes hashing process with fewer I/O and less calculation.

Quick Start

1. Install the stable version from PyPI:

   pip install django-smartstaticfiles

   Or install the stable version with extras for JavaScript and CSS minification (will also install rjsmin and rcssmin):

   pip install django-smartstaticfiles[jsmin,cssmin]

   Or install the latest version from GitHub:

   pip install git+https://github.com/rockallite/django-smartstaticfiles.git

2. Add the following lines to the project’s Django settings module:

   STATIC_ROOT = '/path/for/collecting/static/files'
   
   STATICFILES_STORAGE = 'django_smartstaticfiles.storage.SmartManifestStaticFilesStorage'
   
   # Remove this if you don't need to minify JavaScript and CSS
   SMARTSTATICFILES_CONFIG = {
       'JS_MIN_ENABLED': True,
       'CSS_MIN_ENABLED': True,
   }

3. In the project directory, collect static files by running the following command:

   python manage.py collectstatic --clear --no-input

JavaScript Asset URLs Replacement

(New in v0.2.0)

By default, URLs of referenced assets (images, fonts, etc) in CSS files will be replaced with hashed versions during processing. django-smartstaticfiles extends this feature to JavaScript files by utilizing special loud comments (/*! ... */) markup.

Simple use case

The JavaScript asset URLs replacement feature is disabled by default. To enable it, add the following setting to Django settings module:

SMARTSTATICFILES_CONFIG = {
    # Enable JavaScript asset URLs replacement
    'JS_ASSETS_REPL_ENABLED': True,
}

To replace an asset URL with the hashed version, surround the URL string with /*! rev */ and /*! endrev */ markup:

var imageURL = /*! rev */ '../img/welcome.jpg' /*! endrev */;

Supposed that the hashed filename is welcome.ac99c750806a.jpg, the processing result will be:

var imageURL = '../img/welcome.ac99c750806a.jpg';

Only a single- or double-quoted bare string should be put inside /*! rev */ and /*! endrev */ markup. No comma or semicolon is allowed. Spaces around or inside loud comments are optional, though.

Using a different parent path

By default, relative asset URLs are considered to be relative to the referencing JavaScript file, just the same rule for a CSS file. However, since JavaScript runs in global scope of a browser, the path of a JavaScript file is sometimes not useful for locating relative assets.

Therefore, the markup accepts a parameter as virtual parent path, passing in between a pair of parentheses right behind the loud comment starting tag, like this: /*! rev(path) */. During processing, it will be considered as if it were the parent path of the asset. For example:

/*
 * Supposed there are following files:
 *     STATIC_URL/helloworld/img/welcome.jpg
 *     STATIC_URL/helloworld/js/main.js
 *
 * Then in the main.js:
 */

var imageURLs = [
    // *** Absolute reference ***
    // (STATIC_URL as the root path)

    // Leading and trailing slashes in a virtual parent path are optional
    /*! rev(helloworld/img) */ 'welcome.jpg' /*! endrev */,
    /*! rev(/helloworld/img/) */ 'welcome.jpg' /*! endrev */,
    /*! rev(/helloworld/img) */ 'welcome.jpg' /*! endrev */,
    /*! rev(helloworld/img/) */ 'welcome.jpg' /*! endrev */,

    // A leading dot slash (./) or dot-dot slash (../) in an asset URL is OK
    /*! rev(helloworld/img) */ './welcome.jpg' /*! endrev */,
    /*! rev(helloworld/img) */ '../img/welcome.jpg' /*! endrev */,

    // Use different path portion in a virtual parent path. A single slash means root (STATIC_URL).
    /*! rev(helloworld) */ 'img/welcome.jpg' /*! endrev */,
    /*! rev(/) */ 'helloworld/img/welcome.jpg' /*! endrev */,

    // *** Relative reference ***
    // (Relative to the JavaScript file)

    // A leading dot (.) or dot-dot (..) path part in a virtual parent path indicates a relative reference
    /*! rev(../img) */ 'welcome.jpg' /*! endrev */,
    /*! rev(..) */ 'img/welcome.jpg' /*! endrev */,
    /*! rev(../..) */ 'helloworld/img/welcome.jpg' /*! endrev */
];

After processing, the above code becomes:

/*
 * Supposed there are following files:
 *     STATIC_URL/helloworld/img/welcome.jpg
 *     STATIC_URL/helloworld/js/main.js
 *
 * Then in the main.js:
 */

var imageURLs = [
    // *** Absolute reference ***
    // (STATIC_URL as the root path)

    // Leading and trailing slashes in a virtual parent path are optional
    'welcome.ac99c750806a.jpg',
    'welcome.ac99c750806a.jpg',
    'welcome.ac99c750806a.jpg',
    'welcome.ac99c750806a.jpg',

    // A leading dot slash (./) or dot-dot slash (../) in an asset URL is OK
    './welcome.ac99c750806a.jpg',
    '../img/welcome.ac99c750806a.jpg',

    // Use different path portion in a virtual parent path. A single slash means root (STATIC_URL).
    'img/welcome.ac99c750806a.jpg',
    'helloworld/img/welcome.ac99c750806a.jpg',

    // *** Relative reference ***
    // (Relative to the JavaScript file)

    // A leading dot (.) or dot-dot (..) path part in a virtual parent path indicates a relative reference
    'welcome.ac99c750806a.jpg',
    'img/welcome.ac99c750806a.jpg',
    'helloworld/img/welcome.ac99c750806a.jpg'
];

Notice that STATIC_URL WILL NOT be prepended to the final URL. You have to pass the value of STATIC_URL to the browser, e.g. via Django templates in dynamic generated JavaScript code, and then manually concatenate the value and the URL path in JavaScript.

Customizing the tag name

You can also use a custom tag name in loud comments markup via the following setting in Django settings module:

SMARTSTATICFILES_CONFIG = {
    # ...
    # Tag name of loud comments used in JavaScript asset URLs replacement
    'JS_ASSETS_REPL_TAG': 'hash-it',
}

Then the corresponding JavaScript code should be written as:

var imageURL = /*! hash-it */ '../img/welcome.jpg' /*! endhash-it */;

Hints about minification

If you use a customized JavaScript minification function, you should ensure that loud comments (/*! ... */) are preserved after processing. Otherwise, JavaScript asset URLs replacement won’t work. The default jsmin library takes care of that.

Some JavaScript minification libraries (e.g. jsmin) will deliberately insert a newline at the end of each loud comment after minification. For example, supposed that there is following code:

var imageURL = /*! rev */ '../img/welcome.jpg' /*! endrev */;
var mehFace = 'mehFace';

It would be minified as:

var imageURL=/*! rev */
'../img/welcome.jpg'/*! endrev */
;var mehFace='mehFace';

This is totally acceptable in most cases. However, it is still possible that it causes unexpected results in some edge cases or drives perfectionists nuts. You can tell django-smartstaticfiles to remove one trailing newline (if presents) from each replaced URL in JavaScript by setting "JS_ASSETS_REPL_TRAILING_FIX" to True. The final result after URLs replacement would be:

var imageURL='../img/welcome.ac99c750806a.jpg';var mehFace='mehFace';

(New in v0.3.0: the JS_ASSETS_REPL_TRAILING_FIX setting was added and defaults to True .)

(New in v0.3.1: the JS_ASSETS_REPL_TRAILING_FIX setting was set to False by default.)

Configurations

All configurations of django-smartstaticfiles are in the SMARTSTATICFILES_CONFIG property of Django settings module, a dict containing configuration keys. All keys are optional, which means you don’t even need a SMARTSTATICFILES_CONFIG property at all if the default values meet your needs.

Possible keys and default values are listed below:

SMARTSTATICFILES_CONFIG = {
    # Whether to enable JavaScript minification.
    'JS_MIN_ENABLED': False,

    # Whether to enable CSS minification.
    'CSS_MIN_ENABLED': False,

    # File patterns for matching JavaScript assets (in relative URL without
    # STATIC_URL prefix)
    'JS_FILE_PATTERNS': ['*.js'],

    # File patterns for matching CSS assets (in relative URL without
    # STATIC_URL prefix)
    'CSS_FILE_PATTERNS': ['*.css'],

    # Dotted string of the module path and the callable for JavaScript
    # minification. The callable should accept a single argument of a string
    # of the content of original JavaScript, and return a string of minified
    # content. (Notice that loud comments such as /*! ... */ must be preserved
    # in the result so as to make JavaScript asset URLs replacement work.)
    # The result will be cached and reused when possible.
    'JS_MIN_FUNC': 'rjsmin.jsmin',

    # Extra keyword arguments which are sent to the callable for JavaScript
    # minification. They are sent after the argument of a string of the
    # content of original JavaScript. If no keyword arguments are sent, set it
    # to an empty dict ({}) or None.
    'JS_MIN_FUNC_KWARGS': {
        'keep_bang_comments': True,
    },

    # Dotted string of the module path and the callable for CSS
    # minification. The callable should accept a single argument of
    # string which contains the content of original CSS, and return a
    # string of minified content. The result will be cached and
    # reused when possible.
    'CSS_MIN_FUNC': 'rcssmin.cssmin',

    # Extra keyword arguments which are sent to the callable for CSS
    # minification. They are sent after the argument of a string of the
    # content of original CSS. If no keyword arguments are sent, set it
    # to an empty dict ({}) or None.
    'CSS_MIN_FUNC_KWARGS': {
        'keep_bang_comments': True,
    },

    # A regular expression (case-sensitive by default) which is used to
    # search against assets (in relative URL without STATIC_URL prefix). The
    # mathced assets won't be minified. Set it to None to ignore no assets.
    # (Assets with .min.js or .min.css extensions are always ignored.)
    'RE_IGNORE_MIN': None,

    # Whether to enable deletion of unhashed files.
    'DELETE_UNHASHED_ENABLED': True,

    # Whether to enable deletion of intermediate hashed files.
    'DELETE_INTERMEDIATE_ENABLED': True,

    # A regular expression (case-sensitive by default) which is used to
    # search against assets (in relative URL without STATIC_URL prefix). The
    # matched assets won't be hashed. Set it to None to ignore no assets.
    'RE_IGNORE_HASHING': None,

    # Whether to enable JavaScript asset URLs replacement.
    'JS_ASSETS_REPL_ENABLED': False,

    # Tag name of loud comments used in JavaScript asset URLs replacement.
    'JS_ASSETS_REPL_TAG': 'rev',

    # Whether to remove one trailing newline (if presents) after each
    # replaced URL in JavaScript. This is effective only if "JS_MIN_ENABLED"
    # is set to True. This fixes the problems and annoyances caused by a
    # deliberately added newline at the end of each loud comment by certain
    # minification libraries (e.g. jsmin).
    'JS_ASSETS_REPL_TRAILING_FIX': False,
}

Extensibility

The SmartManifestStaticFilesStorage storage backend provided by django-smartstaticfiles inherits two parent classes:

class SmartManifestStaticFilesStorage(SmartManifestFilesMixin, StaticFilesStorage):
    pass

The main logic is implemented in SmartManifestFilesMixin, which is similar to Django’s ManifestStaticFilesStorage:

class ManifestStaticFilesStorage(ManifestFilesMixin, StaticFilesStorage):
    pass

The goal of this project is to make SmartManifestFilesMixin a drop-in replacement for ManifestFilesMixin, without sacrificing functionalities or performance. So you can combine SmartManifestFilesMixin with other storage class that is compatible with ManifestFilesMixin.

For example, django-s3-storage provides a storage backend which utilizes Django’s ManifestFilesMixin:

# django_s3_storage/storage.py
from django.contrib.staticfiles.storage import ManifestFilesMixin

# ...

class ManifestStaticS3Storage(ManifestFilesMixin, StaticS3Storage):
    pass

You can make a similar but enhanced storage backend by replacing it with SmartManifestFilesMixin:

from django_s3_storage.storage import StaticS3Storage
from django_smartstaticfiles.storage import SmartManifestFilesMixin


class SmartManifestStaticS3Storage(SmartManifestFilesMixin, StaticS3Storage):
    pass

Why Django 1.11.x only?

Until version 1.11, Django shipped with a ManifestStaticFilesStorage storage backend that had a broken implementation. In other words, content changes in referenced files (images, fonts, etc) aren’t represented in hashes of referencing files (CSS files, specifically). This breaks the foundation of cache-busting mechanism.

Then, there are significant code changes in Django 1.11.x in order to fix the behavior of the ManifestStaticFilesStorage storage backend. And it becomes impractical to maintain compatibility of django-smartstaticfiles with older Django code. Therefore, only Django 1.11.x is supported (the latest version at the time of writing).