django_image_lite 1.5.0

Local image handling for Django. Unobtusive, with multiple repositories, powerful filter system and scaleable data.

ImageLite

NB: Version 1, because incompatible change behind API: filters now generate subfolders, not use prefix naming (I never used the feature, don't believe anyone else is)

An app for local filesystem image storage. In truth, it's not 'Lite', with custom repository creation from an abstract base, an easy to configure and massively capable filter stack, enhanced upload options, and powerful template tags for rendering. But it's a reduction of a more general app, to avoid Django annoyances, so 'Lite'.

This is a rewrite of Wagtail's Image app.

Why you would not want to use this app

• It's (currently) written for local filesystem storage only (no remote)
• Raster image files only, no SVG, movie files, documents, etc.
• Filters can not be applied to individual images
• No effort to present filters to users, either admin or visitors
• The user must manage filenames (see below)
• No builtin categorisation and/or tagging

Why you would want to use this app

• Abstract base allows any number of repositories with custom configurations
• If you want to scale beyond the app, the data is unobtrusive and needs no rebuilding
• Auto-generates filtered images
• Filter declarations can travel with apps (like CSS)
• Two (primary) template tags

The key point about this app---please read

If you wish to associate files with pages, the app can do this. But the app is not designed to use a Foreign field. It is designed to avoid use of foreign fields.

So, if you have a model of, say, a product, how do you associate uploaded images with the model? You use a URL. The product is called 'BansheeX4000'. So an image of this product must have the filename 'BansheeX4000.png' (the URLs are easily configurable, but this is the most likely scheme).

There are downsides to this approach,

• It puts the burden on the user to create filenames as intended. A solution using foreign keys could upload any image, then automatically make the connection
• Since the models do not know (much) about the attached images, editing of uploaded images is pushed back onto management commands
• Django admin for the models can not display the upload form (no model field to render).

But there are upsides,

• Django admin displays for foreign fields are very limited. As default they are select boxes. ImageLite's URL system is more pleasant.
• Without foreign fields, the system is easier to scale. If you wish to move image handling out of Django, there are no foreign fields to remove, and image data is naturally organised

Overview

There is a base model called AbstractImage. From this, sub-models can be constructed. Each sub-model is an image repository. Sub-models track uploads in a database table, and store files in a location. Locations are in subfolders of MEDIA_ROOT.

Each original image can generate derivative images. Derivative images are untracked and generated on a successful upload. The derivative images are called 'reforms' and are generated by filters. Reforms can be thumbnails, watermarked etc.

Image delivery is by template tag. The tags write HTML elements with an appropriate URL. The model tag constructs the URL by asking the model instances they receive.

If you have done this before

• subclass the base model for Image to create a repository. Tweak settings.
• Migrate custom repository tables
• Set an admin
• Create an 'image_filters.py' file in the app, then subclass a few filters
• Insert template tags into relevant templates

Quickstart

Dependencies

Pillow,

pip install pillow

Pillow

Optional

To use Wand filters, on Debian-based distros,

sudo apt-get install libmagickwand-dev

Then,

pip install wand

Install

PyPi,

pip install django-imagelite

Or download the app code to Django.

Declare in Django settings,

    INSTALLED_APPS = [
        ...
        'image.apps.ImageLiteConfig',
    ]

Migrate,

./manage.py makemigrations image_lite
./manage.py migrate image_lite

Now you need to declare a repository. Further examples assume the example repository created there.

Make some filters

In the repository, create a file image_filters.py. Paste this into it,

from image_lite import filters_pillow, register

@register()
class Thumb(filters_pillow.Resize):
    width=64
    height=64
    format='png'

Change as you wish.

Upload some images

In Django admin, upload a few images.

I don't know about you, but if I have a new app I like to try with real data. If you have a collection of test images somewhere, try this management command,

./manage.py image_create_bulk news_article_images.NewsArticleImage pathToMyDirectory

Note you need to give a path to a Model. You can create, meaning upload and register, fifteen or twenty images in a few seconds.

View some images

Ok, let's see an image. Find a web view template. Nearly any template will do (maybe not a JSON REST interface, something visible).

Add this tag to the template,

{% load img_tags %}
...
{% image_fixed news_images.NewsImage riverbank-enforcement Thumb %}

'news_images.NewsImage' is the path to the repository model. 'riverbank-enforcement' is the filename. 'Thumb' is the filter we made earlier.

Visit the page. You should see a thumbnail of the 'riverbank-enforcement' image.

QuickStop

Don't like what you see?

• Remove any temporary code.
• Migrate backwards ('./manage.py migrate imagelite zero')
• Remove from 'apps.py'
• Remove the two media directories. Defaults are '/media/originals/' and '/media/reforms/'
• Remove the app folder, or uninstall

That's it, gone.

Full documentation

Index,

• Image Repository Models
• Model Fields
• Auto Delete
• Filters
• Admin
• Rendering
• Management Commands

Installation

For most users, PiPy. The ImageLite core has no migrations, users build custom models for image repositories. However, you must declare the app core in settings, otherwise the app will not be able to find image filters, so you'll get a stream of nasty ''module not found' errors,

    INSTALLED_APPS = [
        ...
        'image.apps.ImageLiteConfig',
    ]

Also, you may want to cross-check or declare in settings.py where images will go,

MEDIA_ROOT = BASE_DIR / 'files_upload'
MEDIA_URL = '/images/'

And there are a couple of configuration options for searching for filters (SEARCH_APP_DIRS=True is default, anyway),

IMAGES = [
    {
        'SEARCH_APP_DIRS': True,
        'SEARCH_MODULES': [
        ],
    },
]

Image Repository models

Overview

New repositories can be made by subclassing the the core model. Reasons you may want to customise repositories,

Repository behaviour

Custom repositories have new DB tables, and can operate with new configurations such as storing files in different directories, auto-deleting original files, not limiting filename sizes and more.

Associate data with images

You may want to associate data with an image. Many people's first thought would be to add a title (ImageLite does not provide titles by default). But other kinds of information can be attached to an image such as captions, credits, dates, and/or data for semantic/SEO rendering.

Split needs

For example, you may want an image repository attached to a main Article model, but also an image pool for general site use such as banners or icons.

Subclassing AbstractImage

Custom Image repository code is placed in 'models.py' files and migrated. You decide how you want your namespacing to work. The code can be placed in an app handling one kind of model or, for general use, in a separate app. Or you can have one seperate app, called for example ''site_images'. handling several repositories (I'm fond of this level of encapsulation).

For a separate app,

./manage.py startapp news_article_images

Declare the app in settings.py,

    INSTALLED_APPS = [
        ...
        'image.apps.ImageConfig',
        'news_article_images.apps.NewsArticleImagesConfig',
    ]

Here is a minimal subclass. In a 'models.py' file, do this,

from django.db import models
from image_lite.models import AbstractImage


class NewsArticleImage(AbstractImage):
    upload_dir='news_originals'
    reform_dir='news_reforms'

    # AbstractImage has a file and upload_date
    # but add whatever new fields you want e.g.
    caption = models.CharField(_('Caption'),
        max_length=255,
    )

    author = models.CharField(_('Author'),
        max_length=255,
        db_index=True
    )

    etc.

Migrate,

./manage.py makemigrations news_article_images
./manage.py migrate news_article_images

You now have a new image upload app. It has it's own DB table. You can change it's configuration (see next section).

Attributes

Subclasses accept some attributes. Note that some of these settings are radical alterations to a model class. To be sure a model setting will take effect, it is best to migrate the class.

An expanded version of the above,

from image.models import AbstractImage, AbstractReform

class NewsArticleImage(AbstractImage):
    upload_dir='news_originals'
    filepath_length=55
    form_limit_filepath_length=True
    accept_formats = ['png']
    max_upload_size=2
    auto_delete_upload_file=True
    filters=[]
    reform_dir='news_reforms'
    ...

I hope what these attributes do is easy to understand. None of them are available through standard Django or, not in this simple way.

The 'filters' attribute may need a little explanation. You provide an 'image_filters' file with each app, which implements image-lite. By default, the code will apply all the 'image_filters' to any uploaded file. This is awkward if there is two or more repositories in one app, because they all apply the same filters. In this case, you can set filters to a list of filter names, which will be the only filters used for that model/repository.

Multiple filters will generate subdirectories for filters after the first. The subdirectories are named from a lowercase version of the filter name.

Since they are easy to create, I usually have many image repositories in every project, Each one usually has a single general filter, such as a ResizeFill and watermark. Further filters, generating subdirectories are there for format variations, like thumbnails. The repositories are categorising for me, and I have no need of further namespacing.

Migrate, and you are up and running.

Blocking file renaming

It is impossible to have two files in the same directory with the same name (this applies also to remote storage). The default behaviour of Django uploading is to enable uploads if possible, so the standard FileSystemStorage module will rename duplicate filenames. It does this by adding a pseudo-random sequence of codepoints to the filename.

For the general outlook of ImageLite, this is not always what is wanted. If a user has uploaded an image called 'Spin_X9000CA', a later attempt to add a 'Spin_X9000CA' image should be blocked, or replace the first image (standard behaviour would be to generate a new filename such as 'Spin_X9000CA_WEN42HDOE5').

Only you can decide if blocking is more appropriate than renaming. There is a builtin way to block duplicate filenames, using a mixin,

...
from image_lite import ModelUniqueFilenameMixin

    class RevImage(ModelUniqueFilenameMixin, AbstractImage):
        ...

If an image repository uses this mixin, attempts to upload duplicate filenames cause a ValidationError. The user sees a message 'filename already exists'.

Inheritance! Can I build repositories using OOP techniques?

No! Python has been cautious about this kind of programming, and Django's solutions are a workround. Do not stack models or you will create unusable migrations.

Can I create different repositories, then point them at the same storage paths?

No, you risk duplicate entries.

Things to consider when subclassing models

Auto delete of files

Good to decide a deletion policy from the start. See Auto Delete

Add Meta information

You may want to configure a Meta class. If you added titles or slugs, for example, you may be interested in making them into unique constrained groups, or adding indexes,

class NewssArticleImage(AbstractImage):
    upload_dir='news_originals'
    filepath_length=100

    etc.

    class Meta:
        verbose_name = _('news_image')
        verbose_name_plural = _('news_images')
        indexes = [
            models.Index(fields=['author']),
        ]
        constraints = [
            models.UniqueConstraint(
                fields=['title', 'author'], 
                name='unique_newsarticle_reform_src'
            )
        indexes = [
            models.Index(fields=['upload_time']),
        ]

Note that the base Image does not apply an index to upload_time, so if you want that, you must specify it.

Auto-delete

Overview

I read somewhere that a long time ago, Django auto-deleted files. This probably happened in model fields. This behaviour is not true now. If objects and fields are deleted, files are left in the host system. However, it suits this application, and some of it's intended uses, to auto-delete files.

ImageLite deletes from the 'post-delete' signal. That means it will remove the DB record, then attempt file deletion. If file-deletion fails, it will leave files orphaned. For the purpose of ImageLite, this is seen to be preferable to the alternative, which would block admin forms because file deletion failed.

Reforms

ImageLite treats reforms as cache. They can expire, be created, moved and deleted as necessary. So, when an image is deleted, ImageLite always attempts to delete the reforms. It may not succeed, because if you have rebuilt filters or changed filter settings it will not know where to look. But ImageLite will try. If it fails, it fails silently. ImageLite works from a signal, so will always try, for bulk deletes also.

Auto-delete Image files

To auto-delete, set the Image model attribute 'auto_delete_upload_file=True'. This action is triggered by a signal, so will work for bulk deletes also. The code will then attempt to remove both the original image and reforms.

Filters

Overview

Filters are used to describe how an uploaded image should be modified for display. On upload, the app will automatically generate reformed images.

You can use the provided filters. but if you want to pass some time with image-processing code, you can add filters to generate ''PuddingColour' and other effects.

Filter placement and registration

Files of filter definitions should be placed in a repository app. Create a file called 'image_filters.py' and off you go.

Base filters

The filter code is a stack of inherited classes. There are some base filters, which you can configure. These are centre-anchored, If you only need different image sizes, you only need to configure these.. Smpling is usually default BILINEAR (because choosing the filter for the material is more important. If you must, make your own filter).

ResizeForce

Reshape the image to the given size.

Crop

Crop within boundaries of a given size. If the image is small, nothing happens, which may leave gaps.

Resize

Resize within boundaries of a given size. Aspect ratio preserved, which may leave gaps.

CropFill

Crop within boundaries of a given size. If the image is small, space filled with given background color.

ResizeFill

Resize within boundaries of a given size. Aspect ratio preserved, space filled with given background color.

Filter declarations

All builtin filter bases accept these attributes,

• width
• height
• format

Most filter code demands 'width' and 'height' and 'format'.'format' defaults to 'jpg'. Formats accepted are conservative,

bmp, gif, ico, jpg, png, rgb, tiff, webp 

which should be written as above (lowercase, and 'jpg', not 'jpeg'). So,

from image import Resize, registry

class MediumImage(Resize)
    width=260
    height=350
    format='png'
    #fill_color="Coral"
    #jpeg_quality=28
    # optional effects

registry.register(MediumImage)

'Fill' filters

Crop and Resize will preserve aspect ratio. This I think is desired behaviour in many situations. However, it will leave gaps if the aspect ratio of the original does not match that given to reshape to.

Sometimes it may be preferable that the image become the exact size given. One way is to distort the image ('Force' filters). Another way is the Fill filters. These make the image fit bounding dimensions, then fill surplus area with a fill colour,

from image import ResizeFill, registry

class MediumImage(ResizeFill):
    width=260
    height=350
    format='jpg'
    fill_color="Coral"

registry.register(MediumImage)

Fill color is defined however the image library handles it. Both Pillow and Wand can handle CSS style hex e.g. '#00FF00' (green), and HTML colour-names e.g. 'AliceWhite'.

Registering filters

Filters need to be registered. Registration style is like ModelAdmin, templates etc. Registration is to 'image.registry' (this is how templatetags find them).

You can use an explicit declaration,

from image_lite import ResizeFill, registry

...

registry.register(single_or_list_of_filters)

Or use the decorator,

from image_lite import register, ResizeFill

@register()
class MediumImage(ResizeFill):
    width=260
    height=350
    format='jpg'
    fill_color="Coral"

Pillow filter attributes

The base filters in the Pillow filter set have an attribute available. You gain,

from image import filters_pillow, register

@register()
class Medium(filters_pillow.ResizeFill):
    width=260
    height=350
    format='jpg'
    anchor=('top', 'right', 'image/logo-inset.png')

''anchor' lets you overlay the reform image with another (fixed) image (presumed smaller?). A kind of rubber-stamp? The overlay image can be set into any corner of the reform image. The overlay image path is currently a Python filepath (sorry, I'm not helping much, am I? Sometime, maybe).

Wand filter attributes

The base filters in the Wand filter set have more attributes available. The 'wand' code needs Wand to be installed on the host computer. Assuming that, you gain these effects,

from image import filters_wand, register

@register()
class Medium(filters_wand.ResizeFill):
    width=260
    height=350
    format='jpg'
    pop=False
    greyscale=False
    night=False
    warm=False
    strong=False
    no=False
    watermark='image/watermark.png'

If you enable more than one effect, they will chain, though you have no control over order.

I lost my way with the Wand effects. There is no 'blur', no 'rotate', no 'waves'. But there is,

pop

Tightens leveling of black and white

greyscale

A fast imitation

night

Pretend the picture is from a movie

warm

A small shift in hue to compensate for a common photography white-balance issue.

strong

Oversaturate image colors (like everyone does on the web). Unlike 'pop' this will not stress contrast so flatten blacks and whites. You may or may not prefer this.

no

Draw a red cross over the image

watermark

Accepts a URL to a watermark image template.

Watermark deserves some explanation. This does not draw on the image, as text metrics are tricky to handle. You configure a URL stub to an image, here's a builtin,

watermark = 'image/watermark.png'

The URL is Django static-aware, but will pass untouched if you give it a web-scheme URL (like the URLs in Django Media). The template is scaled to the image-to-be-watermarked, then composited over the main image by 'dissolve'. So the watermark is customisable, can be used on most sizes of image, and is usually readable since aspect ratio is preserved.

It is probably worth saying again that you can not change the parameters, so the strengths of these effects, without creating a new filter.

Writing custom filter code

First bear in mind that Image uses fixed parameters. So your filter must work with fixed parameters across a broad range of uploaded images. I don't want anyone to dive into code, put in hours of work, then ask me how they can create an online image-editing app. Not going to happen.

However, while I can't make a case for 'waves' or 'pudding-colour' filters, I can see uses. For example, Wagtail CMS uses the OpenCV library to generate images that auto-focus on facial imagery (i.e. not centrally crop). There are uses for that.

Second, bear in mind that image editing is lunging into another world, rather like creating Django Forms without using models and classes. It will take time. But there is help available. Inherit from 'image.Filter'. You will need to provide a 'process' method, which takes an open Python File and returns a ByteBufferIO and a file extension.

If you want the filter to work with the Pillow or Wand libraries, you can inherit from the PillowMixin or WandMixin. These cover filehandling for those libraries. Then you can provide a 'modify' method, which alters then returns an image in the format of those libraries.

See the code for details.

Why can filters not be chained or given parameters?

Because it makes life easy for coders and users. If you want to produce a front-end that can adjust filters, or chain them, that is another step. This is not that app.

Admin

Overview

Repository models start with stock Django admin. However, this is not always suited to the app, it's intended or possible uses. So there is a pre-configured admin.

ImageLiteAdmin

Significant changes from stock admin,

• changelist is tidier and includes filenames not paths
• changelist includes 'view' and 'delete' links
• changelist has searchable filenames

Easy as this,

from django.contrib import admin
from image_lite.admins import ImageLiteAdmin
from news_images.models import NewsImage


# Register your models here.
class NewsImageAdmin(ImageLiteAdmin):
    pass


admin.site.register(NewsImage, NewsImageAdmin)

Notes and alternatives for ImageLiteAdmin

You may provide no admin at all. You can use the './manage.py' commands to do maintenance. The stock admin is provided to get you started.

If you prefer your own admin, look at the code for ImageLiteAdmin in '/image/admins.py'. It provides some clues about how to do formfield overrides and other customisation. You may find it more maintainable to build new admin code, rather than import and override.

Rendering

Overview

You can render the images in HTML. Since filenaming is controlled, this will often be easy,

<img src="/media/reforms/logo.png" alt="image of logo" class="main-logo">

or use template tags.

Template Tags

Currently, the app has two template tags. Both build a full HTML 'img' tag. Both accept keyword parameters which become HTML attributes.

The 'image' tag

The main tag. It depends on calling a model to construct a URL. So you need a callable on the page model (not the image model). This callable is usually very simple, and must return the entire reform filename. You can make a few callables, one for every image reform you want to use e.g.

class NewsArticle():  
    ...
    def image_main_url(self):
        url = self.title + '-Stock.png' 
        return url

    def image_teaser_url(self):
        url = self.title + '-Teaser.png' 
        return url

Ok, let's call the method from a template, with the image repository, template main variable, and callable name as parameters,

{% load img_tags %}

{% image news_images.NewsImage newsartricle image_main_url class="framed" %}

then visit the page. Should see the image? As HTML it should render something like,

<img src="/media/reforms/eoy-report-Stock.png" alt="image of eoy-report" class="framed">

The 'image-fixed' tag

The above tag is mainly to retrieve images associated with page data. This tag lets you call by filename. Add this to template code, with the image repository, filename, and filtername as parameters,

{% load img_tags %}

{% image_fixed news_images.NewsImage forest_fire Stock class="ruled" %}

then visit the page. Should see the image.

Management Commands

They are,

• image_create_bulk
• image delete_bulk
• image_list
• image_sync
• reform_create
• reform_delete
• reform_list

All management commands must be pointed at subclasses of Image. Several have a common switch -c/--contains, which makes a basic string match. 'image_sync' can be particularly useful for half-broken repositories, it will attempt to make models for orphaned files, or delete orphaned files, or delete models with missing files. Use the -h/--help switch to discover what each command can do.

Tests

I've not found a way to test this app without Python ducktape. There should be tests, but I don't expect them.

Credits

This is a rewrite of the Image app from Wagtail CMS. It is now distant from that app, and would not work in the Wagtail system. However, some core ideas remain, such as the replicable repositories.

Wagtail documentation