Skip to main content

Add UTF-8 Validation to Django FileFields, CharFields and TextFields

Project description

Build status

This package was created because at my work, Language and Translation Technology Team at the University of Ghent, we often create demos on the web that allow the user to input and process text or files. These texts are then processed by other scripts that expect clean UTF-8-texts. This library extends the Django FileField, CharField and TextField by checking if the content of a submitted file or text is clean. If not, it generates an error. Checks are executed for four byte long characters and NULL characters.

Requirements

Django >= 1.8

Installation

pip install django-utf8field

Usage

Add the app to your settings:

INSTALLED_APPS = (
    ...
    'utf8field',
    ...

FileField

Create a model like you would do normally, but instead of using FileField you use UTF8FileField:

from django.db import models
from utf8field.fields import UTF8FileField

class YourModel(models.Model):
    title = models.CharField(max_length=255)
    created_on = models.DateTimeField(auto_add_on=True)
    text = models.UTF8FileField()

You also have the option to provide the option max_content_length to limit the number of characters in the file. If the content is longer an error will be displayed. If you want to enable four_byte_detection set the parameter to True.

text = models.UTF8FileField(max_content_length=1000, four_byte_detection=True)

CharField

Create a model like you would do normally, but instead of using CharField you use UTF8CharField. If you want to enable four_byte_detection set the parameter to True.

from django.db import models
from utf8field.fields import UTF8CharField

class YourModel(models.Model):
    title = models.CharField(max_length=255, four_byte_detection=True)
    created_on = models.DateTimeField(auto_add_on=True)
    text = models.UTF8CharField(max_length=1000)

TextField

Create a model like you would do normally, but instead of using TextField you use UTF8TextField. If you want to enable four_byte_detection set the parameter to True.

from django.db import models
from utf8field.fields import UTF8TextField

class YourModel(models.Model):
    title = models.CharField(max_length=255)
    created_on = models.DateTimeField(auto_add_on=True)
    text = models.UTF8TextField(four_byte_detection=True)

Django Rest Framework

The necessary serializers and automatic mapping of fields is provided so you should not be doing anything yourself to get the texts or files validated when using a ModelSerializer.

Development

To run the tests make sure Django, Django Rest Framework and coverage are installed (pip install django djangorestframework coverage) and execute

python manage.py test

To create extra translations, execute

pm makemessages --locale=nl --extension=py --ignore=dev_example --ignore=build

… and modify the resulting django.po file in utf8field/locale/nl/LC_MESSAGES.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for django-utf8field, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size django_utf8field-1.0.0-py2.py3-none-any.whl (7.7 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size django-utf8field-1.0.0.tar.gz (4.3 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page