Skip to main content
Build status

This package was created because at my work, Language and Translation Technology Team at the University of Ghent, we often create demos on the web that allow the user to input and process text or files. These texts are then processed by other scripts that expect clean UTF-8-texts. This library extends the Django FileField, CharField and TextField by checking if the content of a submitted file or text is clean. If not, it generates an error. Checks are executed for four byte long characters and NULL characters.

Requirements

Django >= 1.8

Installation

pip install django-utf8field

Usage

Add the app to your settings:

INSTALLED_APPS = (
    ...
    'utf8field',
    ...

FileField

Create a model like you would do normally, but instead of using FileField you use UTF8FileField:

from django.db import models
from utf8field.fields import UTF8FileField

class YourModel(models.Model):
    title = models.CharField(max_length=255)
    created_on = models.DateTimeField(auto_add_on=True)
    text = models.UTF8FileField()

You also have the option to provide the option max_content_length to limit the number of characters in the file. If the content is longer an error will be displayed. If you want to enable four_byte_detection set the parameter to True.

text = models.UTF8FileField(max_content_length=1000, four_byte_detection=True)

CharField

Create a model like you would do normally, but instead of using CharField you use UTF8CharField. If you want to enable four_byte_detection set the parameter to True.

from django.db import models
from utf8field.fields import UTF8CharField

class YourModel(models.Model):
    title = models.CharField(max_length=255, four_byte_detection=True)
    created_on = models.DateTimeField(auto_add_on=True)
    text = models.UTF8CharField(max_length=1000)

TextField

Create a model like you would do normally, but instead of using TextField you use UTF8TextField. If you want to enable four_byte_detection set the parameter to True.

from django.db import models
from utf8field.fields import UTF8TextField

class YourModel(models.Model):
    title = models.CharField(max_length=255)
    created_on = models.DateTimeField(auto_add_on=True)
    text = models.UTF8TextField(four_byte_detection=True)

Django Rest Framework

The necessary serializers and automatic mapping of fields is provided so you should not be doing anything yourself to get the texts or files validated when using a ModelSerializer.

Development

To run the tests make sure Django, Django Rest Framework and coverage are installed (pip install django djangorestframework coverage) and execute

python manage.py test

To create extra translations, execute

pm makemessages --locale=nl --extension=py --ignore=dev_example --ignore=build

… and modify the resulting django.po file in utf8field/locale/nl/LC_MESSAGES.

Metadata

Release files for django-utf8field 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for django-utf8field 1.0.0
File Size Uploaded
django-utf8field-1.0.0.tar.gz 4.3 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for django-utf8field 1.0.0
File Interpreter ABI Platform
django_utf8field-1.0.0-py2.py3-none-any.whl Python 2, Python 3 none any Details

Total release size: 12.0 kB

Release files / django-utf8field-1.0.0.tar.gz

Download URL django-utf8field-1.0.0.tar.gz
Size 4.3 kB
Tags Source
SHA-256 checksum
How to use checksums
a81e9b2c0c9615ecd31f7324513de8ed6a5b3a2f1f49b3f93cdd1931d29ae383
BLAKE2b-256 checksum
How to use checksums
9b1b78e1e4ae49ef0ec874c03654a751753764513a101217812865a739d93af8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / django_utf8field-1.0.0-py2.py3-none-any.whl

Download URL django_utf8field-1.0.0-py2.py3-none-any.whl
Size 7.7 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
01de6b36e7e3a576f2b11ea24bcdf903804464828d6a6b8a9e76d879c2b2041f
BLAKE2b-256 checksum
How to use checksums
713875bd38e6ba8eaa7a3c8d50336d2c29f8d0f5b81529207f4d8ab971a654a1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 release files

0.4.1

2 release files

0.4.0

2 release files

0.3.2

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.0

2 release files

0.1.0

2 release files

0.0.1

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page