Parallel file copying for Django's collectstatic.
This package extends Django’s collectstatic management command with a --faster argument that activates the parallel file copying. The speed improvement is especially helpful for remote storage backends like S3.
pip install django-collectfaster
Configure installed apps in your settings.py and make sure collectfaster is listed before django.contrib.staticfiles:
INSTALLED_APPS = ( ..., 'collectfaster', 'django.contrib.staticfiles', 'storages', ..., )
If you are using S3 with django-storages you probably already have this configured in your settings.py:
AWS_S3_HOST = 's3-eu-west-1.amazonaws.com' AWS_STORAGE_BUCKET_NAME = '<your_aws_bucket_name>'
Set the storage backends for your static and media files in the settings.py:
STATICFILES_STORAGE = 'collectfaster.backends.boto.S3StaticStorage' DEFAULT_FILE_STORAGE = 'collectfaster.backends.boto.S3MediaStorage' # STATICFILES_STORAGE = 'collectfaster.backends.boto3.S3Boto3StaticStorage' # DEFAULT_FILE_STORAGE = 'collectfaster.backends.boto3.S3Boto3MediaStorage'
You should split your static and media files on your S3 in different folders and configure it in the settings.py:
STATICFILES_LOCATION = 'static' MEDIAFILES_LOCATION = 'media'
Set the STATIC_URL at least on your production settings:
STATIC_URL = 'https://%s/%s/%s/' % (AWS_S3_HOST, AWS_STORAGE_BUCKET_NAME, STATICFILES_LOCATION)
Collect your static files parallel:
python manage.py collectstatic --faster
Set the amount of workers to 30:
python manage.py collectstatic --faster --workers=30
Spawn workers using multiprocessing instead of gevent:
python manage.py collectstatic --faster --use-multiprocessing