django-hadoop-storage 0.1.3

Module for use hadoop file system (hdfs) as django storage

Django hadoop storage provide python classes/functions to use hadoop file system (hdfs) with django storage.

Based on libhdfs official hadoop python connector and pyarrow library.

Requirements

• Install java-1.8.0-openjdk-devel, In centos 7

  sudo yum install -y java-1.8.0-openjdk-devel

In Ubuntu

sudo apt-get install openjdk-8-jdk

• Install hadoop

Settings

For use hdfs as django storage set:

HADOOP_HOST = 'localhost'  # set your own host
HADOOP_PORT = 8020  # set your own port
HADOOP_USER = 'hadoop'  # set your own user
HADOOP_HOME = '/usr/lib/hadoop/'  # set your own path

MEDIA_ROOT = '/media/'
DEFAULT_FILE_STORAGE = 'hadoop_storage.HadoopStorage'

Storage

Use hadoop file storage with same methods and properties that django File Storage