Skip to main content

DJango File Metadata indexer

Use this app to search for local files and index some metadata information and also the content of the file (text) in the database (FileMetadata model). Multiple directories can be configured to be indexed (settings.FILEMETADATA_LOOKUP_DIRS). With the information registered in the DB it is possible to use the features of django (filters, export, etc.) or other apps to manipulate the data. This app can be used, for example, as a basis for implementing protected download pages or for searching the content of files in the search tool.

This version is supported on Python 3.6+; and Django 2.2+.

Installation

Install the package with pip:

$ pip install django-filemetadata

Add the App in the INSTALLED_APPS

INSTALLED_APPS=(
    "django-filemetadata",
),

Execute makemigrations/migrate.

Configuration

Configure the directories to look for the files in the settings:

FILEMETADATA_LOOKUP_DIRS=[‘/folder1/folder2’, ‘/folder3/folder4’]

Utilization

Index the data with the management command

usage:  filemetadata_index [-f FOLDERS] [-c] [-d] [-s] [-x] [-n] [-a]

Update the the file-metadata found in the directories into the DB.

optional arguments:
  -f FOLDERS            Folder(s) to index (coma separated)
  -c                    Clear the data before reindex
  -d                    Delete only the data from these folders and exit
  -s                    Index the symlinks (Do not follow it)
  -x                    Extract the content of the file (text)
  -n                    Non-reentrant mode (Not recursive)
  -a                    Abort on errors

e.g.

Reindex the files configured in settings

python filemetadata_index

or inform the directories

python filemetadata_index -f /folder1/folder2,/folder3

Or just delete the data from these folders (not recursive in this case):

python filemetadata_index -d -n -f /folder1/folder2,/folder3

Go to Admin and check the data in the FileMetadata model.

Customization

Support for .pdf files This app is compatible with the ‘PyPDF4’ library. If it is installed it can be used to extract the content from pdf files if necessary.

Custom extractor It is possible to override the function that extracts the contents of the files by a more specific one if necessary. To do this, overload the function ‘func_extract_text’ in the indexer.py module

from filemetadata import indexer

def my_extractor(posixpath_obj):
    ...
    return file_content

indexer.func_extract_text = my_extractor

or the extract_text method of the FileIndexer class

from filemetadata.indexer import FileIndexer

class MyFileIndexer(FileIndexer):
  def extract_text(self, file_obj):
    ...
    return file_content

Tests

To run the tests

python load_tests.py

Metadata

Release files for django-filemetadata 1.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for django-filemetadata 1.0.4
File Size Uploaded Provenance
django-filemetadata-1.0.4.tar.gz 17.6 kB Without provenance Details

Release files / django-filemetadata-1.0.4.tar.gz

Download URL django-filemetadata-1.0.4.tar.gz
Size 17.6 kB
Tags Source
SHA-256 checksum
How to use checksums
15590f964f94b308aa89ab75acf7a0dc722ffce05c00c19e9c9c1d717c0af593
BLAKE2b-256 checksum
How to use checksums
207ac714e772b9e1eebf1a77fa3edef29c3091170aae9eb9f14b61192ab8ad7e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.8.0 tqdm/4.30.0 CPython/3.8.5

Security for django-filemetadata 1.0.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page