Skip to main content

A file catalog with an HTTP frontend.

Project description

Filecat
=======

This package provides application support for indexing and serving
content in a file system.

An HTTP API is available for querying operations.

Setup
-----

The Xapian database queue processor must be configured with an
indexer which points to a database file on disk, e.g.::

>>> indexer = repoze.filecat.index.create_indexer(db_path)
>>> ore.xapian.queue.QueueProcessor.start(indexer)

To start the directory observer (configured to scan every 3 seconds)::

>>> repoze.filecat.watch.start(directory, 3)

We can then query the Xapian database by opening a connection::

>>> connection = xapian.Connection(db_path)

Formats
-------

Currently supported file formats:

- JPEG (image/jpeg)
- ReStructuredText (text/x-rst)

Authors
-------

Malthe Borch <mborch@gmail.com>
Stefan Eletzhofer <stefan.eletzhofer@inquant.de>
Robert Marianski <rmarianski@gmail.com>

Application
===========

The filecat server application is configured with a ``path`` parameter
which points to a pool of files (a directory structure) and a ``host``
parameter which provides a URL at which a webserver is configured to
serve up the directory as static files.

API
---

All queries are performed using the following HTTP API::

method path description
----------------------------------------------------------
POST /purge Clear index
POST /add Add file to index

@path Relative filename

POST /update Reindex file

@path Relative filename

POST /remove Remove file from index

@path Relative filename

GET /query Perform query

Parameters are passed as-is
to the Xapian query engine.

Results
-------

Queries return a JSON structure which is first a two-tuple
(matches_estimated, results). The result set is a tuple of dicts:

@url The URL where this file can be downloaded
@mimetype MIME-type of the file
@metadata Dict-like structure with mimetype-specific metadata*





Changelog
=========

0.2 - (2009-08-03)
~~~~~~~~~~~~~~~~~~

- The JSON-output of a query is now a tuple of estimated number of
matches and the result set. [malthe]

- Fixed issue where global registry would not be properly
configured. [malthe]

- Handle JPEG-files without a XMP application segment (these are
indexed with a trival metadata tree). [malthe]

- Use XMP instead of IPCT/EXIF; this sheds the dependency on the
Hachoir library. [malthe]

- Added XMP extraction capabilities for JPEG-files. [malthe]

- Updated Xapian library versions in buildout configuration. [malthe]

0.1 - (2008-10-03)
~~~~~~~~~~~~~~~~~~

- Remove print statements. [seletz]

- Added tests for directory watch/observer. Fixed a bug where the observer would
emit absolute paths causing xapian to index absolute paths. [seletz]

- Hook watcher in. The configured ``static`` directory is now watched. Changes
are now added to the Xapian queue. [seletz]

- Created a directory observer thing which will allow us later to atomatically
wahtch a directory and issue Xapian operations. [seletz]

- Hooked in ``modify`` and ``remove`` views. [seletz]

- Changed query parser -- it now just uses the passed-in query string. Also
added ability to limit/batch query results. [seletz]

- Configured Xapian indexer. This has to be done separately from the BFG
components, because the Xapian indexer runs in its own thread and thus uses
the global component registry. [seletz]

- Hooked up ``add`` view. [seletz]

- Implemented ``query`` view. [seletz]

- Fixed ``routes``setup wrt. permissions. [seletz]

- Changed svn layout. [seletz]

- Added buildout. [seletz]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
repoze.filecat-0.2.tar.gz (68.3 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page