insight_reloaded 1.6

A full async docsplit previewer server based on Tornado and Redis

Introduction

We are working to find the best solution for our previewer media server.

The case is simple, we have PDF and we want to preview them (small, medium, large, header).

We first started with Insight which allows people to ask for sync and async previews generations from an url with caching.

The system is able to register engines so that we can manipulate documents on the flow.

The idea of Insight-Reloaded is to remove the sync mode and delegate it to Nginx with disk or S3 storage.

If the document preview is not ready, we want a 404 error.

We will get the document preview url with a callback.

The API

Simple example

curl -X GET "http://localhost:8888/?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg"
{"insight_reloaded": "Job added to queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/status
{"insight_reloaded": "There is 14 job in the queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/
{"version": "1.5", "insight_reloaded": "Bonjour", "name": "insight-reloaded"}

Multi-queues example

You must to configure REDIS_QUEUE_KEYS in your settings.py.

REDIS_QUEUE_KEYS = ['urgent', 'normal']
DEFAULT_REDIS_QUEUE_KEY = 'normal'

Then you can use:

curl -X GET "http://localhost:8888/urgent?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg"
{"insight_reloaded": "Job added to queue 'normal'.", "number_in_queue": 14}

curl -X GET http://localhost:8888/urgent/status
{"insight_reloaded": "There is 14 job in the 'urgent' queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/
{"version": "1.5", "insight_reloaded": "Bonjour", "name": "insight-reloaded"}

Set hash

By default, insight compute preview hash using url (sha1(url)). You can also send it within the request if you want, be sure that each preview has an uniq hash:

curl -X GET "http://localhost:8888/?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg?hash=123456789"
{"insight_reloaded": "Job added to queue.", "number_in_queue": 14}

Service architecture

Server provisioning

You can find insight-reloaded chef cookbooks here : https://github.com/novapost/insight-installer

This will helps you install all requirements to run your insight-reloaded server.