Skip to main content

Meta inquiry middleware for distributed systems

Project description

jomiel

pypi-pyversions pypi-v pypi-wheel pypi-status code-style

jomiel is the meta inquiry middleware for distributed systems. It returns data about content on video-sharing websites (e.g. YouTube). Two technologies form the basis for jomiel:

  • ZeroMQ (also known as ØMQ, 0MQ, or zmq) looks like an embeddable networking library but acts like a concurrency framework

  • Protocol Buffers is a language-neutral, platform-neutral, extensible mechanism for serializing structured data

jomiel is a spiritual successor to (now defunct) libquvi.

Example: jomiel and yomiel working together

Features

  • Language and platform neutral. It communicates using Protocol Buffers and ZeroMQ. There are plenty of client demos. Pick your favorite language.

  • Secure. It can authenticate and encrypt connections using CURVE and SSH.

  • Extensible. It has a plugin architecture.

Getting started

pypi-pyversions

Install from PyPI:

pip install jomiel

Install from the repository, e.g. for development:

git clone https://github.com/guendto/jomiel.git
cd jomiel
pip install -e .  # Install a project in editable mode

Or, if you'd rather not install in "editable mode":

pip install git+https://github.com/guendto/jomiel

Try sending inquiries to jomiel with:

  • the client demos written in different modern programming languages
  • yomiel - the pretty printer for jomiel messages

Be sure to check out:

Usage

usage: jomiel [-h] [--version] [-v] [--config-file FILE] [-D] [-E] [-P]
              [--logger-config FILE] [-L] [--logger-idents-verbose] [-l IDENT]
              [-p] [-m] [--debug-sensitive] [-F] [--http-user-agent STRING]
              [--http-timeout TIME] [--http-debug] [-I] [-r ADDR] [-d ADDR]
              [-w [1-64]] [--curve-enable] [--curve-public-key-dir DIR]
              [--curve-server-key-file FILE] [--curve-domain DOMAIN]
              [--curve-allow ADDR]

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v, --version-long    show version information about program's environment
                        and exit (default: False)
  --config-file FILE    Read configuration from the specified file [env var:
                        CONFIG_FILE] (default: None)
  -D, --print-config    Show the configuration values and exit (default:
                        False)
  -E, --report-config   Report keys, values and where they were set (default:
                        False)
  -P, --config-paths    Print default configuration file paths (default:
                        False)
  -p, --plugin-list     Display the found plugins and exit (default: False)

logger:
  --logger-config FILE  Logger configuration file to read [env var:
                        LOGGER_CONFIG] (default: None)
  -L, --logger-idents   Print logger identities and exit (default: False)
  --logger-idents-verbose
                        Print logger identities in detail, use together with
                        --logger-idents (default: False)
  -l IDENT, --logger-ident IDENT
                        Use the logger identity [env var: LOGGER_IDENT]
                        (default: default)

debug:
  -m, --debug-minify-json
                        Minify JSON messages in the logger (default: False)
  --debug-sensitive     Log sensitive data, e.g. input URIs, serialized
                        messages (default: False)

http:
  -F, --http-allow-redirects
                        Follow HTTP redirections (default: False)
  --http-user-agent STRING
                        Identify as STRING to the HTTP server (default:
                        Mozilla/5.0)
  --http-timeout TIME   Time in seconds allowed for the connection to the HTTP
                        server to take (default: 5)
  --http-debug          Enable verbose HTTP output (default: False)

broker:
  -I, --broker-input-allow-any
                        Disable input URI validation (default: False)
  -r ADDR, --broker-router-endpoint ADDR
                        Bind the frontend (router) socket to the local
                        endpoint [env var: BROKER_ROUTER_ENDPOINT] (default:
                        tcp://*:5514)
  -d ADDR, --broker-dealer-endpoint ADDR
                        Bind the backend (dealer) socket to the local endpoint
                        [env var: BROKER_DEALER_ENDPOINT] (default:
                        inproc://workers)
  -w [1-64], --broker-worker-threads [1-64]
                        Number of worker threads in the pool waiting for
                        client connections (default: 5)

curve:
  --curve-enable        Enable CURVE support (default: False)
  --curve-public-key-dir DIR
                        Directory that holds all public client key files
                        (default: .curve/)
  --curve-server-key-file FILE
                        Secret CURVE key file for the server (default:
                        .curve/server.key_secret)
  --curve-domain DOMAIN
                        Configure CURVE authentication for a given domain
                        (default: *)
  --curve-allow ADDR    Allow (whitelist IP addresses) (default: 127.0.0.1)

 If an arg is specified in more than one place, then commandline values
override environment variables which override defaults.

Website coverage

jomiel --plugin-list  # The current coverage is very limited

See the src/jomiel/plugin/ directory for the existing plugins. The plugin architecture is extensible. When you are contributing new plugins, make sure that the website is not:

  • dedicated to copyright infringement (whether they host the media or only link to it)

  • NSFW

License

jomiel is licensed under the Apache License version 2.0.

Acknowledgements

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jomiel-1.2.0.tar.gz (39.6 kB view hashes)

Uploaded Source

Built Distribution

jomiel-1.2.0-py3-none-any.whl (35.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page