Skip to main content

Skeleton Plugin for VAIS analytic system

Project description

Skeleton Plugin

Plugin skeleton V1

Flow handle job

  • Step 1: Register worker with backend (by env BACKEND_ENDPOINT) with some information (JOB_QUEUE, PLUGIN_CODE, PLUGIN_NAME, PLUGIN_DESCRIPTION, VERSION_CODE)
  • Step 2: Connect to Redis, Elastic search (via env REDIS_HOST, REDIS_PORT, ELASTICSEARCH_URL)
  • Step 3: Listen redis queue job (JOB_QUEUE)
  • Step 4: Handle job

Note: Using file rq_push_job.py for testing push job

Custom an plugin

  • Step 1: Inheritance class PluginSkeleton and overwrite function preload for load model if need (in sample_plugin.py).
  • Step 2: Custom function handle_job() inside file sample_plugin.py (keep input parameters)

Check list migrate skeleton plugin

  • News footprint handle job function: handle_job(doc_index, doc_id, input_data=None, depended_jobs=None)
  • Get input data from depended jod: input_data = SamplePlugin.get_depended_jobs_output(current_job, input_data, depended_jobs)
  • Input data have new structure, so need to change the way to parse data from input data
{
        "plugin_code":{
            "data": {},
            "version_code": {}
        },
        "plugin_code":{
            "data": {},
            "version_code": {}
        },
        ...
    }
  • Change in func push result. Now doc_index come before doc_id and need to input version code: PluginSkeleton.push_result(doc_index, doc_id, output, KeywordPlugin.VERSION_CODE)

  • output parameter in push_result func is just content in "data" field

  • "return" in an plugin now need to using this function: plugin_skeleton.PluginSkeleton.get_output_result(output, KeywordPlugin.VERSION_CODE)

  • In docker-compose.yml file need include DEPENDED_PLUGIN_CODE envs (e.g. 'asr,speech_cluster'). It's list of depended plugin code, separate with a comma

  • build lib

python setup.py sdist bdist_wheel
twine upload dist/*

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for vais-plug, version 0.0.9
Filename, size File type Python version Upload date Hashes
Filename, size vais_plug-0.0.9-py3-none-any.whl (6.5 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size vais-plug-0.0.9.tar.gz (5.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page