Skip to main content

A job reader / writer to define mappings for impulsare.

Project description



A jobs manager, specific to impulsare. It reads, write and delete jobs from an sqlite db.

See tests/static/ for examples of configuration.

Installation / Usage

To install use pip:

$ pip install --upgrade impulsare-job


You need to create a configuration file that contains:

    db: /tmp/test.db # required



impulsare/job implements a writer to :

  • Create / Update jobs (save())
  • Delete jobs (delete())
  • Add / Remove Hooks (hooks_writer.add_hook() and hooks_writer.del_hook())
  • Add / Remove Fields (fields_writer.add_field() and fields_writer.del_field())
  • Add / Remove Rules related to Fields (fields_writer.add_rule() and fields_writer.del_rule())


And a Reader to :

  • Get a Job
  • Get related hooks
  • Get related fields + their rules

Properties of a job

    'name': str, # required
    'active': bool, # default : True
    'description': str,
    'priority': int, # default : 1
    'input': str, # required
    'input_parameters': dict,
    'output': str, # required
    'output_parameters': dict,
    'mode': str # c (create), u (update), cu (create/update), d (delete). Default: c


Create a simple Job (no hooks / rules)

from impulsare_job import Writer

writer = Writer('/etc/impulsare/config.yml')
writer.set_prop('name', 'My Job')
writer.set_prop('input', 'csv')
writer.set_prop('input_parameters', {'delimiter': 'csv'})
writer.set_prop('output', 'sql')
writer.set_prop('output_parameters', {'db': 'test'})
job =

Update a Job

from impulsare_job import Writer

# Lets assume the job id = 1
writer = Writer('/etc/impulsare/config.yml', 'My Job')
job = writer.get_job()
# Output: 'My Job'

# Set the job to Inactive
writer.set_prop('active', False)

Verify if a hook exists, else add it

# .... continuation of code above
if not writer.hooks_writer.hook_exists('test'):
    writer.hooks_writer.add_hook(name='upload_file', method='upload_file', when='after_process')

Allowed properties for hooks:

    'name': str, # required
    'method': str, # required
    'when': str, # required
    'description': str,
    'active': bool, # Default : True
    'priority': int # Default: 1

Other methods:

  • hooks_writer.get_hooks
  • hooks_writer.del_hook

There is no method update, to update a hook, delete it then recreate it.

Verify if a field exists, else update it and add a transformation rule

Warning : a field is identified by its output value that must be unique (we can’t send two values for the same field while we can use the same input field for various output).


# .... continuation of code above
if writer.fields_writer.field_exists('firstname'):

writer.fields_writer.add_field(input='first_name', output='firstname')

Allowed properties for fields:

    'input': str, # required
    'output': str, # required

Other methods:

  • fields_writer.get_field
  • fields_writer.get_fields

There is no method update, to update a field, delete it then recreate it.

Add a rule

writer.fields_writer.add_rule(output_field='firstname', name='uppercase', method='uppercase')

Allowed properties for rules:

    'name': str, # required
    'method': str, # required
    'description': str,
    'active': bool, # Default : True
    'params': list,
    'blocking': bool, # Default : False
    'priority': int # Default: 1

Other methods:

  • fields_writer.del_rule
  • fields_writer.get_rules
  • fields_writer.rule_exists

There is no method update, to update a rule, delete it then recreate it.

Retrieve a Job, its hooks and fields

from impulsare_job import Reader

Reader = Reader('/etc/impulsare/config.yml', 'My Job')
job = Reader.get_job()
hooks = Reader.get_hooks()
fields = Reader.get_fields() # Get rules for first field : rules = fields[0].rules

Development & Tests

$ pip install -r requirements.txt
$ pip install -r requirements-dev.txt
$ py.test


Don’t check if table exists on each model but do it on app installation

To get the DB :

from impulsare_job import models

db = models.get_db('/etc/impulsare/config.yml')
db.create_tables([models.Job, models.Hook, models.Rule])

Refactor writer

To have a class for hooks, and another for rules.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for impulsare-job, version 1.0a2
Filename, size File type Python version Upload date Hashes
Filename, size impulsare-job-1.0a2.tar.gz (9.4 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page