odoo11-addon-queue-job

This addon adds an integrated Job Queue to Odoo.

It allows to postpone method calls executed asynchronously.

Jobs are executed in the background by a Jobrunner, in their own transaction.

Example:

from odoo import models, fields, api
from odoo.addons.queue_job.job import job

class MyModel(models.Model):
   _name = 'my.model'

   @api.multi
   @job
   def my_method(self, a, k=None):
       _logger.info('executed with a: %s and k: %s', a, k)


class MyOtherModel(models.Model):
    _name = 'my.other.model'

    @api.multi
    def button_do_stuff(self):
        self.env['my.model'].with_delay().my_method('a', k=2)

In the snippet of code above, when we call button_do_stuff, a job capturing the method and arguments will be postponed. It will be executed as soon as the Jobrunner has a free bucket, which can be instantaneous if no other job is running.

Features:

Views for jobs, jobs are stored in PostgreSQL
Jobrunner: execute the jobs, highly efficient thanks to PostgreSQL’s NOTIFY
Channels: give a capacity for the root channel and its sub-channels and segregate jobs in them. Allow for instance to restrict heavy jobs to be executed one at a time while little ones are executed 4 at a times.
Retries: Ability to retry jobs by raising a type of exception
Retry Pattern: the 3 first tries, retry after 10 seconds, the 5 next tries, retry after 1 minutes, …
Job properties: priorities, estimated time of arrival (ETA), custom description, number of retries
Related Actions: link an action on the job view, such as open the record concerned by the job

Table of contents

Installation

Be sure to have the requests library.

Configuration

Using environment variables and command line:
- Adjust environment variables (optional):
  - ODOO_QUEUE_JOB_CHANNELS=root:4 or any other channels configuration. The default is root:1
  - if xmlrpc_port is not set: ODOO_QUEUE_JOB_PORT=8069
- Start Odoo with --load=web,queue_job and --workers greater than 1. [1]
Using the Odoo configuration file:

[options]
(...)
workers = 6
server_wide_modules = web,queue_job

(...)
[queue_job]
channels = root:2

Confirm the runner is starting correctly by checking the odoo log file:

...INFO...queue_job.jobrunner.runner: starting
...INFO...queue_job.jobrunner.runner: initializing database connections
...INFO...queue_job.jobrunner.runner: queue job runner ready for db <dbname>
...INFO...queue_job.jobrunner.runner: database connections ready

Create jobs (eg using base_import_async) and observe they start immediately and in parallel.
Tip: to enable debug logging for the queue job, use --log-handler=odoo.addons.queue_job:DEBUG

Usage

To use this module, you need to:

Go to Job Queue menu

Developers

Bypass jobs on running Odoo

When you are developing (ie: connector modules) you might want to bypass the queue job and run your code immediately.

To do so you can set TEST_QUEUE_JOB_NO_DELAY=1 in your enviroment.

Bypass jobs in tests

When writing tests on job-related methods is always tricky to deal with delayed recordsets. To make your testing life easier you can set test_queue_job_no_delay=True in the context.

Tip: you can do this at test case level like this

@classmethod
def setUpClass(cls):
    super().setUpClass()
    cls.env = cls.env(context=dict(
        cls.env.context,
        test_queue_job_no_delay=True,  # no jobs thanks
    ))

Then all your tests execute the job methods synchronously without delaying any jobs.

Known issues / Roadmap

After creating a new database or installing queue_job on an existing database, Odoo must be restarted for the runner to detect it.
When Odoo shuts down normally, it waits for running jobs to finish. However, when the Odoo server crashes or is otherwise force-stopped, running jobs are interrupted while the runner has no chance to know they have been aborted. In such situations, jobs may remain in started or enqueued state after the Odoo server is halted. Since the runner has no way to know if they are actually running or not, and does not know for sure if it is safe to restart the jobs, it does not attempt to restart them automatically. Such stale jobs therefore fill the running queue and prevent other jobs to start. You must therefore requeue them manually, either from the Jobs view, or by running the following SQL statement before starting Odoo:

update queue_job set state='pending' where state in ('started', 'enqueued')

Changelog

[IMP] Dont’ start the Jobrunner if root channel’s capacity is explicitly set to 0
[ADD] Ability to set several jobs to done using an multi-action (port of #59)
[REF] Extract a method handling the post of a message when a job is failed, allowing to modify this behavior from addons
[ADD] Default “related action” for jobs, opening a form or list view (when the job is linked to respectively one record on several). (#46)
[FIX] Error when creating a job channel manually (#96)

11.0.1.1.0 (2018-05-25)

[ADD] New neat OCA readme file format (#71)
[ADD] The Jobrunner will keep the same Web Session and no longer generate a new session per job (#54)
[ADD] Configurable scheme, host and HTTP authentication (#51)
[ADD] base_sparse_field does not longer need to be set in --load (#47)
[FIX] Correct automatic registration of channels and job functions (#69)
[FIX] Correct compatibility with @property methods (#50 and #69)

Camptocamp
ACSONE SA/NV

Contributors

Guewen Baconnier <guewen.baconnier@camptocamp.com>
Stéphane Bidoul <stephane.bidoul@acsone.eu>
Matthieu Dietrich <matthieu.dietrich@camptocamp.com>
Jos De Graeve <Jos.DeGraeve@apertoso.be>
David Lefever <dl@taktik.be>
Laurent Mignon <laurent.mignon@acsone.eu>
Laetitia Gangloff <laetitia.gangloff@acsone.eu>
Cédric Pigeon <cedric.pigeon@acsone.eu>

Maintainers

This module is maintained by the OCA.

OCA, or the Odoo Community Association, is a nonprofit organization whose mission is to support the collaborative development of Odoo features and promote its widespread use.

Current maintainer:

This module is part of the OCA/queue project on GitHub.

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

Metadata

Release files for odoo11-addon-queue-job 11.0.1.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for odoo11-addon-queue-job 11.0.1.4.0
File	Interpreter	ABI	Platform	Reset
odoo11_addon_queue_job-11.0.1.4.0-py2.py3-none-any.whl 85.0 kB Sep 7, 2022	Python 2, Python 3	none	any	Details

Release files / odoo11_addon_queue_job-11.0.1.4.0-py2.py3-none-any.whl

Download URL	odoo11_addon_queue_job-11.0.1.4.0-py2.py3-none-any.whl
Size	85.0 kB
Tags	Python 2 Python 3
SHA-256 checksum How to use checksums	`8f73f3f02a38874f32834575616234fc35063fb5e593a4997234051aea07a6db`
BLAKE2b-256 checksum How to use checksums	`dd00d6471cf272c605a1b904082a7c697acb125d000b50f23e11bab959f6e5c4`
Upload date	Sep 7, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.0 CPython/3.8.5`