beatdrop 0.1.0a8

Simplified, scalable task scheduling with typechecking.

beatdrop

TODO

In general this should be re-architected

• async internals only
  • It's okay if all classes can't do this but it should be async whenever possible
• Instead of inheritance with schedulers
  • A single main beatdrop class that takes
  • a task backend
  • a storage backend
  • a scheduler??
    • YES - Different schedulers for multithreading multiprocess, external workers
• Schedule entries should stay models?
  • YES - need to be able to serialize and deserialize fields
    • serialize is easy with pydantic models
    • deserialize you will also need a way to find the model, then feed in the the task data
      • Storage classes handle this. We only deal with pydantic models
      • Just need to register all entry types
  • still have the due_in and sent methods
    • YES
  • read only fields start with underscores?
    • NO - there will be a private field _client_ro_fields ClassVar that will keep track of them
    • naming convention to start with ro_.
    • RO fields have defaults - leave it up to people not to set.
    • keep the flag when saving entries to save the client_ro_fields
  • args and kwargs should only be pydantic serializable?
    • NO - only pydantic
  • should I leave it up to a serializer for the entries as a whole?
    • NO - not all serializers will support doing a whole schedule entry

See the full Documentation.

The goal of beatdrop is to provide schedulers and schedule entries that are easy to use, extensible, scalable, and backend agnostic.

It does not run tasks or python functions on a schedule. It will simply interface with task backends to send tasks when they are due.

Installation

Install the base package with pip from PyPi.

$ pip install beatdrop

For particular schedulers and backends you will also need to install their extra dependencies.

$ pip install beatdrop[redis]

Extra dependencies for task backends:

• celery

Extra dependencies for scheduler storage:

• redis

• sql

The all extra dependency will install all extra dependencies for task backends and scheduler storage.

$ pip install beatdrop[all]

Usage

There are 2 main pieces to using beatdrop.

• Schedule Entry - holds the task definitions and scheduling info.

• Schedulers - have 2 main roles

  • They can be run as a scheduler to monitor and send tasks to the task backend.
  • Act as clients for reading and writing schedule entries.

To run the scheduler simply make a python file, create the scheduler and call the run method:

from beatdrop import CeleryRedisScheduler

from my_app import celery_app


sched = CeleryRedisScheduler(
    max_interval=60,
    celery_app=celery_app,
    lock_timeout=180,
    redis_py_kwargs={
        "host": "my.redis.host",
        "port": 6379,
        "db": 0,
        "password": "mys3cr3t"
    }
)
sched.run()

To use the scheduler as a client, you create the scheduler the same as you would to run it:

from beatdrop import CeleryRedisScheduler, IntervalEntry

from my_app import celery_app


# Create a scheduler
sched = CeleryRedisScheduler(
    max_interval=60,
    celery_app=celery_app,
    lock_timeout=180,
    redis_py_kwargs={
        "host": "my.redis.host",
        "port": 6379,
        "db": 0,
        "password": "mys3cr3t"
    }
)
# create a schedule entry
inter = IntervalEntry(
    key="my-interval-entry",
    enabled=True,
    task="test_task",
    args=("my_args", 123),
    kwargs={
        "my_kwargs": 12.4
    },
    period=10
)

# save or update an entry 
sched.save(inter)
# list all entries, this will automatically paginate
schedule_entries = sched.list()
# retrieve a specific entry
my_inter_entry = sched.get(inter.key)
# equivalent to the line above
my_inter_entry = sched.get("my-interval-entry")
# Delete an entry from the scheduler
sched.delete(inter)

Changelog

Changelog for beatdrop. All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.1.0a8] - 2024-02-19

Fixed

• pydantic less than 2
• deprecations for use of datetime.datetime.utcnow()

Removed

• support for python 3.7

[0.1.0a7] - 2023-02-25

Fixed

• tests
• docs

[0.1.0a6] - 2023-02-22

Added

• RQScheduler - Building block scheduler for the RQ (Redis Queue) task backend.
• RQRedisScheduler - Complete scheduler with RQ task backend and redis entry storage.
• RQSQLScheduler - Complete scheduler with RQ task backend and SQL DB entry storage.

Fixed

• PYPI logo

[0.1.0a5] - 2023-02-06

Fixed

• packaging files
• README Links

[0.1.0a4] - 2023-02-05

Fixed

Docstrings updated and documentation added.

[0.1.0a3] - 2023-01-18

Update for pypi formatting.

[0.1.0a2] - 2023-01-17

Update for pypi formatting.

[0.1.0a1] - 2023-01-17

Initial release