Skip to main content

Time and memory tracking task manager for Machine Learning applications

Project description

Syllabus

Time and memory tracking task manager for Machine Learning applications

Print It!

Dependencies

  • Python 3
  • print
  • ansiwrap

Requirements

  • min terminal width of 66 columns
  • ANSI color support

Usage

The Task Class

The Task class is the core of Syllabus. Only one task should ever be created from scratch in any program; all other tasks should be spawned off of this task using subtask.

Parameters

  • name: str; name of the task
  • desc: str; task description
  • mp: bool; True if multiprocessing should be enabled. This creates a multiprocessing managed queue, which allows the queue to be shared with other processes; however, this operation requires creation of a dedicated process, and has a significant memory cost. Therefore, the mp flag should not be enabled unless multiprocessing is to be used.

Core Methods

  • start(name=None, desc=None): start the task (sets the start time). If name or description are not None, updates the task's name and description.
  • done(*objects, name=None, desc=None, nowait=False): mark the task as done (sets the end time).
    • name, desc: name and description; if not None, updates the name and description.
    • *objects: list of objects. Adds the memory footprints of the objects, and stores it in order to track memory usage.
    • nowait: is only considered if the task is a root task. If nowait is set to True, the program continues immediately; if nowait is set to False, the method blocks until the task's accountant finishes processing all messages.
  • subtask(name='Child Task', desc=None) -> Task: create a subtask. This method should always be used when creating new tasks, since it passes the parent task's reporting queue on to the child.
    • name: subtask name
    • desc: subtask description

Status, Reporting

  • info(msg): send message; msg can be arbitrary type, as long as msg can be turned into a string with str())
  • print(msg): alias for info
  • error(e): send error. e does not have to be an Exception.
  • warn(e): send warning. e does not have to be a Warning.
  • runtime() -> float: get current runtime.
  • status() -> (int, int): get progress as (complete tasks, total tasks)
  • progress() -> float: get progress as (complete tasks / total tasks)
  • add_task(n): add n tasks to the task counter
  • add_progress(n): mark an additional n tasks as completed

Multithreading

  • pool(target, args, shared_args=[], shared_kwargs={}, reducer=None, recursive=True, split=2, name='Child Task Thread', threads=None, process=False): Create a thread pool.
    • target: target function
    • args: list of args to pass in
    • shared_args, shared_kwargs: shared arguments to pass to all threads
    • reducer(retval[]): if a function is passed, is used to combine results. Should take in a list of results (as returned by target) and return a combined result.
    • recursive: reduce recursively?
    • split: if recursive, how many entries per reduce?
    • name: child task default name
    • threads: number of threads; if None, uses cpu_count() instead

Multiprocessing

  • pool(target, args, shared_args=None, shared_init=None, reducer=None, recursive=True, split=2, name='Child Task Process', cores=None): Create a process pool.
    • target: target function
    • args: list of args to pass in
    • shared_args: shared arguments to pass to all threads
    • shared_init: initializer for shared args
    • reducer(retval[]): if a function is passed, is used to combine results. Should take in a list of results (as returned by target) and return a combined result.
    • recursive: reduce recursively?
    • split: if recursive, how many entries per reduce?
    • name: child task default name
    • cores: number of cores; if None, uses cpu_count() instead

App

Parameters

  • BasicTaskApp(*args, refresh_rate=20, **kwargs): basic app with a updating output

    • refresh_rate: output refresh rate, in Hz
    • Extends Task; configuration options *args and **kwargs are passed on.
  • InteractiveTaskApp(*args, refresh_rate=20, **kwargs): more sophisticated terminal app with keyboard controls

    • refresh_rate: output refresh rate, in Hz
    • Extends Task; configuration options *args and **kwargs are passed on.
  • TaskViewer(file): viewer for saved task trees.

    • file: input filename to open and parse
    • Methods:
      • print(): print tree (same format as BasicTaskApp)
      • save(file, color=False): save rendered output to file
        • file: output filename
        • color: if False, ANSI escape sequences are stripped before saving

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Syllabus-2.0.tar.gz (18.0 kB view details)

Uploaded Source

Built Distribution

Syllabus-2.0-py3-none-any.whl (23.0 kB view details)

Uploaded Python 3

File details

Details for the file Syllabus-2.0.tar.gz.

File metadata

  • Download URL: Syllabus-2.0.tar.gz
  • Upload date:
  • Size: 18.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for Syllabus-2.0.tar.gz
Algorithm Hash digest
SHA256 5003cb083af9cc0e3847f4bc4ffe530dae3d378753c896528fe6c6b09691e340
MD5 41cc3f01b1994908c80500cf34611066
BLAKE2b-256 c8c80d28f2d8a9c4f1d6a9ac20b6cb29f247b88a36cfa117b709544f987e62fe

See more details on using hashes here.

File details

Details for the file Syllabus-2.0-py3-none-any.whl.

File metadata

  • Download URL: Syllabus-2.0-py3-none-any.whl
  • Upload date:
  • Size: 23.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.7

File hashes

Hashes for Syllabus-2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b69363ffd6fd4b70742f1eebcb0074d1d9f004d2926730f62591e0c4f1457ebc
MD5 0b64256382fcbf2d029afaf6c38aa378
BLAKE2b-256 584e14e1a04b7885970dadd6d43d67f466d775ec34cffc60b7e2aea640245316

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page