Time and memory tracking task manager for Machine Learning applications
Project description
Syllabus
Time and memory tracking task manager for Machine Learning applications
Dependencies
- Python 3
- ansiwrap
Requirements
- min terminal width of 66 columns
- ANSI color support
Usage
The Task Class
The Task
class is the core of Syllabus
. Only one task should ever be created from scratch in any program; all other tasks should be spawned off of this task using subtask
.
Parameters
name
: str; name of the taskdesc
: str; task descriptionmp
: bool; True if multiprocessing should be enabled. This creates a multiprocessing managed queue, which allows the queue to be shared with other processes; however, this operation requires creation of a dedicated process, and has a significant memory cost. Therefore, themp
flag should not be enabled unless multiprocessing is to be used.
Core Methods
start(name=None, desc=None)
: start the task (sets the start time). If name or description are not None, updates the task's name and description.done(*objects, name=None, desc=None, nowait=False)
: mark the task as done (sets the end time).name
,desc
: name and description; if not None, updates the name and description.*objects
: list of objects. Adds the memory footprints of the objects, and stores it in order to track memory usage.nowait
: is only considered if the task is a root task. If nowait is set to True, the program continues immediately; if nowait is set to False, the method blocks until the task's accountant finishes processing all messages.
subtask(name='Child Task', desc=None) -> Task
: create a subtask. This method should always be used when creating new tasks, since it passes the parent task's reporting queue on to the child.name
: subtask namedesc
: subtask description
Status, Reporting
info(msg)
: send message;msg
can be arbitrary type, as long asmsg
can be turned into a string withstr()
)print(msg)
: alias forinfo
error(e)
: send error.e
does not have to be an Exception.warn(e)
: send warning.e
does not have to be a Warning.runtime() -> float
: get current runtime.status() -> (int, int)
: get progress as (complete tasks, total tasks)progress() -> float
: get progress as (complete tasks / total tasks)add_task(n)
: addn
tasks to the task counteradd_progress(n)
: mark an additionaln
tasks as completed
Multithreading
pool(target, args, shared_args=[], shared_kwargs={}, reducer=None, recursive=True, split=2, name='Child Task Thread', threads=None, process=False)
: Create a thread pool.target
: target functionargs
: list of args to pass inshared_args
,shared_kwargs
: shared arguments to pass to all threadsreducer(retval[])
: if a function is passed, is used to combine results. Should take in a list of results (as returned bytarget
) and return a combined result.recursive
: reduce recursively?split
: ifrecursive
, how many entries per reduce?name
: child task default namethreads
: number of threads; ifNone
, usescpu_count()
instead
Multiprocessing
pool(target, args, shared_args=None, shared_init=None, reducer=None, recursive=True, split=2, name='Child Task Process', cores=None)
: Create a process pool.target
: target functionargs
: list of args to pass inshared_args
: shared arguments to pass to all threadsshared_init
: initializer for shared argsreducer(retval[])
: if a function is passed, is used to combine results. Should take in a list of results (as returned bytarget
) and return a combined result.recursive
: reduce recursively?split
: ifrecursive
, how many entries per reduce?name
: child task default namecores
: number of cores; ifNone
, usescpu_count()
instead
App
Parameters
-
BasicTaskApp(*args, refresh_rate=20, **kwargs)
: basic app with a updating outputrefresh_rate
: output refresh rate, in Hz- Extends
Task
; configuration options*args
and**kwargs
are passed on.
-
InteractiveTaskApp(*args, refresh_rate=20, **kwargs)
: more sophisticated terminal app with keyboard controlsrefresh_rate
: output refresh rate, in Hz- Extends
Task
; configuration options*args
and**kwargs
are passed on.
-
TaskViewer(file)
: viewer for saved task trees.file
: input filename to open and parse- Methods:
print()
: print tree (same format asBasicTaskApp
)save(file, color=False)
: save rendered output to filefile
: output filenamecolor
: if False, ANSI escape sequences are stripped before saving
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Syllabus-2.0.tar.gz
(18.0 kB
view hashes)
Built Distribution
Syllabus-2.0-py3-none-any.whl
(23.0 kB
view hashes)