py-pal 1.3.0

Installation

Command line usage of the py-pal module

python -m py_pal <target-module/file>

or

py-pal <target-module/file>

There are multiple aliases to the same command: py-pal, py_pal and pypal. If py-pal is executed this way, all functions called in the code are captured and analyzed. The output is in the form of a pandas data frame.

See the help message:

py-pal -h

py-pal can perform cost analysis on a line-by-line basis:

py-pal <file> -l/–line

The –separate flag can be used to examine the cost function of individual arguments (caution: this function assumes the independence of the arguments):

py-pal <file> -s/–separate

The output of the results can be noisy, to limit this you can use –filter-function to filter the desired functions from the result. Regular expressions are supported:

py-pal <file> -ff/–filter-function .*_sort

Similarly, the result can also be filtered by modules with –filter-module, e.g. to exclude importlib modules

py-pal <file> -fm/–filter-module “^(?!<frozen.*>).*”

To save the results in a specified folder use –out:

py-pal <file> -o/–out results

The output format can be changed with –format:

py-pal <file> -o/–out results –format json

With the additional specification of the –plot flag, the cost functions of the result set are stored as images:

py-pal <file> -o/–out results -p/–plot

For the –log-level flag see the development docs.

Example, creating plots for selected functions:

py-pal tests/examples/sort.py -o results -p -f sort

Programmatic usage of the py-pal module

To profile a single function and get the complexity estimate there is profile_function.

from py_pal.core import profile_function
from py_pal.data_collection.opcode_metric import OpcodeMetric
from py_pal.datagen import gen_random_growing_lists
from algorithms.sort import bubble_sort

profile_function(OpcodeMetric(), gen_random_growing_lists(), bubble_sort)

The profile decorator:

from py_pal.core import profile, DecoratorStore

@profile
def test():
    pass

# Must be called at some point
test()

estimator = AllArgumentEstimator(DecoratorStore.get_call_stats(), DecoratorStore.get_opcode_stats())
res = estimator.export()

By using the profile decorator, it is possible to annotate Python functions such that only the annotated Python functions will be profiled. It acts similar to a whitelist filter.

Another possibility is to use the context-manager protocol:

from py_pal.analysis.estimator import AllArgumentEstimator
from py_pal.data_collection.tracer import Tracer

with Tracer() as t:
    pass

estimator = AllArgumentEstimator(t.get_call_stats(), t.get_opcode_stats())
res = estimator.export()

# Do something with the resulting DataFrame
print(res)

The most verbose way to use the py-pal API:

from py_pal.analysis.estimator import AllArgumentEstimator
from py_pal.data_collection.tracer import Tracer


t = Tracer()
t.trace()

# Your function
pass

t.stop()
estimator = AllArgumentEstimator(t.get_call_stats(), t.get_opcode_stats())
res = estimator.export()

# Do something with the resulting DataFrame
print(res)

All examples instantiate a tracer object that is responsible for collecting the data. After execution, the collected data is passed to the analysis module. Finally, an estimate of the asymptotic runtime of the functions contained in the code is obtained.

Licensing Notes

This work integrates some code from the big_O project. More specifically, most code in py_pal.analysis.complexity, py_pal.datagen and py_pal.analysis.estimator.Estimator.infer_complexity is adapted from bigO.

Changelog