Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

An extension to Scons which enables dependency tracking on python script imports.

Project description

PyScons is a tool which works with Scons. It is installed into a new environment with either of the two commands:

from pyscons import PYTOOL
env = Environment(tools = ['default', PYTOOL()])

or:

from pyscons import PYTOOL
env = Environment()
PYTOOL()(env)

This does three things:

  1. Installs a builder: PyScript.
  2. Installs a builder: PyScons.
  3. Installs a new scanner for python scripts.

PyScript

This Builder runs python scripts and modules.

First, it will automatically find the “.py” files referred to when running a module as a script with the ‘-m’ option. For example the following code will run a module as script and add the appriate files to the dependencies:

env.PyScript("out", ["-m timeit", "myscript.py"], "python $SOURCES > $TARGET")

Second, it defaults the command string to “python $SOURCES” and using the “capture” keyword argument, can automatically append the appropriate strings to capture the output or error (or both) to the targets:

env.PyScript("out", ["-m timeit", "myscript.py"], capture="output")

or to capture both the output and error:

env.PyScript(["out","err"], ["-m timeit", "myscript.py"], capture="both")

Just like Command, multiple steps can be used to create a file:

env.PyScript("out", ["step1.py", "step2.py"],
     ["python ${SOURCES[0]} > temp", "python ${SOURCES[1]} > $TARGET", Delete("temp")])

PyScons (experimental)

This Builder enables running a python script as if it is a scons script.

This is distinct from SConscript which functions like an include. Instead, PyScons spawns a new scons process. Spawning a new process allows for temporary files to be automatically deleted without triggering a rebuild.

To use this builder, create a .py file with, for example, the following code in a file (my_scons.py):

from pyscons import PySconsSetup
targets, sources, basename = PySconsSetup()

temp = basename + ".temp"

PyScript(temp, ["step1.py"] + sources, capture="out")
Pyscript(targets, ["step2.py", temp], capture="out")

Now, this file can be called from a SConstruct file like so:

PyScons(targets, sources, "my_scons.py", options = "-j4")

The temp file be generated if it is required to generate targets, but will be immediately deleted. This is useful for builders which generate large intermediate files which would should be deleted without triggering a rebuild. This can be better than passing a list to the Command function for a few special cases:

  1. PyScons enables parallel execution of a multistep submodule(if you pass the -j option to the spawned scons)

  2. PyScons creates a workflow environment (like Pipeline Pilot) in scons which enables complex tasks to be packaged in scons files for use in other scons files.

  3. PyScons can turn intermediate file deletion on and off with a single flag:

    PyScons(targets, sources, "my_scons.py", clean = True) # intermediate file deleted
    PyScons(targets, sources, "my_scons.py", clean = False) # intermediate file retained
    

Unfortunately, dependency tracking does not propagate up from the spawned scons. So sources that are not explicitely included in the “sources” variable will not be tracked. There may be no good way around this, short of creating a new scanner and file extension (which I may do if there interest in this feature).

PyScanner

This scanner uses the modulefinder module to find all import dependencies in all sources with a ‘py’ extension. It can take two options in its constructor:

  1. filter_path: a callable object (or None) which takes a path as input and returns true if this file should be included as a dependency by scons, or false if it should be ignored. By default, this variable becomes a function which ensures that no system python modules or modules from site-packages are tracked. To track all files, use “lambda p: True”.
  2. recursive: with a true (default) or false, it enables or disables recursive dependency tracking.

For example to track all files (including system imports) in a nonrecursive scanner, use the following install code in your SConstruct:

from pyscons import PYTOOL
env = Environment(tools = ['default', PYTOOL(recursive = False, filter_path = lambda p: True)])

Known Issues

Relative imports do not work. This seems to be a bug in the modulefinder package that I do not know how to fix.

Author

  1. Joshua Swamidass (homepage)

Project details


Release history Release notifications

History Node

1.0.91

History Node

1.0.88

History Node

1.0.87

History Node

1.0.86

History Node

1.0.85

History Node

1.0.84

History Node

1.0.83

History Node

1.0.82

History Node

1.0.81

History Node

1.0.80

History Node

1.0.78

History Node

1.0.77

History Node

1.0.76

History Node

1.0.75

History Node

1.0.74

History Node

1.0.73

History Node

1.0.72

History Node

1.0.71

History Node

1.0.70

History Node

1.0.69

History Node

1.0.68

History Node

1.0.67

History Node

1.0.66

History Node

1.0.63

History Node

1.0.62

History Node

1.0.61

History Node

1.0.60

History Node

1.0.59

History Node

1.0.58

History Node

1.0.57

History Node

1.0.56

History Node

1.0.55

History Node

1.0.54

History Node

1.0.52

History Node

1.0.51

History Node

1.0.50

History Node

1.0.49

History Node

1.0.48

History Node

1.0.47

History Node

1.0.46

History Node

1.0.45

History Node

1.0.44

History Node

1.0.43

History Node

1.0.42

History Node

1.0.41

History Node

1.0.40

History Node

1.0.38

History Node

1.0.37

History Node

1.0.36

History Node

1.0.35

History Node

1.0.34

History Node

1.0.33

History Node

1.0.32

History Node

1.0.31

History Node

1.0.30

History Node

1.0.29

History Node

1.0.28

History Node

1.0.27

History Node

1.0.25

History Node

1.0.24

History Node

1.0.23

History Node

1.0.22

This version
History Node

1.0.21

History Node

1.0.20

History Node

1.0.19

History Node

1.0.18

History Node

1.0.16

History Node

1.0.15

History Node

1.0.14

History Node

1.0.12

History Node

1.0.10

History Node

1.0.8

History Node

1.0.69+

History Node

1.0.67+

History Node

1.0.50+

History Node

1.0.42+

History Node

1.0.30+

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
PyScons-1.0.21.tar.gz (4.8 kB) Copy SHA256 hash SHA256 Source None Sep 25, 2010

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page