Sage Deps - A Sage Dependency Analysis Tool

This is a dependency analysis tool used for understanding Sage's structure. It can be used to detect circular imports, upstream dependencies, and provide various dependency-graph metrics including PageRank.

Warning: This package is still in its testing phase, and may have unexpected behaviour.

Installation

Dependencies

The package requires , , , and .

Install from PyPi:

Run pip install sage-deps to install as a pypi package. Make sure the script sdeps.exe is added to PATH to run the sdeps command.

Manual Install

To install, simply download or clone the repository. Navigate to the root folder (where pyproject.toml lives) and run pip install ..

Additional Setup

The package requires access to the . By default, it assumes you have the repository cloned to the same parent directory as your sage-deps folder. You may modify this by modifying the configuration file, as shown below.

Getting started

To ensure the package is successfully installed, run sdeps. It should output a blank line by default.

The package comes with some pre-compiled resource files generated by SageMath version 10.6. To update these pre-generated files, run:

sdeps -gm -gg

To store and use local resource files, use the -m and -g flags or run:

sdeps -set-config modules_src <modules-path>
sdeps -set-config graph_src <graph-path>

Similarly, to specify a custom SageMath installation path, use the -s flag or run:

sdeps -set-config sage_path <sage-path>

Relative and absolute files are both supported.

Full options list

usage: sdeps [-h] [-s SOURCE_FILE] [-m SOURCE_FILE] [-g SOURCE_FILE] [-o OUTPUT_FILE] [-up SOURCE_CLASS DEPTH]
             [-cc SOURCE_CLASS TIMEOUT] [-cl SIZE] [-gm] [-gi] [-gd] [-gg] [-gdg SOURCE DISTANCE DIRECTION] [-f SOURCE_FILE] [-nf]
             [-view] [-set-config NAME VALUE] [--verbose]

A program top help manage SageMath dependencies.

options:
  -h, --help            show this help message and exit
  -s, --sage-source SOURCE_FILE
                        The source file of Sage.
  -m, --modules-source SOURCE_FILE
                        Specify the modules file.
  -g, --graph-source SOURCE_FILE
                        Specify the graph file.
  -o, --output-file OUTPUT_FILE
                        Specify the file to dump the output.
  -up SOURCE_CLASS DEPTH
                        Generates a breadth-first dependency tree rooted at SOURCE_CLASS, up to a given depth.
  -cc SOURCE_CLASS TIMEOUT
                        Finds cycles starting at SOURCE_CLASS with a timeout of TIMEOUT seconds.
  -cl SIZE              Finds cliques of size SIZE.
  -gm, --generate-modules
                        Generate a modules file. Will output to default location or `--modules-source`.
  -gi, --generate-imports
                        Generate an imports file.
  -gd, --generate-dependencies
                        Generate a dependencies file.
  -gg, --generate-graph
                        Generate an graph file. Will output to default location or `--graph-source`.
  -gdg, --generate-dependency-graph SOURCE DISTANCE DIRECTION
                        Generate a dependency graph rooted at a SOURCE node.
  -f, --ff SOURCE_FILE  Load a custom filter. If not, a default filter is used.
  -nf                   Disables the filter.
  -view, --view         Run a cytoscape.js instance. Specify graph source using `--graph-source`.
  -set-config NAME VALUE
                        Updates the configuration file.
  --verbose             Enable verbose output.

Note: Currently, verbose output doesn't do anything.

Example Usage

Command line examples

We can use the -up flag to compute all upwards dependencies of a particular class in SageMath.

% sdeps -up sage.rings.polynomial.polynomial\_element.Polynomial 1
{0: ['sage.rings.polynomial.polynomial_element.Polynomial'], 1: ['sage.rings.integer.Integer', 'sage.rings.rational_field.RationalField', 'sage.rings.integer_ring.IntegerRing_class', 'sage.rings.padics.padic_generic.pAdicGeneric', 'sage.rings.ideal.Ideal_generic', 'sage.rings.polynomial.polynomial_ring.PolynomialRing_generic', 'sage.rings.polynomial.multi_polynomial.MPolynomial', 'sage.rings.finite_rings.finite_field_base.FiniteField', 'sage.rings.number_field.number_field_base.NumberField', 'sage.rings.finite_rings.finite_field_constructor.FiniteFieldFactory', 'sage.rings.fraction_field.FractionField_generic']}

Similarly, we can do cycle detection. Notice that we need to set a time limit as finding cycles is, in general, NP-hard.

% sdeps -cc sage.rings.integer.Integer 120
[['sage.rings.integer.Integer', 'sage.rings.rational.Rational', 'sage.rings.integer_ring.IntegerRing_class', 'sage.rings.ring.CommutativeRing', 'sage.rings.polynomial.polynomial_element.Polynomial', 'sage.rings.rational_field.RationalField', 'sage.rings.number_field.number_field_base.NumberField', 'sage.rings.qqbar.AlgebraicRealField', 'sage.rings.qqbar.AlgebraicField_common', 'sage.rings.qqbar.AlgebraicNumberPowQQAction', 'sage.rings.real_mpfi.RealIntervalFieldElement', 'sage.rings.integer.Integer'], 
...
]

As a package

There is more freedom of use when treated as a package. An example workflow is shown below.

# Import relevant files
from sagedeps.main import *
from sagedeps.deps.loader import Loader
from sagedeps.deps.data import *
from sagedeps.deps.model.dependency import *
from sagedeps.deps.filter import *
from sagedeps.analysis import *

# Load from resources
Loader.initialize(scorer=DefaultScorer())

# Get classes and directories
filter = EmptyFilter()
classes = Data.get_classes_filtered(
    filter
)
dirs = Data.get_modules_filtered(
    filter
)
print(len(dirs))
print(len(classes))

# Count number of outdegrees over all classes to get an average
total_deg = 0
for sc in classes:
    total_deg += sc.out_degree(
        relations=[Relation.INHERITANCE, Relation.CLASS_ATTRIBUTE, Relation.DECLARED_SUB_IMPORT, Relation.DECLARED_TOP_IMPORT]
    )

print(total_deg/len(classes))

# Analyze for cliques of size 3
analyzer = CliquesAnalyzer(
    filter=EmptyFilter(),
    size=3,
    edge_types=[Relation.INHERITANCE, Relation.CLASS_ATTRIBUTE, Relation.DECLARED_SUB_IMPORT, Relation.DECLARED_TOP_IMPORT]
)

# Analyze PageRank
analyzer = PageRankAnalyzer(
    filter=EmptyFilter(),
    edge_types=[Relation.INHERITANCE, Relation.CLASS_ATTRIBUTE, Relation.DECLARED_SUB_IMPORT, Relation.DECLARED_TOP_IMPORT],
    weights=[1,1,1,1]
)
res = analyzer.run()
print(res[:15])
print(len(res))

The above snippet outputs

3202
5854
3.3659036556200888
[('sage.structure.sage_object.SageObject', np.float64(0.037002533039364974)), ('sage.rings.integer.Integer', np.float64(0.02290838092408753)), ('sage.rings.integer_ring.IntegerRing_class', np.float64(0.02273003398295342)), ('sage.structure.element.Element', np.float64(0.02090351208515615)), ('sage.structure.parent.Parent', np.float64(0.01437274473122256)), ('sage.categories.category.Category', np.float64(0.011457959949590773)), ('sage.categories.category_singleton.Category_singleton', np.float64(0.010601666149250333)), ('sage.typeset.character_art.CharacterArt', np.float64(0.010176442750443552)), ('sage.misc.lazy_attribute.lazy_attribute', np.float64(0.008751776770612866)), ('sage.categories.category_with_axiom.CategoryWithAxiom', np.float64(0.008289012215276066)), ('sage.structure.element.ModuleElement', np.float64(0.008201175230707807)), ('sage.rings.rational_field.RationalField', np.float64(0.008088172337409227)), ('sage.misc.lazy_attribute._lazy_attribute', np.float64(0.008085176090604891)), ('sage.categories.objects.Objects', np.float64(0.007503707995711554)), ('sage.structure.unique_representation.UniqueRepresentation', np.float64(0.007267035285008635))]
5854