Draw a graph of your data to see the structure of its references.
Project description
Graph your Memory
Want to draw a graph of your data in Python to better understand its structure or the Python memory model in general?
Just call memory_graph.show( your_data )
, an example:
data = [ (1, 2), [3, 4], {5:'five', 6:'six'} ]
import memory_graph
memory_graph.show( data, block=True )
This shows the graph with the starting point of your 'data' drawn using thick lines, the program blocks until the ENTER key is pressed.
If show()
doesn't work well on your system (the PDF viewer
integration is platform specific) use render()
to output the graph
in the format of your choosing and open it yourself.
memory_graph.render( data, "my_graph.png", block=True )
Install
Install using pip:
pip install memory-graph
Additionally Graphviz needs to be installed.
Graph all Local Variables
Often it is useful to graph all the local variables using:
memory_graph.show( locals(), block=True )
So much so that function d()
is available as alias for easier
debugging. Additionally it logs all locals by printing them which
allows for comparing them over time. For example:
from memory_graph import d
my_squares = []
my_squares_ref = my_squares
for i in range(5):
my_squares.append(i**2)
d() # 'd' for debug, logs and graphs all local variables and blocks
my_squares_copy = my_squares.copy()
d(block=False) # debug without blocking
d(log=False,block=False) # debug without logging and blocking
import memory_graph
memory_graph.log_file=open("log.txt","w") # now log to file instead of screen (sys.stdout)
d(graph=False) # debug without showing the graph
Which in the end results in:
my_squares: [0, 1, 4, 9, 16]
my_squares_ref: [0, 1, 4, 9, 16]
i: 4
my_squares_copy: [0, 1, 4, 9, 16]
Notice that in the graph it is clear that 'my_squares' and 'my_squares_ref' share their data while 'my_squares_copy' has its own copy. This can not be observed in the log and shows the benefit of the graph.
Alternatively debug by setting this expression as 'watch' in a debugger tool and open the output file:
memory_graph.render( locals(), "my_debug_graph.pdf" )
Larger Example
This larger example shows objects that share a class (static) variable and also shows we can handle recursive references although the graph layout might suffer a bit.
my_list = [10, 20, 10]
class My_Class:
my_class_var = 20 # class variable: shared by different objects
def __init__(self):
self.var1 = "foo"
self.var2 = "bar"
self.var3 = 20
obj1 = My_Class()
obj2 = My_Class()
data=[my_list, my_list, obj1, obj2]
my_list.append(data) # recursive reference
import memory_graph
memory_graph.show( locals() )
Config
Different aspects of memory_graph can be configured.
Config Visualization, graphviz_nodes
Configure how the nodes of the graph are visualized with:
- memory_graph.graphviz_nodes.linear_layout_vertical : bool
- if False, linear node layout is horizontal
- memory_graph.graphviz_nodes.linear_any_ref_layout_vertical : bool
- if False, linear node layout is horizontal if any of its elements is a refence
- memory_graph.graphviz_nodes.linear_all_ref_layout_vertical : bool
- if False, linear node layout is horizontal if all elements are reference
- memory_graph.graphviz_nodes.key_value_layout_vertical : bool
- if False, key_value node layout is horizontal
- memory_graph.graphviz_nodes.key_value_any_ref_layout_vertical : bool
- if False, key_value node layout is horizontal if any of its elements is a refence
- memory_graph.graphviz_nodes.key_value_all_ref_layout_vertical : bool
- if False, key_value node layout is horizontal if all elements are reference
- memory_graph.graphviz_nodes.padding : int
- the padding in nodes
- memory_graph.graphviz_nodes.padding : int
- the spacing in nodes
- memory_graph.graphviz_nodes.join_references_count : int
- minimum number of reference we join together
- memory_graph.graphviz_nodes.join_circle_size : string
- size of the join circle
- memory_graph.graphviz_nodes.join_circle_minlen : string
- extra space for references above a join circle
- memory_graph.graphviz_nodes.max_string_length : int
- maximum string length where the string is cut off
- memory_graph.graphviz_nodes.category_to_color_map : dict
- mapping van type/caterogries to node colors
- memory_graph.graphviz_nodes.uncategorized_color : dict
- color for unkown types/categories
- memory_graph.graphviz_nodes.graph_attr : dict
- allows to set various graphviz graph attributes
- memory_graph.graphviz_nodes.node_attr : dict
- allows to set various graphviz node attributes
- memory_graph.graphviz_nodes.edge_attr : dict
- allows to set various graphviz edges attributes
See for color names: graphviz colors
To configure more about the visualization use:
digraph = memory_graph.create_graph( locals() )
and see the graphviz api to render it in many different ways.
Config Graph Structure, rewrite_to_node
Configure the structure of the nodes in the graph with:
- memory_graph.rewrite_to_node.reduce_reference_parents : set
- the node types/categories for which we remove the reference to children
- memory_graph.rewrite_to_node.reduce_reference_children : bool
- the node types/categories for which we remove the reference from parents
Config Node Creation, rewrite
Configure what nodes are created based on reading the given data structure:
- memory_graph.rewrite.ignore_types : dict
- all types that we ignore, these will not be in the graph
- memory_graph.rewrite.singular_types : set
- all types rewritten to node as singular values (bool, int, float, ...)
- memory_graph.rewrite.linear_types : set
- all types rewritten to node as linear values (tuple, list, set, ...)
- memory_graph.rewrite.dict_types : set
- all types rewritten to node as dictionary values (dict, mappingproxy)
- memory_graph.rewrite.dict_ignore_dunder_keys : bool
- determines if we ignore dunder keys ('
__example
') in dict_types
- determines if we ignore dunder keys ('
- memory_graph.rewrite.custom_accessor_functions : dict
- custom accessor functions to defined how to read various types
Config Examples
With configuration:
memory_graph.graphviz_nodes.linear_layout_vertical = False # draw lists,tuples,sets,... horizontally
memory_graph.graphviz_nodes.category_to_color_map['list'] = 'yellow' # change color of 'list' type
memory_graph.graphviz_nodes.spacing=15 # more spacing in each node
memory_graph.graphviz_nodes.graph_attr['ranksep']='1.2' # more vertical separation
memory_graph.graphviz_nodes.graph_attr['nodesep']='1.2' # more horizontal separation
memory_graph.rewrite_to_node.reduce_reference_children.remove("int") # draw references to 'int' type
the last example looks like:
Custom Accessor Functions
For any type a custom accessor function can be introduced. For example Panda DataFrames and Series are not visualized correctly by default. This can be fixed by adding custom accessor functions:
import pandas as pd
data = {'Name':['Tom', 'Anna', 'Steve', 'Lisa'],
'Age':[28,34,29,42],
'Length':[1.70,1.66,1.82,1.73] }
df = pd.DataFrame(data)
import memory_graph
memory_graph.rewrite.custom_accessor_functions[pd.DataFrame] = lambda d: list(d.items())
memory_graph.rewrite.custom_accessor_functions[pd.Series] = lambda d: list(d.items())
memory_graph.rewrite_to_node.reduce_reference_parents.add("DataFrame")
memory_graph.rewrite_to_node.reduce_reference_parents.add("Series")
memory_graph.graphviz_nodes.category_to_color_map['Series'] = 'lightskyblue'
memory_graph.show( locals() )
which results in:
Troubleshooting
When edges overlap it can be hard to distinguish them. Using an interactive graphviz viewer, such as xdot, on a '*.gv' output file will help.
Author
Bas Terwijn
Inspiration
Inspired by PythonTutor.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for memory_graph-0.1.19-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f7b2661f9687aee7e0e7e8234e256672e48ca46d92943ae1929a8e9b8b6b21a7 |
|
MD5 | a7b94e7cb9bc4ab91fb4ba95fb8c3d53 |
|
BLAKE2b-256 | f1d42f0704eb0f05323ef589dbb7bad54775aaedeca739f769a30f921a268e95 |