grasp 0.3.2

Useful introspection tools.

====================
Grasp
====================

A set of python functions to help with interactive object inspection
and discovery.

These help one grok, grasp, or get the gist of running code. They're
most useful in the context of an interactive IPython session, but can
be used in any Python shell. They're also helpful in the debugger.
They produce output that can be parsed programmatically if you find
that useful. I'll start with examples using the provided IPython
magic commands, since that's how I use them. Then I'll show how to
use them in other contexts.

There are basically three functions provided:
* gist -- object inspection
* rtype -- recursive type, deep object inspection
* apropos -- deep search for things with a given name, value, etc.

You can find code and dowloads at the Launchpad page or the PyPI page

http://launchpad.net/grasp
http://pypi.python.org/pypi/grasp/

Written and maintained by Greg Novak <[e-mail]>

Introduction
============

gist
----

Suppose you are confronted with an unfamiliar object. What are
its characteristics? What kinds of things can it do? What kind
information does it contain?

gist returns all the attributes of the object organized by type:

In [1]: foo = 5
In [2]: %gist foo
Out[2]: {builtin_function_or_method: [bit_length, conjugate],
int: [denominator, imag, numerator, real]}

The return value is a dict with one key for each type. The value of
each key is a list of strings giving the names of all the attributes
of the given type. So integer objects have four integer attributes,
named numerator, denomintor, real, and imag, (accessed via foo.real,
etc.) as well as two function attributes (accessed via
foo.bit_length(), etc) named bit_length and conjugate.

By default, attributes with leading underscores are omitted. You can
include them by asking for verbose output with -v (the output below is
trimmed for brevity):

In [3]: %gist -v foo
Out[3]: {method-wrapper: [__abs__, __add__],
int: [denominator, imag, numerator, real],
builtin_function_or_method: [__format__, __getnewargs__,
bit_length, conjugate],
str: [__doc__],
type: [__class__]}


You can pass python code to the magic command, which is evalated.
(again, output trimmed):

In [4]: %gist numpy.array([1,2,3])
Out[4]: {buffer: [data],
int: [itemsize, nbytes, ndim, size],
builtin_function_or_method: [all, any, argmax]
tuple: [shape, strides],
ndarray: [T, imag, real]}

rtype
-----

Suppose you are confronted with a list that contains a deeply nested
structure of tuples, lists, and so on. What is this object all about?
Are there regularities in the structure? You want a function like
type(), but you want it to be recursive so that it summarizes the
type structure of the object as much as possible.

Start with a trivial case:

In [5]: %rtype 1
Out[5]: 'int'

If the object is a tuple of objects, all of the same type, say so.

In [6]: %rtype [1, 2, 3]
Out[6]: 'list of 3 int'

What if it's a tuple of heterogeneous types? List them all. Note
that the return value is now a list of strings.

In [7]: %rtype [1, 1.1, 2]
Out[7]: ['list of', 'int', 'float', 'int']

The rtype function is recursive, so this gets interesting when you add
another layer of container objects:

In [8]: %rtype [(1,2), (3,4), (5,6)]
Out[8]: ['list of 3', 'tuple of 2 int']

The rtype function knows about numpy arrays and classifies them
according to shape and type.

In [9]: %rtype [numpy.array([1,2]), numpy.array([3,4]), numpy.array([5,6])]
Out[9]: ['list of 3', 'ndarray of (2,) int64']

apropos
-------

Suppose I know that matplotlib (a Python plotting library) defines a
bunch of colormaps, but I don't have any idea where to find them
within the module. I can search recursively through the whole module
namespace, returning all of the ways to 'reach' objects with names
having to do with colormaps:

In [10]: import matplotlib
In [11]: %apropos cmap matplotlib
Out[11]: ['matplotlib.cm.cmapname',
'matplotlib.cm.get_cmap']

Note the many layers of indirection that apropos digs through to
arrive at the results. Apropos is similar to the standard %psearch
magic command that's included in IPython. The difference is that that
psearch only handles one level at a time (although it can search
intermediate modules, as long as you know how many dots separate the
target from the module).

In [12]: %psearch matplotlib.cmap*
In [13]: %psearch matplotlib.*.cmap*

Using apropos, you can also search for objects whose string
representation contains a given string. If no object to search is
given, search the entire namespace given by globals()

In [14]: %apvalue blue

You can search for objects whose docstring contains a given string.
Use quotes if the search string contains a space (this works for any
of the aporpos commands).

In [15]: %apdoc "colormap instance" matplotlib

There are versions of each of the above that accept regular
expressions.

In [16]: %apname_regex [Cc]olors

In [17]: %apvalue_regex [Cc]olors

In [18]: %adoc_regex [Cc]olors

You can also pass python code as the object in which to search and it
will be evaluated, should you find that useful. The apropos commands
assume that the first argument is the search string and everything
else is the object in which to search, so the second argument doesn't
need to be quoted if it contains spaces.

In [18]: %apdoc "colormap instance" dict(a=matplotlib, b=numpy)

You can search for python objects (rather than strings) using %apobj.
This gives the name of any object equal to the tuple (1,3,5) in the
numpy module.

In [19]: %apobj (1,3,5) numpy

If the search object contains spaces, it must be quoted

In [20]: %apobj "(1, 3, 5)" numpy

You can refer to variables in the user's namespace

In [21]: foo = numpy.array([1,2,3])
In [22]: %apobj foo numpy
In [23]: %apobj [foo,37] numpy

With the %apropos and %aobj commands, you can provide your own
function that returns True if the object should be considered a match
and false otherwise. This can be a named function on an anonymous
function (probably requiring quotation marks). In the latter case the
code will be evaluated. See the docstrings for %apropos and %apobj
for details.

In [24]: def my_search_fn(needle, name, obj): return name and needle in name

In [25]: %apropos -s my_search_fn Colors
In [26]: %apropos -s "lambda needle, name, obj: name and needle in name" Colors

For examples, see the search functions in the grasp module:

In [27]: %apname search grasp
Out[27]: ['grasp.search_doc',
'grasp.search_doc_regexp',
'grasp.search_equal',
'grasp.search_name',
'grasp.search_name_regexp',
'grasp.search_value',
'grasp.search_value_regexp']
In [28]: grasp.search_doc?

A final note is that apropos is meant to be exhaustive, so it tends to
return more than you need. You generally have to pick through the
results a little to find what you want.

Installation
============

There are several possible installation techniques listed here in
rough order of preference:

1) Using pip (https://pypi.python.org/pypi/pip):

To install system-wide:

pip install grasp

To install in the user-specific Python directory
(~/Library/Python/2.7/lib/python/site-packages or similar on OS X):

pip install --user grasp

2) Using easy_install (https://pypi.python.org/pypi/distribute):

The same as above, except substitute easy_install for pip

3) Using distutils (included with Python)

This is the standard distutils routine: download the code, extract
from the archive, and install system-wide via something like
(adjusting links and version numbers as appropriate):

wget https://pypi.python.org/packages/source/g/grasp/grasp-0.3.0.tar.gz
tar xzf grasp-0.3.0.tar.gz
cd grasp-0.3.0
python setup.py install

If you want to install into a user directory, the last line becomes
python setup.py install --user

To install somewhere under the user's home directory:
python setup.py install --home=~/some/place


Usage
=====

To use grasp from IPython, type

%load_ext grasp

There are two ways to ensure that grasp is loaded automatically
when IPython starts:

1. Open or create the file ~/.ipython/profile_default/ipython_config.py
and add this line to it:

c.InteractiveShellApp.extensions = ['grasp']

2. Put the following line into a file with the extension .ipy in the
directory ~/.ipython/profile_default/startup/

%load_ext grasp

I have used this package on Gnu/Linux and OS X. I have not tested it
on Windows. It should work fine -- the main difference will be
getting it installed.


Non-IPython Environments
========================

If you don't use IPython, you just call the python functions upon
which the magic functions are based directly. The translation between
the magic command arguments given above and the arguments of the
Python functions should be straightforward. The magic commands
sometimes have shorter, more cryptic names with a view toward saving
keystrokes during heavy interactive use. In plain Python it will look
something like this:

>>> import grasp
>>> grasp.gist([1,2,3], verbose=True)
>>> grasp.apropos_name('foo', grasp)
>>> grasp.recursive_type([1,2,3])

IPython has nice pretty-printing facilities, and I took advantage of
those in deciding how grasp presents the information it finds. Thus
gist() returns a dict and relies on IPython to format it in a readable
way for interactive use. This means that it's possible to use the
output programatically. One could, for example, do this to set
every attribute of a class with integer type to 42:

atts = grasp.gist(object)
for att in atts['int']:
setattr(object, att, 42)

Version Information
===================

Grasp passes all tests with Python 2.4 through 2.7.

When translated to Python 3 via the 2to3 script, Grasp passes all
tests on Python 3.1, 3.2, and 3.3. Strictly speaking, for 3.1 there
are errors with the unittest.expectedFailure decorator, but that seems
to be a problem with unittest, not Grasp.

The IPython magic commands work for versions of IPython with the
decorators IPython.core.magic.magics_class and
IPython.core.magic.line_magic. It looks like these were introduced in
IPython version 0.13. Making it work with older versions of IPython
would only involve registering the magic commands in a different way,
which is probably not difficult, but I see no compelling reason to do
it.

License
=======

I've released the code under the CC0 licence, essentially putting it
into the public domain. You can do whatever you want with it. If you
incorporate grasp into another project, I ask for the courtesy of two
favors:

1) Include an appropriate acknowledgement of the fact that your
project uses grasp.

2) Let me know ([e-mail]) so that I can link to your
project from the grasp web site.

Acknowledgements
================

Grasp was written in 2006 by Greg Novak <[e-mail]> and
cleaned up for public consumption by the same in 2013.

This was written for my own use when developing Python code to produce
and analyze simulation output using the excellent IPython, Numpy,
Scipy, and matplotlib packages. I have benefited enormously from the
work of the authors of those packages over the years.