Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Python Tools

Project Description
.. -*- rst -*- -*- restructuredtext -*-

.. This file should be written using the restructure text
.. conventions. It will be displayed on the bitbucket source page and
.. serves as the documentation of the directory.

.. |virtualenv.py| replace:: ``virtualenv.py``
.. _virtualenv.py: https://raw.github.com/pypa/virtualenv/master/virtualenv.py

.. |EPD| replace:: Enthough Python Distribution
.. _EPD: http://www.enthought.com/products/epd.php
.. _Anaconda: https://store.continuum.io/cshop/anaconda
.. _Conda: http://docs.continuum.io/conda
.. _Miniconda: http://conda.pydata.org/miniconda.html

.. _Enthought: http://www.enthought.com
.. _Continuum Analytics: http://continuum.io

.. _Spyder: https://code.google.com/p/spyderlib/
.. _Wakari: https://www.wakari.io
.. _Canopy: https://www.enthought.com/products/canopy/

.. _mercurial: http://mercurial.selenic.com/
.. _virtualenv: http://www.virtualenv.org/en/latest/
.. _IPython: http://ipython.org/
.. _Ipython notebook: \
http://ipython.org/ipython-doc/dev/interactive/htmlnotebook.html
.. _NBViewer: http://nbviewer.ipython.org
.. |pip| replace:: ``pip``
.. _pip: http://www.pip-installer.org/
.. _git: http://git-scm.com/
.. _github: https://github.com
.. _RunSnakeRun: http://www.vrplumber.com/programming/runsnakerun/
.. _GSL: http://www.gnu.org/software/gsl/
.. _pygsl: https://bitbucket.org/mforbes/pygsl
.. _Sphinx: http://sphinx-doc.org/
.. _SciPy: http://www.scipy.org/
.. _Mayavi: http://code.enthought.com/projects/mayavi/
.. _NumPy: http://numpy.scipy.org/
.. _Numba: https://github.com/numba/numba#readme
.. _NumbaPro: http://docs.continuum.io/numbapro/
.. _Blaze: http://blaze.pydata.org
.. _Python: http://www.python.org/
.. _matplotlib: http://matplotlib.org/
.. _Matlab: http://www.mathworks.com/products/matlab/
.. _MKL: http://software.intel.com/en-us/intel-mkl
.. _Intel compilers: http://software.intel.com/en-us/intel-compilers
.. _Bento: http://cournape.github.com/Bento/
.. _pyaudio: http://people.csail.mit.edu/hubert/pyaudio/
.. _PortAudio: http://www.portaudio.com/archives/pa_stable_v19_20111121.tgz
.. _MathJax: http://www.mathjax.org/
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _Emacs: http://www.gnu.org/software/emacs/
.. _Pymacs: https://github.com/pinard/Pymacs
.. _Ropemacs: http://rope.sourceforge.net/ropemacs.html
.. _PyPI: https://pypi.python.org/pypi

.. _FFTW: http://www.fftw.org
.. _EC2: http://aws.amazon.com/ec2/
.. _QT: http://qt.digia.com

.. |site.USER_BASE| replace:: ``site.USER_BASE``
.. _site.USER_BASE: https://docs.python.org/2/library/site.html#site.USER_BASE


.. default-role:: math

.. This is so that I can work offline. It should be ignored on bitbucket for
.. example.

.. sidebar:: Sidebar

.. contents::

===========
mmf_setup
===========
This meta-project provides an easy way to install all of the python
tools I typically use. It also serves as a fairly minimal example of
setting up a package the |pip|_ can install, and specifying
dependencies.

In particular, I structure it for the following use-cases:

1. Rapid installation and configuration of the tools I need. For
example, I often use [Sage Mathcloud](cloud.sagemath.com).
Whenever I create a new project, I need to perform some
initialization. With this project, it is simply a matter of using
|pip|_ to install this package, and then using some of the tools.
2. Initial setup of a python distribution on a new computer. This is
a little more involved since one needs to first install python (I
recommend using Miniconda_) and then updating the tools.
3. A place to document various aspects of installing and setting up
python and related tools. Some of this is old, but kept here for
reference.


====================
Quickstart (TL;DR)
====================

1. To get the notebook initialization features without having to install the
package, just copy `nbinit.py <nbinit.py>`_ to your project. Importing this
will try to execute `import mmf_setup;mmf_setup.nbinit()` but failing this,
will manually run a similar code.

2. Install this package from the source directory, PyPI_, etc. with
one of the following:

* **Directly from PyPI**

``pip install --process-dependency-links --user mmf_setup[nbextensions]``

* **From Source**

``pip install --process-dependency-links --user hg+https://bitbucket.org/mforbes/mmf_setup[nbextensions]``

* **From Local Source** (*Run this from the source directory after you unpack it.*)

``pip install --process-dependency-links --user .[nbextnensions]``

Note: these can be run without the ``--user`` flag if you want to
install them system-wide rather than into |site.USER_BASE|_.

3. To get the notebook tools for Jupyter (IPython) notebooks, execute
the following as a code cell in your notebook and then trust the
notebook with ``File/Trust Notebook``::

import mmf_setup; mmf_setup.nbinit()

This will download and enable the calico extensions, as well as set
the theme which is implemented in the output cell so it is stored
for use online such as when rendered through NBViewer_. One can
specify different themes. (Presently only ``theme='default'`` and
``theme='mmf'`` are supported.)

4. To use the mercurial notebook cleaning tools, simply source the
``mmf_setup`` script::

. mmf_setup -v

To do this automatically when you login, add this line to your
``~/.bashc`` or ``~/.bash_profile`` scripts. These can also be
enabled manually by adding the following to your ``~/.hgrc`` file::

[extensions]
strip=
mmf_setup.nbclean=$MMF_UTILS/nbclean.py


where ``$MMF_UTILS`` expands to the install location for the
package (which can be seen by running ``mmf_setup -v``).

This will provide commands for committing clean notebooks such as
``hg cstatus``, ``hg cdiff`` and ``hg ccommit``.


==================
Installing Tools
==================

The following code will download and install the `Calico notebook
extensions`__ from `here`__::

import mmf_setup.notebook_configuration
mmf_setup.notebook_configuration.install_extensions()

======================
Mercurial (hg) Tools
======================

If you source the output of the ``mmf_setup`` script::

. mmf_setup

then your ``HGRCPATH`` will be amended to include this projects
``hgrc`` file which does the following:

1. Adds some useful extensions.
2. Adds the following commands:

* ``hg lga`` (or ``hg lg``): provides a nice concise graphical
display of the repo.
* ``hg cstatus`` (or ``hg cst``):
* ``hg cdiff``: same for ``hg diff``
* ``hg cediff``: same for ``hg ediff``
* ``hg crecord``: same for ``hg record``. Note, this does not
function like commit - it will not record the notebooks with the
full output.
* ``hg ccommit`` (or ``hg ccom``): same for ``hg com`` but also
commits the full version of the notebooks with output as a new
node off the present node with the message ``..: Automatic commit of
output``. This command has two behaviours depending on the
configuration option ``nbclean.output_branch``. If this is not
set::

[nbclean]
output_branch =

then ``hg ccommit`` will commit a cleaned copy of your notebooks
with the output stripped, and then will commit the full notebook
with output (provided that the notebooks have output) as a new
head::

| o 4: test ...: Automatic commit with .ipynb output
|/
@ 3: test ccommit 3
|
| o 2: test ...: Automatic commit with .ipynb output
|/
o 1: test ccommit 1
|
o 0: test commit 0

The parent will always be set to the clean node so that the output
commits can be safely stripped from your repository if you choose
not to keep them.

The other mode of operation can be enabled by specifying a name for
the output branch::

[nbclean]
output_branch = auto_output

This will merge the changes into a branch with the specified name::

| o 4: test ...: Automatic commit with .ipynb output (...) auto_output
|/|
@ | 3: test ccommit 3
| |
| o 2: test ...: Automatic commit with .ipynb output (...) auto_output
|/
o 1: test ccommit 1
|
o 0: test commit 0

This facilitates stripping the output ``hg strip 2`` for example
will remove all output. It also allows you to track the changes in
the output.


Developer Notes
---------------

There are a couple of subtle points here that should be mentioned.

* I explored both ``(un)shelve`` and ``commit/strip`` versions of
saving the current state. While the former allows for shorter
aliases, it can potentially trigger merges, so we use the latter.
* I sometimes write commit hook. These should only be run on the real
commit, so we define the alias ``_commit`` which will bypass the
hooks as `discussed here`__.
* The list of files to strip is generated by ``hg status -man``. This
only includes added or modified files. This, if a notebook was
commited with output (using ``hg com``) then it will not be
stripped.
* Our approach of ``. mmf_setup`` sets ``HGRCPATH`` but if this was
not set before, then this will skip the default search. As such, we
insert ``~/.hgrc`` if ``HGRCPATH`` was not previously set. This is
not ideal, but short of sticking an ``%include`` statement in the
users ``~/.hgrc`` file, or creating an ``hg`` alias, I do not see a
robust solution. Note: we only insert ``~/.hgrc`` if ``HGRCPATH``
is not defined - I ran into problems when always inserting it since
this can change the order of precedence.
* Chain commands with semicolons ``;`` not ``&&`` so that things are
restored even if a command fails. (For example, ``hg ccom`` with a
notebook that only has output changes used to fail early.)

__ https://selenic.com/pipermail/mercurial-devel/2011-December/036480.html


Notes
=====

Various notes about python, IPython, etc. are stored in the docs folder.



__ http://jupyter.cs.brynmawr.edu/hub/dblank/public/Jupyter%20Help.ipynb#2.-Installing-extensions
__ https://bitbucket.org/ipre/calico/downloads/


Release History

Release History

This version
History Node

0.1.10

History Node

0.1.9

History Node

0.1.9dev

History Node

0.1.8

History Node

0.1.7

History Node

0.1.7.dev0

History Node

0.1.6

History Node

0.1.5

History Node

0.1.5.dev0

History Node

0.1.4

History Node

0.1.4.dev0

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

History Node

0.1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
mmf_setup-0.1.10-py2-none-any.whl (346.8 kB) Copy SHA256 Checksum SHA256 py2 Wheel Nov 30, 2017
mmf_setup-0.1.10.tar.gz (368.1 kB) Copy SHA256 Checksum SHA256 Source Nov 30, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting