importnb 0.3.1

Import Jupyter (ne IPython) notebooks into tests and scripts.

importnb imports notebooks as modules & packages. Use notebooks as tests, source code, importable modules, and command line utilities.

Installation

pip install importnb

---

conda install -c conda-forge importnb

---

importnb for testing

After importnb is install, pytest will discover and import notebooks as tests.

pytest readme.ipynb

Notebooks are often used as informal tests, now they can be formally tested with pytest plugins

importnb can run unittests and doctests against notebook modules.

ipython -m importnb.test readme

importnb interprets the first markdown cell as a docstring. This is a nice place to put doctests to improve the reusability of a notebook.

---

importnb for the commmand line

importnb can run notebooks as command line scripts. Any literal variable in the notebook, may be applied as a parameter from the command line.

ipython -m importnb -- readme.ipynb --foo "A new value"

---

importnb for Python and IPython

Restart and run all or it didn't happen.

importnb excels in an interactive environment and if a notebook will Restart and Run All then it may reused as python code. The Notebook context manager will allow notebooks with valid names to import with Python.

>>> from importnb import Notebook

For brevity

    with __import__('importnb').Notebook(): 
        import readme

importnb.loader will find notebooks available anywhere along the sys.path.

or explicity

    from importnb import Notebook
    with Notebook(): 
        import readme

    foo = 42
    import readme
    assert readme.foo is 42
    assert readme.__file__.endswith('.ipynb')

Modules may be reloaded

The context manager is required to reload a module.

    from importlib import reload
    with Notebook():
        reload(readme)

Partial loading

The importnb.loader.Notebook will import a notebook even if there is an exception by supplying the exceptions option. The exception is found on module._exception.

    with Notebook(exceptions=BaseException):
        try: from . import readme
        except: import readme

Lazy imports

The lazy option will delay the evaluation of a module until one of its attributes are accessed the first time.

    with Notebook(lazy=True):
        import readme

Capture Outputs

importnb can capture the stdout, stderr, and display in the context manager. The arguments are similar to IPython.util.capture.capture_output.

    with Notebook(stdout=True, stderr=True, display=True) as output:
        import readme

Docstring

The first markdown cell will become the module docstring.

    if __name__ == '__main__':
        print(readme.__doc__.splitlines()[0])

__importnb__ imports notebooks as modules & packages.  Use notebooks as tests, source code, importable modules, and command line utilities.

Meaning non-code blocks can be executeb by doctest.

    if __name__ == '__main__':
        __import__('doctest').testmod(readme)

Import notebooks from files

Notebook names may not be valid Python paths. In this case, use Notebook.from_filename.

   Notebook.from_filename('readme.ipynb')

Import under the __main__ context.

   Notebook('__main__').from_filename('readme.ipynb')

Parameterize Notebooks

Literal ast statements are converted to notebooks parameters.

In readme, foo is a parameter because it may be evaluated with ast.literal_val

    from importnb import Parameterize
    f = Parameterize().from_filename(readme.__file__)

The parameterized module is a callable that evaluates with different literal statements.

    assert callable(f)
    f.__signature__

<Signature (*, foo=42)>



assert f().foo == 42
assert f(foo='importnb').foo == 'importnb'

Run Notebooks from the command line

Run any notebook from the command line with importnb. Any parameterized expressions are available as parameters on the command line.

!ipython -m importnb -- readme.ipynb --foo "The new value"

Integrations

IPython

IPython Extension

Avoid the use of the context manager using loading importnb as IPython extension.

%load_ext importnb

%unload_ext importnb will unload the extension.

Default Extension

importnb may allow notebooks to import by default with

!importnb-install

If you'd like to play with source code on binder then you must execute the command above. Toggle the markdown cell to a code cell and run it.

This extension will install a script into the default IPython profile startup that is called each time an IPython session is created.

Uninstall the extension with importnb-install.

Run a notebook as a module

When the default extension is loaded any notebook can be run from the command line. After the importnb extension is created notebooks can be execute from the command line.

ipython -m readme

In the command line context, __file__ == sys.arv[0] and __name__ == '__main__' .

See the deploy step in the travis build.

py.test

importnb installs a pytest plugin when it is setup. Any notebook obeying the py.test discovery conventions can be used in to pytest. This is great because notebooks are generally your first test.

!ipython -m pytest -- src 

Will find all the test notebooks and configurations as pytest would any Python file.

Setup

To package notebooks add recursive-include package_name *.ipynb

Developer

• Source Notebooks
• Transpiled Python Source
• Tests

Format and test the Source Code

    if __name__ == '__main__':
        if globals().get('__file__', None) == __import__('sys').argv[0]:
            print(foo, __import__('sys').argv)
        else:
            from subprocess import call
            from importnb.capture import capture_output
            with capture_output() as out:  __import__('pytest').main("src".split())
            print('plugins'+out.stdout.split('plugins', 1)[-1])
            """Formatting"""
            from pathlib import Path
            from importnb.utils.export import export
            root = 'src/importnb/notebooks/'
            for path in Path(root).rglob("""*.ipynb"""):                
                if 'checkpoint' not in str(path):
                    export(path, Path('src/importnb') / path.with_suffix('.py').relative_to(root))
            call("jupyter nbconvert --to markdown --NbConvertApp.output_files_dir=docs/{notebook_name}_files readme.ipynb".split())

plugins: cov-2.5.1, benchmark-3.1.1, hypothesis-3.56.5, importnb-0.3.0
collecting ... collected 25 items

src/importnb/tests/test_importnb.ipynb::test_single_file_with_context PASSED [  4%]
src/importnb/tests/test_importnb.ipynb::test_from_filename PASSED        [  8%]
src/importnb/tests/test_importnb.ipynb::test_from_execute PASSED         [ 12%]
src/importnb/tests/test_importnb.ipynb::test_with_doctest PASSED         [ 16%]
src/importnb/tests/test_importnb.ipynb::test_from_filename_main PASSED   [ 20%]
src/importnb/tests/test_importnb.ipynb::test_parameterize PASSED         [ 24%]
src/importnb/tests/test_importnb.ipynb::test_commandline PASSED          [ 28%]
src/importnb/tests/test_importnb.ipynb::test_python_file PASSED          [ 32%]
src/importnb/tests/test_importnb.ipynb::test_single_file_with_capture PASSED [ 36%]
src/importnb/tests/test_importnb.ipynb::test_capturer PASSED             [ 40%]
src/importnb/tests/test_importnb.ipynb::test_single_file_with_lazy PASSED [ 44%]
src/importnb/tests/test_importnb.ipynb::test_single_file_without_context XPASS [ 48%]
src/importnb/tests/test_importnb.ipynb::test_single_file_relative 42
xfail  [ 52%]
src/importnb/tests/test_importnb.ipynb::test_single_with_extension PASSED [ 56%]
src/importnb/tests/test_importnb.ipynb::test_package PASSED              [ 60%]
src/importnb/tests/test_importnb.ipynb::test_package_failure xfail       [ 64%]
src/importnb/tests/test_importnb.ipynb::test_package_failure_partial PASSED [ 68%]
src/importnb/tests/test_importnb.ipynb::test_tmp_dir PASSED              [ 72%]
src/importnb/tests/test_unittests.ipynb::TestExtension::test_failure xfail [ 76%]
src/importnb/tests/test_unittests.ipynb::TestExtension::test_import PASSED [ 80%]
src/importnb/tests/test_unittests.ipynb::TestContext::test_import PASSED [ 84%]
src/importnb/tests/test_unittests.ipynb::TestContext::test_reload_with_context PASSED [ 88%]
src/importnb/tests/test_unittests.ipynb::TestPartial::test_exception PASSED [ 92%]
src/importnb/tests/test_unittests.ipynb::TestPartial::test_traceback PASSED [ 96%]
src/importnb/tests/test_unittests.ipynb::TestRemote::test_imports SKIPPED [100%]

========== 20 passed, 1 skipped, 3 xfailed, 1 xpassed in 3.84 seconds ==========

    if __name__ == '__main__':
        try:
            from IPython.display import display, Image
            from IPython import get_ipython
            with capture_output(): 
                get_ipython().system("cd docs && pyreverse importnb -opng -pimportnb")
            display(Image(url='docs/classes_importnb.png', ))
        except: ...

CHANGELOG

0.3.0

• In loaders Notebook, Interactive, Execute, and Parameterize
• Remove Partial, Lazy, and NotebookTest loaders.
• The first Markdown cell imports as a docstrings, permitting doctests on markdown cells.
• Notebook(globals={}) passes global values to the module
• Notebook(dir="..") will change the working directory and path.
• The code is pure python and uses IPython when possible.
• ipython -m importnb nodebook.ipynb runs a notebook.

0.2.9

• Include Partial, Lazy, and NotebookTest loaders.
• Transform markdown cells to literate block strings so they are included in the ast.
  • __doc__'s are extracted from the first markdown cell or normal source code from a code cell.
• Export the python source code with black.
• Notebook.from_filename is a loader for paths and strings.
• Add importnb.nbtest for notebook testing tools..
• Benchmark importnb against existing notebooks.
• Include a watchdog trick to watch tests..
• Extend the project to >= 3.4
• Use nbviewer/github hierachy for the docs.

0.2.4

• Use tox for testing
• Use a source directory folder structure for pytest and tox testing.
• Create a pytest plugin that discovers notebooks as tests. With importnb notebooks can be used as fixtures in pytest.
• Install importnb as an IPython extension.
• Support running notebooks as modules from the ipython command line
• Create a setuptools command to allow notebooks as packages.

0.2.1

• importnb supports notebook inputs from pure python environments. Two compatible compiler were created from IPython and Python

• importnb.Partial works appropriately by improving exceptions.

• All of the IPython magic syntaxes were removed to support Pure Python.

• The generated Python files are formatted with black.

• Tests were added to:

  • Validate the line number in tracebacks
  • Test someone elses notebooks

0.1.4

• Pypi supports markdown long_description with the proper mimetype in long_description_content_type.

0.1.3

• Include the RST files in the MANIFEST.in.

0.1.2 (Unreleased)

• Use RST files to improve the literacy of the pypi description.

0.1.1

• Released on PyPi

0.0.2

• Initial Testing Release