ood-detection 0.0.1

A tutorial of NBDev, its use cases and making libraries from it.

NBDev Tutorial

nbdev is a notebook-driven development platform. Simply write notebooks with lightweight markup and get high-quality documentation, tests, continuous integration, and packaging for free!

nbdev makes debugging and refactoring your code much easier than in traditional programming environments since you always have live objects at your fingertips. nbdev also promotes software engineering best practices because tests and documentation are first class. In a min or two, you can provide installable libraries to anyone. Even smallest of the code, can be just turned into a library and can be useful for someone.

Tests, docs and code are part of the same context and are co-located. NBDev not only does make the code more approachable, but forcing you to write docs actually forces you to think more about code. In my personal projects that use nbdev, I often refactor my code to be simpler and better after forcing myself to explain it.

What you can do using NBDev

• Searchable, hyperlinked documentation - Documentation is generated automoatically using Quarto and gets hosted on Github pages using the automatic workflows designed when you intialize a repository with nbdev. Documents support LaTeX, are searchable and also supports automatic hyperlinking with other parts of your code.

• Two way sync between notebooks and editors - Using simple commands, the notebook prepares the python scripts which you can also edit in your editors like VSCode and then syncs the notebooks with the new changes.

• Pip and conda installers - Publish packages to PyPI and conda directly from your notebook code. Creates python modules and provides tools to simplify package releases. Python best practices for releasing packages are automatically followed, which are sometimes very difficult to do manually in dead coding environments.

• Testing - Written as part of the notebook cells along with your main code. Using a single command, nbdev runs all the tests in parallel when you prepare your packages. Having tests as part of your main code, makes sure they are updated when changes are made to code and are easily accessible unlike in dead coding enviroments where they are stored seperately and needs a lot of context switching to understand which test case belong to which function.

• Continuous Integration - Automatically create workflows for using Github Actions that runs the tests to make sure all the test cases are passing, rebuilds the docs and host them on Github pages

• Git friendly - Provides Jupyter/Git hooks which cleans unwanted metadata, thus making it easy to compare git diffs. Also, in case of merge conflicts, instead of giving errors like can’t open notebook renders a clean merge conflicts in human readable format.

• Easy Updates - Your Readme, PyPi page, Conda page always stays updated based on the things you write in index.ipynb.

I Like Notebooks - Jeremy Howard

FastAI has built a lot of amazing tools just out of notebooks. Most of there things are built using nbdev. In one of his video I Like Notebooks, Jeremy Howard shares why he likes notebooks with proper reasons on how it’s time to start rethinking the software engineering practices. He gives some really amazing examples on how Jupyter Notebooks coupled with nbdev follows the best practices and are a great way to teach / write technical blogs, share codes, create reproducible issues.

Summary

• Literate Programming - Literate programming is a methodology that combines a programming language with a documentation language, thereby making programs more robust to, more easily maintained, and arguable more fun to write. The main idea is to treat program as a piece of literature, addressed to human beings rather than to a computer. Notebooks supports this by default and acts like a journal you can go through from top to bottom, understanding the thought process of developer along with code and it’s outputs.

• Less chances of errors - Since you can run a small part of code, see it’s output, make plots, visualize images, videos makes it easy to debug the issues and make sure the inputs are correct. In dead coding enviroments, there are no ways to visualize the inputs and outputs which often leads to errors.

• Easily sharable - With notebooks, you can easily share the results, issues with others and they can easily reproduce those using something like a colab enviroment. And, most important thing, you can not only share text but also share images, videos, plots etc. With Software 2.0 we are not just working with text but have a varied kind of data that needs a lot of exploration.

• Tests live along with code - In dead coding environments, it can be very easy to miss out tests completely. They live seperately from the main code. In Nbdev or in general notebooks, the tests live along with the main code.

• Better suggestions - Jupyter notebooks are more helpful, as they are more correct is suggesting functions. VSCode doesn’t know the output of the previous line, but jupyter knows that as you ran the code.

Examples

• FastAI Documentation - The whole documentation is written out of the notebooks. The good thing about this is, the documentation and the tests along with always stay updated with new changes in the library.

• Fastpages - Create technical blogs with latex, images, videos, plots, code snippets directly from your notebooks.

• Fastdoc - Create publication quality books directly from Jupyter Notebooks. The biggest example of this is the book Deep Learning for Coders written completely out of notebooks. This Github repository has the exact notebooks that were used for creating the publication ready book. Best part of writing a book from notebook is that, the example code you add in your book are actual code, that will run and give the correct output, unlike other books that have many errors or dependency issues. Book s available on Amazon with high quality plots, figures, and working code.

Install

pip install ood_detection