nbdevAuto

Unlike most sites in this collection, this one documents an installable library rather than a topic. nbdevAuto exists because the nbdev release cycle is a fixed sequence of commands typed in the same order every time: export the notebooks, run the tests, clean them, bump the version, commit, push, build the docs. Each step is one nbdev call, and forgetting one produces a confusing failure two steps later.

The library collapses that sequence into single-word shell commands. It is what just upload and just full_upload invoke under the hood for every submodule in the parent Knowledge repo.

Install

pip install nbdevAuto          # the publish-cycle commands and the helpers
pip install 'nbdevAuto[gh]'    # also installs githubkit, which `ghstatus` needs

Requires Python 3.10 or newer.

Use

Every exported function is also a console script, so the common case is a bare word in the terminal at the root of an nbdev project:

upload -m "commit message"    # export, test, clean, then add/commit/push

Or import the helpers into a notebook:

from nbdevAuto.functions import download_search_images, classify_images
from nbdevAuto.pdf import PDFreader

Contents

Page Covers
Functions Dataset and image helpers: reading a list out of a text file, downloading single images or whole search-driven datasets, verifying and resizing what came back, building the folder layout a classifier expects, running a fastai learner over one image, Kaggle competition and dataset shortcuts, and graph, a graphviz.Digraph subclass preloaded with a rounded, filled style
Automate The command line surface: the prep/commit/push pipeline, the GitHub and PyPI release halves, the help output that lists them all, and the one-line-per-stage console grid every command reports through
PDF Reader PDFreader, a class that converts a PDF through pdf2image and renders page ranges inline when you slice it (pdf[0:5])
GitHub ghstatus: reading .gitmodules, resolving a token, and asking the GitHub API about every submodule at once - CI runs, Pages deploys, Pages builds, remote main shas - plus the --report and --audit views, the exit codes that keep “absent” apart from “cannot ask”, and newest_run, which picks a run by run_number because the API’s page order is not a contract
Fleet fleet: running one command across every submodule of a superproject. A walker that reports only what did work and counts the rest, plus the verbs built on it - upload, push, copy, status, update and sync

The Command Line Surface

Installing the package puts these on your PATH. Most are ordinary functions in Automate, exposed as scripts through [project.scripts]; ghstatus comes from GitHub and fleet from Fleet.

Command Does
prep Bump the version, then export, test and clean the notebooks, refreshing _quarto.yml and the README
gacp git add, commit, and push. Without -m the message is built from the staged paths
status Show the working tree state
upload prep then gacp, the one you actually type
gitrelease Bump the minor version, then tag and create a GitHub release
piprelease Build the sdist and wheel, then upload to PyPI with twine
release gitrelease then piprelease
ghstatus Remote status for every submodule of an nbdev superproject, from the GitHub API. --report for the sync view, --audit to check .gitmodules against the repos the account owns, --json for machine output
fleet Run one command across every submodule: `fleet upload
h Print the list of available commands

Both prep and upload take -p to choose which version part to increment; it defaults to 2, the patch, so every upload ships a new version.

piprelease is the manual path. Pushing to main publishes to PyPI on its own: publish.yaml runs after CI passes, and skips quietly when the version is already there.

ghstatus needs the gh extra (pip install 'nbdevAuto[gh]'), which brings in githubkit. It is an extra rather than a dependency because githubkit pulls pydantic, httpx and hishel behind it, which is a lot of weight for anyone who only wants upload. Without it the command exits 3 with an install hint rather than a traceback. It takes its token from $GITHUB_TOKEN / $GH_TOKEN, then gh auth token.

The Output Contract

Every command reports through one grid, defined in Automate and reused by Fleet. It exists because these commands run 19 times in a row, so anything that costs three lines per stage costs sixty over a fleet.

1/6  bump version (part 2)  OK       75ms  0.3.67 -> 0.3.68
2/6  nbdev_export           OK      129ms
3/6  nbdev_test             OK       1.5s  6 notebooks, slowest 01_Automate.ipynb 1.0s

Four rules hold it together, and a change to any stage has to keep them:

  • One stage, one line, inside 88 columns. The columns are fixed: n/total (5), label (22), outcome (4), duration (7), detail (the rest). A unit test renders real stages through a console pinned to that width and asserts every line fits.
  • A summariser returns short strings, not a transcript. _test_summary turns a line per notebook into 6 notebooks, slowest X 1.0s; _push_summary turns nine lines of git progress into a sha range and a size. Anything too long for the detail column moves to an indented continuation line, so a verbose summariser degrades the layout rather than breaking it.
  • A stage that shells out must capture at the file-descriptor level. redirect_stdout only rebinds Python’s sys.stdout, so a child process writing to fd 1 walks straight past it - which is how nbdev_readme used to spill sixteen lines of pandoc metadata into the middle of a run. _quiet swaps fds 1 and 2 for a temp file as well and merges both captures.
  • A green OK has to mean something. _run raises on a non-zero exit rather than returning a code nobody reads, and the failure prints the command’s own output. Before that, a failed git push still reported OK.

On a terminal the label is written first without a newline, so a slow stage is visible while it runs, then rewritten in place. Piped output skips that entirely, because a carriage return in a log file is just noise.

Where It Is Used

The parent Knowledge repo’s justfile calls upload once per submodule. just upload runs it only where the working tree is dirty, just full_upload runs it everywhere. That is the main consumer, so a change to automate.py affects the publish path for every site in the collection.

ghstatus is the other half of that loop. upload pushes to 19 repos and then goes blind: whether each repo’s CI passed and whether its Pages deploy actually landed is invisible from the terminal. just ci, just report and just audit in the parent repo are all thin wrappers around this one command, which answers for the whole fleet in about 1.5 seconds instead of 19 browser tabs.

Releases

This package publishes itself. A publish.yaml workflow runs after CI passes on main and uploads to PyPI through Trusted Publishing, so no API token is stored in the repository. It reads __version__ out of nbdevAuto/__init__.py and asks PyPI whether that version already exists: a new version is published, an existing one is skipped without failing the build.

Because upload bumps the patch version on every push, an ordinary commit made with upload ships a release. A plain git commit leaves the version alone and the workflow skips. gitrelease, which bumps the minor version and creates the GitHub release and tag, stays a deliberate manual step.

Not Covered Yet

  • No tests for the helpers. 01_Automate.ipynb, 03_GitHub.ipynb and 04_Fleet.ipynb all carry unit tests for their pure functions - the fleet ones against a throwaway superproject of real git repos - but the dataset and image helpers in 00_Functions.ipynb are exercised only by being used, so a broken one surfaces in a downstream repo rather than here.
  • Undeclared dependencies. pyproject.toml declares graphviz and fastcore, plus githubkit behind the gh extra. The helpers additionally import fastai, fastbook, fastdownload, PIL, matplotlib, tqdm, pdf2image (with poppler behind it), nbdev and kaggle (which also needs its credentials). All of those are imported lazily inside the function that needs them, so installing the package and running the console scripts still works - only the specific helper you call fails. Install them yourself.
  • core.py is an empty nbdev stub (a single foo) left over from 00_core.ipynb, which no longer exists.
  • CHANGELOG.md is still the nbdev stub (Version 1.0.0 - Initial release) while the package is past 0.3.68. Nothing generates it, and upload bumps the patch on every push, so it would need writing by hand or dropping.
  • The docstrings are one-liners, so the rendered API pages are thin. The notebooks carry the real explanation.

Project links

Data verified by PyPI on Aug 26, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for bthek1 from gravatar.com bthek1

Credits

Author: Benedict Thekkel

GitHub Statistics

Data verified by PyPI on Aug 26, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

Apache-2.0
View SPDX License List

Requires

Python >=3.10

Provides Extra

gh

Tags

nbdev jupyter notebook python

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nbdevauto-0.3.69.tar.gz (40.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

nbdevauto-0.3.69-py3-none-any.whl (38.7 kB view details)

Uploaded Python 3

File details

Details for the file nbdevauto-0.3.69.tar.gz.

File metadata

  • Download URL: nbdevauto-0.3.69.tar.gz
  • Upload date:
  • Size: 40.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for nbdevauto-0.3.69.tar.gz
Algorithm Hash digest
SHA256 398e4c3555ac8a3910db400f35b75a151251d99c981b1594c33553b85ff75d95
MD5 084762885a311d97d0951c07ff520469
BLAKE2b-256 963d48ecb68598f885f0115df819e30584d89d682834b1d15fc094272b20e02b

See more details on using hashes here.

Provenance

The following attestation bundles were made for nbdevauto-0.3.69.tar.gz:

Publisher: publish.yaml on bthek1/nbdevAuto

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nbdevauto-0.3.69-py3-none-any.whl.

File metadata

  • Download URL: nbdevauto-0.3.69-py3-none-any.whl
  • Upload date:
  • Size: 38.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for nbdevauto-0.3.69-py3-none-any.whl
Algorithm Hash digest
SHA256 3bb575a208a5f037d47404411b6f05052f958bbe4524843dd66131c63f42b0c2
MD5 5e2fe6acd84386141e5fce52606946a8
BLAKE2b-256 875565387a01168c54615a07c3b013d3fb8393a869f7e2b6af340a4439b98d75

See more details on using hashes here.

Provenance

The following attestation bundles were made for nbdevauto-0.3.69-py3-none-any.whl:

Publisher: publish.yaml on bthek1/nbdevAuto

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.3.69 This release

2 files

0.3.68

2 files

0.3.67

2 files

0.3.66

2 files

0.3.65

2 files

0.3.64

2 files

0.3.63

2 files

0.3.61

2 files

0.3.60

2 files

0.3.26

2 files

0.3.23

2 files

0.0.130

2 files

0.0.119

2 files

0.0.112

2 files

0.0.110

2 files

0.0.108

2 files

0.0.104

2 files

0.0.102

2 files

0.0.100

2 files

0.0.95

2 files

0.0.93

2 files

0.0.90

2 files

0.0.81

2 files

0.0.76

2 files

0.0.74

2 files

0.0.72

2 files

0.0.70

2 files

0.0.67

2 files

0.0.64

2 files

0.0.62

2 files

0.0.60

2 files

0.0.57

2 files

0.0.52

2 files

0.0.51

2 files

0.0.44

2 files

0.0.41

2 files

0.0.34

2 files

0.0.33

2 files

0.0.32

2 files

0.0.30

2 files

0.0.28

2 files

0.0.27

2 files

0.0.25

2 files

0.0.22

2 files

0.0.21

2 files

0.0.20

2 files

0.0.18

2 files

0.0.17

2 files

0.0.16

2 files

0.0.15

2 files

0.0.14

2 files

0.0.13

2 files

0.0.12

2 files

0.0.11

2 files

0.0.10

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page