nengo-bones 22.11.15

Tools for managing Nengo projects

NengoBones

The aim of this project is to automate the configuration of repositories throughout the Nengo ecosystem. The idea is that we can centralize the design and maintenance of “meta” project code (such as CI testing infrastructure) in this project, rather than each repository maintaining that code independently.

The basic methodology for this project is a templating system in which there are common templates for meta files that are populated with data in each downstream project. Projects control this templating through the .nengobones.yml configuration file, which defines the information used to fill in the templates.

Wherever possible, we try to do this templating in such a way that downstream projects will be automatically updated when an update is made in nengo-bones. However, some files cannot be updated automatically and require downstream repos to manually run a script to update those files.

Note that this repository itself is configured using the nengo-bones templating system, so if you would like an example of how to use it, check out the source code.

Installation

NengoBones is designed to be used based on the master branch (although we release occasional stable checkpoints to PyPI). This can be installed via

git clone https://github.com/nengo/nengo-bones.git
pip install -e ./nengo-bones

Basic usage

The first step is to fill in the .nengobones.yml configuration file. You can use the one in this repository as a starting point, or see the documentation for more details. This file should be placed in the top level of your project.

All of the manually generated template files can then be rendered by running this command in the same folder as the .nengobones.yml file:

bones generate

See bones generate --help or the documentation for a full list of command line options.

Documentation

• Command line usage

• Demonstration of available configuration options

• API reference

Release History

22.11.15 (November 15, 2022)

Added

• Added support for new one_page option in nengo-sphinx-theme. (#101)

• Added the bones-format-notebook script to apply automated formatting, cleanup, and static checking to Jupyter notebooks. (#32)

• Static script will now check that bones-format-notebook has been applied to all notebooks in the docs directory. (#32)

• Added remote-script.sh CI script for running other CI scripts on a remote device. (#124)

• Added isort configuration to pyproject.toml and check import order during static checks. (#121)

• Officially support and test against Python 3.9. (#136)

• Added skip option to codespell config, which can be used to specify files that should be ignored. (#138)

• Added support for projects with type hints through the py_typed section. (#140)

• Added min_python option to the main section. The default python_requires in setup.py is now based on this, if not overridden. (#140)

• Added support for changing the main branch name with the main_branch config option. (#145)

• Added template for version.py. (#151)

• Added apache license type. (#160)

• Added proprietary license type. (#162)

• Added several actions for use with Github Actions: setup initializes a python environment, generate-and-check runs bones-generate and bones-check, run-script runs a CI script such as static.sh, and coverage-report generates combined coverage reports. (#166)

• Added support for docformatter (and static checks will now enforce that docformatter has been applied). (#168)

• Added support for building and releasing Python projects through the deploy CI script. (#172)

• Added a bones check-deploy script that checks several items that we depend on for releases. (#172)

Changed

• Restrict pylint version to <2.5 due to issue when the package being checked is not installed in the current environment. (#103)

• Increased minimum pylint version to 2.5.1 (#32)

• Documentation [source] links now point to GitHub. (#117)

• Separate tokens, PYPI_TOKEN and PYPI_TEST_TOKEN, must now be specified for deploying to PyPI and TestPyPI, respectively. (#127)

• Builds for this repository now run on TravisCI.com instead of TravisCI.org. (#130)

• Drop support for Python 3.5. (#123)

• Doc script will now install Sphinx>=3.1.2. (#137)

• Codespell ignore words is now specified via the codespell.ignore_words section of setup_cfg (instead of in the static CI script config). As a result, it will now apply to all invocations of codespell (not just from the static script). (#138)

• bones-format-notebook --check will now require that all codespell checks pass. (#138)

• Switched to new version of Codecov uploader and added checksum verification. (#150)

• Increased minimum black version to 21.12b0. (#104, #159)

• License fields in setup.py are now set using license_rst.type. (#161)

• Trove classifiers in setup.py are now sorted alphabetically. (#161)

• Updated Nengo license and renamed to ABR license. (#157)

• The license type is now specified through a license key at the top level. If not specified, it defaults to proprietary. (#157)

• All CI scripts now support running either through Github Actions or TravisCI. (#166)

• Added additional config options to mypy setup.cfg template. (#170)

• Restrict setuptools version to <64 (fixes issues with various code inspection tools like mypy, pylint, IDE integrations). (#173)

• Removed the deploy_dists configuration option for the deploy script, and replaced it with a boolean wheel option. (#172)

• Consolidated all scripts under a single bones command. Subcommands have the same names as before, without the bones- prefix (e.g., bones-generate is now bones generate). (#172)

Removed

• Removed before_script, before_cache, before_deploy, and after_deploy steps from CI scripts (they weren’t actually used anywhere). (#166)

• Removed analytics_id option for docs/conf.py. Create a custom docs/conf.py template and override the analytics block if you need to customize analytics tracking. (#169)

• Removed support for TravisCI in CI scripts. All downstream repositories should switch to use Github Actions. (#172)

Fixed

• Intermittent failures installing miniconda in the remote script have been fixed by retrying the miniconda install if it fails. (#130)

• Fixed an issue in which Codecov would not report on files where a file in a deeper folder shares the same name. (#132)

• Fixed an issue in which CI scripts did not have access to the same configuration options as other templates. (#144)

• Removed pylint disable for bad-continuation as it no longer exists. (#163)

• Removed pylint disable for no-self-use as it is now optional. (#163)

• Static checks will not try to check docs directory if it doesn’t exist. (#165)

• Static checks will no longer assume that CI scripts are in a folder named .ci (it will look for any scripts in the same folder as the static script). (#166)

• Black will now ignore excluded files even if that filename is passed directly to black (e.g. when running through a precommit hook or IDE plugin). (#161)

• Only delete remote build directories more than a day old. (#172)

0.11.1 (April 13, 2020)

Changed

• Rendered documentation will not be uploaded if the html build fails (it will still be uploaded if the linkchecker/doctest builds fail). (#98)

• Rendered documentation will not be uploaded on cron builds. (#98)

• Docs script will now clean up the built doc directory before execution, if it exists (e.g., because the docs job is being rerun). (#96)

0.11.0 (April 13, 2020)

Added

• Downstream repos will now be automatically updated when nengo-bones is updated. (#97)

• Added slack_notifications option to .travis.yml to enable Slack notifications for failing builds. (#97)

Changed

• Will now use nengo-bones and nengo-sphinx-theme master builds (instead of the latest release), to streamline the process of distributing changes to those core repos. (#97)

0.10.0 (March 19, 2020)

Added

• Added autoautosummary_change_modules option (for use with nengo_sphinx_theme.ext.autoautosummary). (#86)

Changed

• Docs script will now use nengo_sphinx_theme.ext.backoff, which adds exponential backoff functionality to Sphinx requests. (#86)

0.9.1 (March 17, 2020)

Fixed

• Fixed deployment tag conditional check in .travis.yml template. (#83)

0.9.0 (January 28, 2020)

Changed

• The bones-check that TravisCI does now prints diffs for easier debugging. (#80)

Fixed

• Fixed an issue with the .travis.yml template caused by the new Jinja2 release. (#80)

0.8.0 (January 10, 2020)

Changed

• The default distribution used in builds is now xenial. (#79)

0.7.3 (January 8, 2020)

Removed

• Removed coverage.py early starting logic. This is no longer necessary as of Nengo 3.0 and causes problems with the new coverage.py 5.0 release. (#78)

0.7.2 (December 2, 2019)

Changed

• Failing to install miniconda in remote.sh is no longer considered a build error (this can occur, for example, when rerunning a build that already has miniconda installed). (#71)

0.7.1 (November 14, 2019)

Added

• Added support for nengo_sphinx_theme.ext.redirects, which can be used to automatically add redirects for renamed documentation pages. (#68)

Fixed

• Added locking to remote.sh script to avoid possible race conditions during cleanup. (#69)

0.7.0 (November 7, 2019)

Added

• Added support for sphinx.ext.doctest, which can be used to automatically test code snippets in docstrings. (#67)

Changed

• Updated the black version used in pre-commit hooks. (#67)

0.6.0 (October 30, 2019)

Added

• Added a remote.sh CI script template for remotely executing commands on an SSH-accessible machine. (#65)

Fixed

• Fixed a crash when a file contained the text “Automatically generated by nengo-bones”, but was not present in the config file. (#61, #66)

0.5.0 (September 3, 2019)

Added

• Added nengo_simulator, nengo_simloader and nengo_neurons options to the pytest section of the setup.cfg template to support testing changes in Nengo 3.0. (#58)

Changed

• The default value for pytest.addopts in setup.cfg has been removed because Nengo 3.0 does not require -p nengo.tests.options. (#58)

0.4.2 (August 8, 2019)

Added

• Added plt_dirname option to the pytest section of the setup.cfg template to set the plot directory for pytest-plt. (#52)

• Added plt_filename_drop option to the pytest section of the setup.cfg template to set pruning patterns for pytest-plt. (#52)

• Added rng_salt option to the pytest section of the setup.cfg template to set the salt for pytest-rng. (#55)

0.4.1 (July 26, 2019)

Added

• Added allclose_tolerances option to the pytest section of the setup.cfg template to set tolerances for pytest-allclose. (#47)

0.4.0 (July 26, 2019)

Added

• Added style guide and release instructions to documentation. (#44)

• Added templates for .pre-commit-config.yaml and pyproject.toml so downstream repositories can easily adopt Black. (#49)

Changed

• We now check that Python source files are autoformatted with Black in the static.sh script. (#49)

• Templates will now be autoformatted with Black during the rendering process, if Black is installed. (#49)

• Take advantage of multiprocessing to speed up pylint static checks. (#49)

• The E203 flake8 check and bad-continuation pylint check are now disabled by default. (#50)

0.3.0 (July 19, 2019)

Added

• The nengo_bones.templates module was added to consolidate code that loads and renders templates. (#45)

Changed

• The docs/conf.py template has been updated for new versions of Nengo Sphinx Theme. (#46)

• static.sh and examples.sh will now check any notebooks in the docs folder (not just docs/examples). (#46)

• bones-check now checks that the content of the generated files matches the expected content, rather than relying on version numbers. This means most files will not need to be regenerated when new NengoBones versions are released, and that bones-check will be sensitive to changes within a dev version. (#45)

Fixed

• The static.sh/examples.sh script will no longer fail if there are no notebooks in the docs folder. (#46)

0.2.1 (May 24, 2019)

Added

• Added codespell_ignore_words option to static.sh.template, which is a list of words that codespell will ignore. (#35)

• Added analytics_id option to docs_conf.py.template, which will enable Google Analytics tracking. (#35)

Changed

• codespell will now ignore _vendor directories. (#36)

Fixed

• Fixed an issue with static.sh.template in which Python files that were not converted from notebooks were deleted. (#16)

0.2.0 (May 15, 2019)

Added

• Added apt_install option that can be set in the jobs section to apt install any custom apt requirements for a job. (#14)

• Added templates for CONTRIBUTING.rst, CONTRIBUTORS.rst, LICENSE.rst, MANIFEST.in, docs/conf.py, setup.cfg, and setup.py (#17)

• Templates will now be automatically loaded from a <repo>/.templates directory if it exists. When overriding existing templates, the built-in templates can be accessed in include and extend tags with the templates/ prefix. (#17)

• Added flake8 to the static check script. (#17)

• Added the bones-pr-number script to predict the next PR number for a repository. This helps when writing a changelog entry before a PR has been made. (#18)

Changed

• The Python version is now specified by the python option (instead of python_version), for consistency with .travis.yml. (#14)

• All nengo-bones scripts now start with bones-, to make them easier to find with autocompletion. generate-bones is now bones-generate, and check-bones is now bones-check. (#18)

Removed

• Removed conda from the CI setup; all installations should be done through pip instead. (#14)

• Removed the --template-dir option from the generate-bones script; use a .templates directory instead. (#17)

Fixed

• Order of templated dicts should now be deterministic for all Python versions. (#14)

0.1.0 (April 15, 2019)

Initial release of NengoBones! Thanks to all of the contributors for making this possible!