New version checker for software
Project description
nvchecker (short for new version checker) is for checking if a new version of some software has been released.
Contents
Dependency
Python 3.5+
Python library: structlog
One of these Python library combinations (ordered by preference):
tornado + pycurl
aiohttp
tornado
All commands used in your version source files
Install and Run
To install:
pip3 install nvchecker
To use the latest code, you can also clone this repository and run:
python3 setup.py install
To see available options:
nvchecker --help
Run with one or more software version source files:
nvchecker source_file
You normally will like to specify some “version record files”; see below.
JSON logging
With --logger=json or --logger=both, you can get a structured logging for programmatically consuming. You can use --json-log-fd=FD to specify the file descriptor to send logs to (take care to do line buffering). The logging level option (-l or --logging) doesn’t take effect with this.
The JSON log is one JSON string per line. The following documented events and fields are stable, undocumented ones may change without notice.
- event=updated
An update is detected. Fields name, old_version and version are available. old_version maybe null.
- event=up-to-date
There is no update. Fields name and version are available.
- event=no-result
No version is detected. There may be an error. Fields name is available.
- level=error
There is an error. Fields name and exc_info may be available to give further information.
Version Record Files
Version record files record which version of the software you know or is available. They are simple key-value pairs of (name, version) separated by a space:
fcitx 4.2.7 google-chrome 27.0.1453.93-200836 vim 7.3.1024
Say you’ve got a version record file called old_ver.txt which records all your watched software and their versions, as well as some configuration entries. To update it using nvchecker:
nvchecker source.ini
See what are updated with nvcmp:
nvcmp source.ini
Manually compare the two files for updates (assuming they are sorted alphabetically; files generated by nvchecker are already sorted):
comm -13 old_ver.txt new_ver.txt
# or say that in English:
comm -13 old_ver.txt new_ver.txt | awk '{print $1 " has updated to version " $2 "."}'
# show both old and new versions
join old_ver.txt new_ver.txt | awk '$2 != $3'
The nvtake Command
This command helps to manage version record files. It reads both old and new version record files, and a list of names given on the commandline. It then update the versions of those names in the old version record file.
This helps when you have known (and processed) some of the updated software, but not all. You can tell nvchecker that via this command instead of editing the file by hand.
This command will help most if you specify where you version record files are in your config file. See below for how to use a config file.
Version Source Files
The software version source files are in ini format. Section names is the name of the software. Following fields are used to tell nvchecker how to determine the current version of that software.
See sample_source.ini for an example.
Configuration Section
A special section named __config__ is special, it provides some configuration options.
Relative path are relative to the source files, and ~ and environmental variables are expanded.
Currently supported options are:
- oldver
Specify a version record file containing the old version info.
- newver
Specify a version record file to store the new version info.
- proxy
The HTTP proxy to use. The format is proto://host:port, e.g. http://localhost:8087.
- max_concurrent
Max number of concurrent jobs. Default: 20.
- keyfile
Specify an ini config file containing key (token) information. This file should contain a keys section, mapping key names to key values. See specific source for the key name(s) to use.
Global Options
The following options apply to all check sources.
- prefix
Strip the prefix string if the version string starts with it. Otherwise the version string is returned as-is.
- from_pattern, to_pattern
Both are Python-compatible regular expressions. If from_pattern is found in the version string, it will be replaced with to_pattern.
- missing_ok
Suppress warnings and errors if a version checking module finds nothing. Currently only regex supports it.
If both prefix and from_pattern/to_pattern are used, from_pattern/to_pattern are ignored. If you want to strip the prefix and then do something special, just use from_pattern`/to_pattern. For example, the transformation of v1_1_0 => 1.1.0 can be achieved with from_pattern = v(\d+)_(\d+)_(\d+) and to_pattern = \1.\2.\3.
List Options
The following options apply to sources that return a list. See individual source sections to determine whether they are supported.
- include_regex
Only consider version strings that match the given regex. The whole string should match the regex. Be sure to use .* when you mean it!
- exclude_regex
Don’t consider version strings that match the given regex. The whole string should match the regex. Be sure to use .* when you mean it! This option has higher precedence that include_regex; that is, if matched by this one, it’s excluded even it’s also matched by include_regex.
- sort_version_key
Sort the version string using this key function. Choose between parse_version and vercmp. Default value is parse_version. parse_version use pkg_resources.parse_version. vercmp use pyalpm.vercmp.
- ignored
Version strings that are explicitly ignored, separated by whitespace. This can be useful to avoid some known mis-named versions, so newer ones won’t be “overridden” by the old broken ones.
Search in a Webpage
Search through a specific webpage for the version string. This type of version finding has these fields:
- url
The URL of the webpage to fetch.
- encoding
(Optional) The character encoding of the webpage, if latin1 is not appropriate.
- regex
A regular expression used to find the version string.
It can have zero or one capture group. The capture group or the whole match is the version string.
When multiple version strings are found, the maximum of those is chosen.
- proxy
The HTTP proxy to use. The format is host:port, e.g. localhost:8087.
- user_agent
The User-Agent header value to use. Use something more like a tool (e.g. curl/7.40.0) in Europe or the real web page won’t get through because cookie policies (SourceForge has this issue).
- sort_version_key
Sort the version string using this key function. Choose between parse_version and vercmp. Default value is parse_version. parse_version use pkg_resources.parse_version. vercmp use pyalpm.vercmp.
This source supports list options.
Find with a Command
Use a shell command line to get the version. The output is striped first, so trailing newlines do not bother.
- cmd
The command line to use. This will run with the system’s standard shell (i.e. /bin/sh).
Check AUR
Check Arch User Repository for updates.
- aur
The package name in AUR. If empty, use the name of software (the section name).
- strip-release
Strip the release part.
- use_last_modified
Append last modified time to the version.
Check GitHub
Check GitHub for updates. The version returned is in date format %Y%m%d.%H%M%S, e.g. 20130701.012212, unless use_latest_release or use_max_tag is used. See below.
- github
The github repository, with author, e.g. lilydjwg/nvchecker.
- branch
Which branch to track? Default: master.
- path
Only commits containing this file path will be returned.
- use_latest_release
Set this to true to check for the latest release on GitHub.
GitHub releases are not the same with git tags. You’ll see big version names and descriptions in the release page for such releases, e.g. zfsonlinux/zfs’s, and those small ones like nvchecker’s are only git tags that should use use_max_tag below.
Will return the release name instead of date.
- use_latest_tag
Set this to true to check for the latest tag on GitHub.
This requires a token because it’s using the v4 GraphQL API.
- query
When use_latest_tag is true, this sets a query for the tag. The exact matching method is not documented by GitHub.
- use_max_tag
Set this to true to check for the max tag on GitHub. Unlike use_latest_release, this option includes both annotated tags and lightweight ones, and return the largest one sorted by the sort_version_key option. Will return the tag name instead of date.
- max_page
How many pages do we search for the max tag? Default is 1. This works when use_max_tag is set.
However, with current API in use, GitHub seems to always return all data in one page, making this option obsolete.
- proxy
The HTTP proxy to use. The format is host:port, e.g. localhost:8087.
- include_tags_pattern, ignored_tags, sort_version_key
Deprecated. Use list options instead.
An environment variable NVCHECKER_GITHUB_TOKEN or a key named github can be set to a GitHub OAuth token in order to request more frequently than anonymously.
This source supports list options when use_max_tag is set.
Check BitBucket
Check BitBucket for updates. The version returned is in date format %Y%m%d, e.g. 20130701, unless use_max_tag is used. See below.
- bitbucket
The bitbucket repository, with author, e.g. lilydjwg/dotvim.
- branch
Which branch to track? Default is the repository’s default.
- use_max_tag
Set this to true to check for the max tag on BitBucket. Will return the biggest one sorted by pkg_resources.parse_version. Will return the tag name instead of date.
- ignored_tags, sort_version_key
Deprecated. Use list options instead.
- max_page
How many pages do we search for the max tag? Default is 3. This works when use_max_tag is set.
This source supports list options when use_max_tag is set.
Check GitLab
Check GitLab for updates. The version returned is in date format %Y%m%d, e.g. 20130701, unless use_max_tag is used. See below.
- gitlab
The gitlab repository, with author, e.g. Deepin/deepin-music.
- branch
Which branch to track? Default: master.
- use_max_tag
Set this to true to check for the max tag on GitLab. Will return the biggest one sorted by pkg_resources.parse_version. Will return the tag name instead of date.
- host
Hostname for self-hosted GitLab instance.
- token
GitLab authorization token used to call the API.
- ignored_tags, sort_version_key
Deprecated. Use list options instead.
To set an authorization token, you can set:
a key named gitlab_{host} in the keyfile (where host is formed the same as the environment variable, but all lowercased).
an environment variable NVCHECKER_GITLAB_TOKEN_{host} must provide that token. The host part is the uppercased version of the host setting, with dots (.) and slashes (/) replaced by underscores (_), e.g. NVCHECKER_GITLAB_TOKEN_GITLAB_COM.
the token option
This source supports list options when use_max_tag is set.
Check PyPI
Check PyPI for updates.
- pypi
The name used on PyPI, e.g. PySide.
- use_pre_release
Whether to accept pre release. Default is false.
Check RubyGems
Check RubyGems for updates.
- gems
The name used on RubyGems, e.g. sass.
Check NPM Registry
Check NPM Registry for updates.
- npm
The name used on NPM Registry, e.g. coffee-script.
Check Hackage
Check Hackage for updates.
- hackage
The name used on Hackage, e.g. pandoc.
Check CPAN
Check MetaCPAN for updates.
- cpan
The name used on CPAN, e.g. YAML.
- proxy
The HTTP proxy to use. The format is host:port, e.g. localhost:8087.
Check Packagist
Check Packagist for updates.
- packagist
The name used on Packagist, e.g. monolog/monolog.
Check Local Pacman Database
This is used when you run nvchecker on an Arch Linux system and the program always keeps up with a package in your configured repositories for Pacman.
- pacman
The package name to reference to.
- strip-release
Strip the release part.
Check Arch Linux official packages
This enables you to track the update of Arch Linux official packages, without needing of pacman and an updated local Pacman databases.
- archpkg
Name of the Arch Linux package.
- strip-release
Strip the release part, only return part before -.
- provided
Instead of the package version, return the version this package provides. Its value is what the package provides, and strip-release takes effect too. This is best used with libraries.
Check Debian Linux official packages
This enables you to track the update of Debian Linux official packages, without needing of apt and an updated local APT database.
- debianpkg
Name of the Debian Linux source package.
- suite
Name of the Debian release (jessie, wheezy, etc, defaults to sid)
- strip-release
Strip the release part.
Check Ubuntu Linux official packages
This enables you to track the update of Ubuntu Linux official packages, without needing of apt and an updated local APT database.
- ubuntupkg
Name of the Ubuntu Linux source package.
- suite
Name of the Ubuntu release (xenial, zesty, etc, defaults to None, which means no limit on suite)
- strip-release
Strip the release part.
Check Repology
This enables you to track updates from Repology (repology.org).
- repology
Name of the project to check.
- repo
Check the version in this repo. This field is required.
Check Anitya
This enables you to track updates from Anitya (release-monitoring.org).
- anitya
distro/package, where distro can be a lot of things like “fedora”, “arch linux”, “gentoo”, etc. package is the package name of the chosen distribution.
Check Android SDK
This enables you to track updates of Android SDK packages listed in sdkmanager --list.
- android_sdk
The package path prefix. This value is matched against the path attribute in all <remotePackage> nodes in an SDK manifest XML. The first match is used for version comparisons.
- repo
Should be one of addon or package. Packages in addon2-1.xml use addon and packages in repository2-1.xml use package.
Check Sparkle framework
This enables you to track updates of macOS applications which using Sparkle framework.
- sparkle
The url of the sparkle appcast.
Manually updating
This enables you to manually specify the version (maybe because you want to approve each release before it gets to the script).
- manual
The version string.
Version Control System (VCS) (git, hg, svn, bzr)
Check a VCS repo for new commits. The version returned is currently not related to the version of the software and will increase whenever the referred VCS branch changes. This is mainly for Arch Linux.
- vcs
The url of the remote VCS repo, using the same syntax with a VCS url in PKGBUILD (Pacman’s build script). The first VCS url found in the source array of the PKGBUILD will be used if this is left blank. (Note: for a blank vcs setting to work correctly, the PKGBUILD has to be in a directory with the name of the software under the path where nvchecker is run. Also, all the commands, if any, needed when sourcing the PKGBUILD need to be installed).
- use_max_tag
Set this to true to check for the max tag. Currently only supported for git. This option returns the biggest tag sorted by pkg_resources.parse_version.
- ignored_tags
Ignore certain tags while computing the max tag. Tags are separate by whitespaces. This option must be used together with use_max_tag. This can be useful to avoid some known badly versioned tags, so the newer tags won’t be “overridden” by the old broken ones.
This source supports list options when use_max_tag is set.
Other
More to come. Send me a patch or pull request if you can’t wait and have written one yourself :-)
Bugs
Finish writing results even on Ctrl-C or other interruption.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
