roundup 2.3.0

A simple-to-use and -install issue-tracking system with command-line, web and e-mail interfaces. Highly customisable.

I’m proud to release version 2.3.0 of the Roundup issue tracker. This release is a bugfix and feature release, so make sure to read docs/upgrading.txt to bring your tracker up to date.

The changes, as usual, include some new features and many bug fixes.

Note that you should run roundup-admin ... migrate to update the database schema version. Do this before you use the web, command-line or mail interface and before any users access the tracker.

You can download it with:

pip download roundup

then unpack and test/install the tarball. Also:

pip install roundup

(preferably in a virtual environment) can be used.

Among the notable improvements from the 2.2.0 release are:

• Dockerfile demo mode implemented. This allows quick evaluation as well as the ability to spin up a configured tracker to customise.

• SQLite backends can use WAL mode to reduce blocking between readers and writers improving concurrent use.

• Redis can be used for session database with SQLite and dbm backends. Provides a major performance improvement.

• roundup-mailgw can use OAUTH authentication to SMTP server. (roundup-mailgw command line options changed as a result.)

• Postgres full text index can now be enabled.

• Modifications to in-reply-to threading when there are multiple matches resulting in more predictable handling of messages.

• Many updates to documentation to make it scannable, useful and work on mobile.

• Admin documentation includes a section on setting up Content Security Policy (CSP) to better secure your Roundup trackers.

• REST now allows rate limiting headers to be accessed by client JavaScript.

• Default number of rounds for PBKDF2 updated to 2M to account for improvements in password crackers and CPU power.

• Support PBKDF2 with SHA512 for password storage to improve resistance to password crackers.

• Deprecate SSHA password hash function.

• roundup-admin reindex can be done in batches to manage load incurred by reindexing.

• roundup-admin can list available templates and their installed locations. This is useful when installing via pip or in a docker container as supporting files are not stored in the usual locations like /usr/share/roundup.

• Crash fixes in detector handling

The file CHANGES.txt has a detailed list of feature additions and bug fixes (53) for each release. The most recent changes from there are at the end of this announcement. Also see the information in doc/upgrading.txt.

If you find bugs, please report them to issues AT roundup-tracker.org or create an account at https://issues.roundup-tracker.org and open a new ticket. If you have patches to fix the issues they can be attached to the email or uploaded to the tracker.

Upgrading

If you’re upgrading from an older version of Roundup you must follow all the “Software Upgrade” guidelines given in the doc/upgrading.txt documentation.

Note that you should run roundup-admin ... migrate for all your trackers to update the database schema version. Do this before you use the web, command-line or mail interface and before any users access the tracker.

Roundup requires Python 2 newer than version 2.7.12 or Python 3 newer than or equal to version 3.6 for correct operation. (Python 3.4 or 3.5 may work, but are not tested.) Note that Python 2 support is being removed from the CI platforms, so you should deploy new trackers with Python 3 and plan on upgrading older trackers from Python 2 to Python 3. See the upgrade guide.

To give Roundup a try, just download (directions above), unpack and run:

python demo.py

then open the url printed by the demo app.

Release info and download page:

https://pypi.org/project/roundup

Source and documentation is available at the website:

https://www.roundup-tracker.org/

Mailing lists - the place to ask questions:

https://sourceforge.net/p/roundup/mailman/

About Roundup

Roundup is a simple-to-use and install issue-tracking system with command-line, web and e-mail interfaces. It is based on the winning design from Ka-Ping Yee in the Software Carpentry “Track” design competition.

Note: Ping is not responsible for this project. The contact for this project is rouilj at users.sourceforge.net. Use this address for security or other sensitive issues. Development discussions occur on the roundup-devel at lists.sourceforge.net mailing list. Tickets can be opened at https://issues.roundup-tracker.org.

Roundup manages a number of issues (with flexible properties such as “description”, “priority”, and so on) and provides the ability to:

1. submit new issues,

2. find and edit existing issues, and

3. discuss issues with other participants.

The system facilitates communication among the participants by managing discussions and notifying interested parties when issues are edited. One of the major design goals for Roundup that it be simple to get going. Roundup is therefore usable “out of the box” with any Python 2.7.2+ (or 3.6+) installation. It doesn’t even need to be “installed” to be operational, though an install script is provided.

It comes with five basic issue tracker templates

• a classic bug/feature tracker

• a more extensive devel tracker for bug/features etc.

• a responsive version of the devel tracker

• a jinja2 version of the devel template (work in progress)

• a minimal skeleton

and supports four database back-ends (anydbm, sqlite, mysql and postgresql).

Recent Changes

From 2.2.0 to 2.3.0

Fixed:

• Updated directions for verifying Roundup distribution using pgp.

• Dockerfile healthcheck fixed so it works when trackers are specified on command line. Also cleanup of unneeded packages. (John Rouillard)

• issue2551224 - Replace dbm db for sessions and otks when using sqlite. New databases are created for session data (db-session) and one time key data (db-otk). The data is ephemeral so no need to migrate. (John Rouillard)

• issue2551223 - Timestamps are truncated in mysql and postgresql for session and otk database tables. Modify db schema to use a numeric type that preserves more significant figures. See upgrading.txt for required steps. (John Rouillard)

• added more testing of BasicDatabase to support use of SQLite for that purpose. Had to fix memory, rdbms and dbm edge cases due to new tests. (John Rouillard)

• issue2551138 - roundup-server with ssl under python2 throws traceback on socket close. Not sure how this got fixed, but after fixing issue2551137 it was not an issue anymore.

• issue2551137 - roundup-server won’t run with ssl under python3 Fixed by using SocketIO and manually adding buffering io and catching SSL.ZeroReturnError indicating SSL has been shut down.

• add caching header for text/javascript in addition to depricated application/javascript. (John Rouillard)

• Enable postgres-fts: fix indexer-common::get_indexer so it returns a postgresql-fts Test code paths in get_indexer. (John Rouillard)

• Fix Postgres native-fts, implement a two phase initialization of the indexer. The native-fts one gets assigned after the database connection is open. (John Rouillard)

• fix crash if postgresql native-fts backend is asked to index content with null bytes. (John Rouillard)

• issue2551232 - modify in-reply-to threading when multiple matches Change how in-reply-to threading works in the mailgw. If there is more than one issue with a matching parent message, fall back to subject matching. See upgrading.txt for details. (John Rouillard)

• issue2551195 - port scripts from optparse to argparse (Ralf Schlatterbeck)

• issue2551246 - mitigation, document how -u doesn’t work for roundup-admin. (John Rouillard)

• Document better that files in the template or static_files directories accessed via @@file are available to any user with the url. (John Rouillard)

• Fix final exception handler in roundup-server to send proper Content-Length header to the client. (John Rouillard)

• Fix traceback if Origin header is missing. (John Rouillard)

• issue2551250: Fix sorting of detectors even if there are two with the same name and priority (can happen if they are created in two different files).

• Fix Traceback when a numeric order attribute is empty (Ralf Schlatterbeck)

• Update some template schema files to assign Register permissions for the Anonymous user. Replaces the old Create permission. (John Rouillard)

• Allow ‘*’ and explicit origins in allowed_api_origins. Only return ‘Access-Control-Allow-Credentials’ when not matching ‘*’. Fixes security issue with rest when using ‘*’. (John Rouillard)

• issue2551263: In REST response expose rate limiting, sunset, allow HTTP headers to calling JavaScript. (John Rouillard)

• issue2551257: When downloading an attached (user supplied file), make sure that an ‘X-Content-Type-Options: nosniff’ header is sent. (John Rouillard)

• issue2551252 - default number of rounds for PKDF2 password increased to 2,000,000. (John Rouillard)

• issue2551251 - migrate/re-encrypt PBKDF2 password if stored password used a smaller number of rounds than set in password_pbkdf2_default_rounds. (John Rouillard)

• upgrade from jquery-3.5.1 to jquery-3.6.3. Update user.help.html to new version. (John Rouillard)

• Dockerfile scanned with hadolint. Fixed multiple issues. (John Rouillard)

• fix crash due to invalid initialization/reset of configuration.py option_validators. Crashed roundup-admin on second command if an option_validator was added by a detector or extension. (John Rouillard)

• Dockerfile uses dumb-init to properly wait for child/zombie processes. Defense against child process starting from detector and becoming a zombie when its roundup-server instance exits. (John Rouillard)

• Move installed frontend/Zope back to frontend/ZRoundup directory. This better identifies the directory when copied into the Zope framework. It also matches existing documentation. (John Rouilard)

• Multiple fixes/updates for installation documentation. Including docker shell/admin/demo mdoes. (John Rouillard)

• Invalid item identifiers passed to REST endpoint return a 404 rather than a 400 error. E.G. /rest/data/issue/issue4 (rather than …/issue/4). (John Rouillard)

• issue2551280 - sorted() method of MultilinkHTMLProperty is broken? (Gabor Nagy report and fix; commit John Rouillard)

Features:

• Add warning about limited Python 2 support lifetime to install and upgrading docs. (John Rouillard)

• Dockerfile supports demo mode for instant gratification 8-). Also supports shell and admin mode (John Rouillard)

• Dockerfile build allows adding additional python packages via pip, setting UID tracker is run under. (John Rouillard)

• issue2551140 - Added redis as a session and otk database for use with anydbm and sqlite primary databases. (John Rouillard)

• issue2550559 - Pretty printing / formatting for Number types. Added pretty(format=’%0.3f’) method to NumberHTMLProperty to print numeric values. If value is None, return empty string otherwise str() of value. (John Rouillard)

• sqlite native-fts backend now uses the stopwords list in config.ini to filter words from queries. (Stopwords are still indexed so that phrase/proximity searches still work.) (John Rouillard)

• sqlite databases use WAL mode when created to improve read concurrency. Existing sqlite database still use rollback journal mode. See upgrading.txt for details. (John Rouillard)

• issue2551233 - create new roundup-admin command “templates” list all template names, location and descriptions. Should help find where /usr/share/roundup/templates is buried during some install mechanisms. Does not need a tracker home to run. (John Rouillard)

• Add OAuth authentication to the mailgw script. Now IMAPS can be used with OAuth as required by several large cloud providers. Move command line processing of the mailgw script to argparse. Note that the command line options of the mailgw have changed, see upgrading.txt for details. (Ralf Schlatterbeck)

• issue2551243: schema-dump.py enhanced with anti-CSRF headers. Flake8 cleanup and python2 support. (John Rouillard)

• issue2551253 - new password hash PBDKF2-SHA512 added. Not available by default. Follow directions in upgrading document to use. (John Rouillard)

• roundup-admin migrate command reports the schema version.

• issue2551262 - the mail gateway subject prefix now allows spaces before/after prefix. Also allow spaces between classname and id number in prefix designator. So “[ issue 23 ] subject” is parsed like “[issue23] subject”. (John Rouillard)

• [doc]: add section on implementing CSP for Roundup to admin doc. (John Rouillard)

• issue2551265 - deprecate SSHA password hash method. Users using SSHA passwords will have their passwords transprently upgraded to PBKDF2 derived hash on next login. (John Rouillard)

• issue2551253 - Modify password PBKDF2 method to use SHA512. New hash function using PBKDF2-SHA512 available. Will be default in future. Directions for upgrading security by using it now is included in upgrading.txt. (John Rouillard)

• issue2551275 - Allow configuring max_children in roundup-server. When using roundup-server in fork mode, allow raising number of forked children above the default of 40. (Original patch by Joseph Myers, config settings and docs by John Rouillard.)

• roundup-admin genconfig does not need a tracker home to run. (John Rouillard)

• issue2551190 - Allow roundup-admin reindex to work in batches. Running roundup-admin -i … reindex issue:1-1000 will reindex the first 1000 issues while reporting any missing issues in the range. Also completion progress is reported when indexing a specific class.

• doc updates: add explanation for SQL code in 1.3.3->1.4.0 upgrade. document schema table in rdbms backends and how to dump/extract version from them. (John Rouillard)