Challenge utility functions
Project description
Sage Bionetworks Challenge Utilities
The challengeutils
package provides an interface for managing Sage Bionetworks Challenges administered using Synapse, including DREAM Challenges. This package is being actively developed and maintained by DREAM and Informatics & Biocomputing (IBC), Computational Oncology Group at Sage Bionetworks.
Automation | Status |
---|---|
pypi | |
travis |
Install
pip install challengeutils
challengeutils -v
Usage
Below is documentation for some of the key features in the challengeutils
command line client. The documentation can be found here.
challengeutils -h
Creating Challenge Templates
To begin all challenge infrastructure, you will want to create several Projects, and Teams. This script pulls from a standard DREAM template and creates the Projects and Teams that you will need for a challenge.
challengeutils createchallenge "Challenge Name Here"
Mirroring wikis
For all challenges, you should be editting the staging site and then using the merge script to mirror staging to live site. The script will compare wiki titles between the staging and live site and update the live site with respect to what has changed on the staging site. Note, this is different from copying the wikis. To copy the wikis, please look at synapseutils.
challengeutils mirrorwiki syn12345 syn23456
Querying an evaluation queue
Evaluation queues offer a separate query service from the rest of Synapse. This query function will print the leaderboard in a csv format in standard out. Proceed here to learn more about this query service.
challengeutils query "select objectId, status from evaluation_12345"
Changing submission status
This is a convenience function to change the status of a submission
challengeutils changestatus 1234545 INVALID
Contributing
Fork and clone this repository
See the Github docs for how to make a copy (a fork) of a repository to your own Github account.
Then, clone the repository to your local machine so you can begin making changes.
Add this repository as an upstream remote on your local git repository so that you are able to fetch the latest commits.
On your local machine make sure you have the latest version of the develop
branch:
git checkout develop
git pull upstream develop
The development life cycle
- Pull the latest content from the
develop
branch of this central repository (not your fork). - Create a feature branch which off the
develop
branch. If there is a GitHub issue that you are addressing, name the branch after the issue with some more detail (likeissue-123-add-some-new-feature
). - After completing work and testing locally (see below), push to your fork.
- In Github, create a pull request from the feature branch of your fork to the
develop
branch of the central repository.
A code maintainer must review and accept your pull request. A code review (which happens with both the contributor and the reviewer present) is required for contributing. This can be performed remotely (e.g., Skype, Hangout, or other video or phone conference).
This package uses semantic versioning for releasing new versions. The version should be updated on the develop
branch as changes are reviewed and merged in by a code maintainer. The version for the package is maintained in the challengeutils/version.py file. A github release should also occur every time develop
is pushed into master
and it should match the version for the package.
Testing
Please add tests for new code. These might include unit tests (to test specific functionality of code that was added to support fixing the bug or feature), integration tests (to test that the feature is usable - e.g., it should have complete the expected behavior as reported in the feature request or bug report), or both.
This package uses pytest
to run tests. The test code is located in the test subdirectory.
Here's how to run the test suite:
pytest -vs tests/
Tests are also run automatically by Travis on any pull request and are required to pass before merging.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for challengeutils-1.6.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 355d6dc2e8706f6505804ab1e36965146419dde2e8bcee7e75e49e4ef540835a |
|
MD5 | fce992b0e4f0ac2433498d2211e11b5b |
|
BLAKE2b-256 | 8a51f9afd09aba5630e3d31cca19d10ea12cb6a5e0ca130a8f0277b68aefc3f0 |