Challenge utility functions
Sage Bionetworks Challenge Utilities
pip install challengeutils challengeutils -v
Creating Challenge Templates
To begin all challenge infrastructure, you will want to create several Projects, and Teams. This script pulls from a standard DREAM template and creates the Projects and Teams that you will need for a challenge.
challengeutils createchallenge "Challenge Name Here"
For all challenges, you should be editting the staging site and then using the merge script to mirror staging to live site. The script will compare wiki titles between the staging and live site and update the live site with respect to what has changed on the staging site. Note, this is different from copying the wikis. To copy the wikis, please look at synapseutils.
challengeutils mirrorwiki syn12345 syn23456
Querying an evaluation queue
Evaluation queues offer a separate query service from the rest of Synapse. This query function will print the leaderboard in a csv format in standard out. Proceed here to learn more about this query service.
challengeutils query "select objectId, status from evaluation_12345"
Changing submission status
This is a convenience function to change the status of a submission
challengeutils changestatus 1234545 INVALID
Fork and clone this repository
See the Github docs for how to make a copy (a fork) of a repository to your own Github account.
Then, clone the repository to your local machine so you can begin making changes.
Add this repository as an upstream remote on your local git repository so that you are able to fetch the latest commits.
On your local machine make sure you have the latest version of the
git checkout develop git pull upstream develop
The development life cycle
- Pull the latest content from the
developbranch of this central repository (not your fork).
- Create a feature branch which off the
developbranch. If there is a GitHub issue that you are addressing, name the branch after the issue with some more detail (like
- After completing work and testing locally (see below), push to your fork.
- In Github, create a pull request from the feature branch of your fork to the
developbranch of the central repository.
A code maintainer must review and accept your pull request. A code review (which happens with both the contributor and the reviewer present) is required for contributing. This can be performed remotely (e.g., Skype, Hangout, or other video or phone conference).
This package uses semantic versioning for releasing new versions. The version should be updated on the
develop branch as changes are reviewed and merged in by a code maintainer. The version for the package is maintained in the challengeutils/version.py file. A github release should also occur every time
develop is pushed into
master and it should match the version for the package.
Please add tests for new code. These might include unit tests (to test specific functionality of code that was added to support fixing the bug or feature), integration tests (to test that the feature is usable - e.g., it should have complete the expected behavior as reported in the feature request or bug report), or both.
Here's how to run the test suite:
pytest -vs tests/
Tests are also run automatically by Travis on any pull request and are required to pass before merging.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size challengeutils-1.5.2-py3-none-any.whl (47.2 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size challengeutils-1.5.2.tar.gz (36.9 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for challengeutils-1.5.2-py3-none-any.whl