mlcontester 0.1.0

A unified Python client for ML competition platforms (Kaggle, Zindi, DrivenData, AIcrowd, and more)

MLContester

A unified Python client for ML competition platforms. Connect to Kaggle, Zindi, DrivenData, AIcrowd, EvalAI, CrunchDAO, Codabench, Hugging Face, Solafune, and more through a single, consistent API.

Features

• 🔐 Unified Authentication - Single interface for all platforms
• 📋 List Competitions - Browse active competitions across platforms
• 🏆 Join Competitions - Participate in challenges programmatically
• 📥 Download Data - Fetch competition datasets with one command
• 📤 Submit Solutions - Upload predictions directly from your code
• 📊 Track Progress - View leaderboards and your submissions
• 💬 Discussions - Read and post in competition forums
• 🖥️ CLI Interface - Command-line tools for quick operations

Supported Platforms

Platform	Status	Features	Notes
Kaggle	✅ Full	All features	Uses official Kaggle API
Zindi	✅ Full	All features	African ML competitions
DrivenData	✅ Full	All features	Social impact competitions
AIcrowd	✅ Full	All features	Research challenges
EvalAI	✅ Full	All features	Open-source evaluation platform
CrunchDAO	⚠️ Partial	List, Download, Leaderboard	Submissions via crunch-cli
Codabench	✅ Full	All features	CodaLab successor, self-hostable
Hugging Face	⚠️ Partial	List, Browse, Discussions	Submission varies by competition
Solafune	✅ Full	All features	Satellite/geospatial ML

Installation

pip install mlcontester

# With Kaggle official API support
pip install mlcontester[kaggle]

# With development dependencies
pip install mlcontester[dev]

Or install from source:

git clone https://github.com/mlcontester/mlcontester.git
cd mlcontester
pip install -e .

Quick Start

Python API

import asyncio
from pathlib import Path
from mlcontester import MLContester, CompetitionStatus

async def main():
    async with MLContester() as client:
        # Authenticate with platforms
        await client.authenticate("kaggle", username="your_username", api_key="your_key")
        await client.authenticate("zindi", email="[e-mail]", password="your_pass")
        
        # List active competitions
        competitions = await client.list_all_competitions(status=CompetitionStatus.ACTIVE)
        for comp in competitions[:5]:
            print(f"[{comp.platform}] {comp.title} - {comp.deadline}")
        
        # Download competition data
        await client.download_data("kaggle", "titanic", Path("./data/titanic"))
        
        # Make a submission
        submission = await client.submit(
            "kaggle",
            "titanic",
            Path("./submission.csv"),
            message="My first submission"
        )
        print(f"Submitted! Score: {submission.public_score}")
        
        # Check leaderboard
        leaderboard = await client.get_leaderboard("kaggle", "titanic")
        for entry in leaderboard[:10]:
            print(f"{entry.rank}. {entry.team_name}: {entry.score}")

asyncio.run(main())

Command Line Interface

# List supported platforms
mlcontester platforms

# Authenticate (uses environment variables)
mlcontester auth kaggle

# List active competitions
mlcontester competitions kaggle --active

# Get competition details
mlcontester info kaggle titanic

# Download data
mlcontester download kaggle titanic -o ./data

# Make a submission
mlcontester submit kaggle titanic submission.csv -m "My submission"

# View leaderboard
mlcontester leaderboard kaggle titanic -n 20

# List your submissions
mlcontester submissions kaggle titanic

Configuration

Environment Variables

Set credentials via environment variables:

# Kaggle
export KAGGLE_USERNAME="your_username"
export KAGGLE_KEY="your_api_key"

# Zindi
export ZINDI_EMAIL="[e-mail]"
export ZINDI_PASSWORD="your_password"
# Or use token
export ZINDI_TOKEN="your_token"

# DrivenData
export DRIVENDATA_USERNAME="your_username"
export DRIVENDATA_PASSWORD="your_password"
# Or use token
export DRIVENDATA_TOKEN="your_token"

# AIcrowd
export AICROWD_API_KEY="your_api_key"

# EvalAI
export EVALAI_AUTH_TOKEN="your_auth_token"
# Or for self-hosted instances
export EVALAI_HOST_URL="https://your-evalai-instance.com"

# CrunchDAO
export CRUNCHDAO_API_KEY="your_api_key"

# Codabench
export CODABENCH_TOKEN="your_token"
# Or for self-hosted instances
export CODABENCH_URL="https://your-codabench-instance.com"

# Hugging Face
export HF_TOKEN="hf_your_access_token"
# Or
export HUGGINGFACE_TOKEN="hf_your_access_token"

# Solafune
export SOLAFUNE_TOKEN="your_token"

Kaggle Configuration

You can also use the standard ~/.kaggle/kaggle.json file:

{
    "username": "your_username",
    "key": "your_api_key"
}

API Reference

MLContester Client

The main client class that aggregates all platform functionality.

from mlcontester import MLContester

async with MLContester() as client:
    # Your code here
    pass

Methods

Method	Description
authenticate(platform, **credentials)	Authenticate with a platform
authenticate_from_env(platform)	Authenticate using environment variables
list_competitions(platform, status, search)	List competitions from a platform
list_all_competitions(status, search)	List from all authenticated platforms
get_competition(platform, competition_id)	Get competition details
join_competition(platform, competition_id)	Join a competition
download_data(platform, competition_id, destination)	Download competition data
submit(platform, competition_id, file_path, message)	Submit a solution
list_submissions(platform, competition_id)	List your submissions
get_leaderboard(platform, competition_id)	Get leaderboard
list_discussions(platform, competition_id)	List discussions
post_discussion(platform, competition_id, title, content)	Post a discussion

Data Models

from mlcontester import Competition, Submission, LeaderboardEntry, Discussion

# Competition
comp.id          # Competition identifier
comp.title       # Competition title
comp.platform    # Platform name
comp.url         # Web URL
comp.status      # CompetitionStatus (ACTIVE, ENDED, UPCOMING)
comp.deadline    # Deadline datetime
comp.prize       # Prize amount
comp.is_joined   # Whether user has joined

# Submission
sub.id           # Submission identifier
sub.filename     # Submitted file name
sub.status       # Submission status
sub.public_score # Public leaderboard score
sub.submitted_at # Submission datetime

# LeaderboardEntry
entry.rank       # Position on leaderboard
entry.team_name  # Team or user name
entry.score      # Score value
entry.entries    # Number of submissions

# Discussion
disc.id           # Discussion identifier
disc.title        # Discussion title
disc.author       # Author name
disc.votes        # Vote count
disc.comments_count # Number of comments

Platform-Specific Notes

Kaggle

• Uses official Kaggle API under the hood
• Some competitions require accepting rules via web interface before joining
• Discussion posting not supported via API (use web interface)

Zindi

• Full API support for all features
• Supports both email/password and token authentication

DrivenData

• Uses session-based authentication
• Some features may require web interface interaction

AIcrowd

• Get your API key from https://www.aicrowd.com/participants/me
• Some challenges use git-based submissions (not yet fully supported)

EvalAI

• Open-source platform for ML evaluation challenges
• Get your auth token from Settings > Auth Token on the EvalAI dashboard
• Supports self-hosted instances via host_url parameter
• Full support for code and remote evaluation challenges

CrunchDAO

• Quantitative research competition platform
• Get your API key from https://hub.crunchdao.com/settings/api
• Full submissions require the crunch-cli tool: pip install crunch-cli
• MLContester provides competition listing, data download, and leaderboard access

Codabench

• Successor to CodaLab, fully open source
• Can be self-hosted - configure with base_url parameter
• Get your token from API settings in your profile
• Full support for phase-based benchmarking competitions

Hugging Face

• ML model hub with competition Spaces and Datasets
• Get your access token from https://huggingface.co/settings/tokens
• Competitions are identified by Space/Dataset paths (e.g., org/competition-name)
• Submission methods vary by competition (check individual competition rules)
• Full discussion support on Spaces and Datasets

Solafune

• Japanese platform specializing in satellite imagery and geospatial ML
• Full API support for all features
• Competitions focus on Earth observation data

Examples

Automated Competition Workflow

import asyncio
from pathlib import Path
from mlcontester import MLContester

async def compete():
    async with MLContester() as client:
        # Setup
        await client.authenticate_from_env()
        
        # Find an interesting competition
        competitions = await client.list_competitions(
            "kaggle",
            status="active",
            search="classification"
        )
        
        comp = competitions[0]
        print(f"Working on: {comp.title}")
        
        # Join and download data
        await client.join_competition("kaggle", comp.id)
        await client.download_data("kaggle", comp.id, Path("./data"))
        
        # ... train your model ...
        
        # Submit predictions
        submission = await client.submit(
            "kaggle",
            comp.id,
            Path("./predictions.csv"),
            "Baseline model v1"
        )
        
        print(f"Score: {submission.public_score}")

asyncio.run(compete())

Cross-Platform Competition Search

async def find_competitions():
    async with MLContester() as client:
        # Authenticate with all platforms
        results = await client.authenticate_from_env()
        print(f"Authenticated: {[k for k, v in results.items() if v]}")
        
        # Search across all platforms
        competitions = await client.list_all_competitions(
            status="active",
            search="nlp"
        )
        
        print(f"Found {len(competitions)} NLP competitions:")
        for comp in competitions:
            print(f"  [{comp.platform}] {comp.title}")
            print(f"    Deadline: {comp.deadline}")
            print(f"    Prize: {comp.prize}")

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Adding a New Platform

1. Create a new file in src/mlcontester/platforms/
2. Implement the BasePlatform interface
3. Add to the registry in src/mlcontester/client.py
4. Add environment variables documentation
5. Write tests

License

MIT License - see LICENSE for details.

Acknowledgments

• Inspired by the ML competition community
• Thanks to all the platforms for providing APIs
• Built with httpx, pydantic, typer, and rich