Skip to main content

Download kernels from Kaggle.

Project description

kaggle-downloader

A wrapper around the Kaggle CLI to download multiple kernels at once.

Installation

  1. Grab the latest release of Python.
  2. Install this tool with pip install kaggle-downloader.
  3. Follow the steps from the Kaggle API documentation to create an API token. We never access this directly but let the official Kaggle API take care of authentication.

Usage

usage: kaggle-downloader [-h] {competition-refs,kernel-refs,kernels} ...

Download kernels from Kaggle.

positional arguments:
  {competition-refs,kernel-refs,kernels}
    competition-refs    Fetch competition references.
    kernel-refs         Fetch kernel references for a list of competition references.
    kernels             Fetch kernels for a list of kernel references.

optional arguments:
  -h, --help            show this help message and exit

Getting to the actual kernels takes three steps:

  1. Get a list of references to competitions:
    usage: kaggle-downloader competition-refs [-h] -o OUT
    
    optional arguments:
      -h, --help         show this help message and exit
      -o OUT, --out OUT  Output file.
    
  2. Get a list of references to kernels:
    usage: kaggle-downloader kernel-refs [-h] -c COMPETITIONS -e EXCLUDE -o OUT
    
    optional arguments:
      -h, --help            show this help message and exit
      -c COMPETITIONS, --competitions COMPETITIONS
                            File with list of competitions references.
      -e EXCLUDE, --exclude EXCLUDE
                            File with list of competitions references to exclude. Gets updated with competitions as they are processed.
      -o OUT, --out OUT     Output directory.
    
  3. Get kernels themselves:
    usage: kaggle-downloader kernels [-h] -k KERNELS -e EXCLUDE -o OUT
    
    optional arguments:
      -h, --help            show this help message and exit
      -k KERNELS, --kernels KERNELS
                            Directory with files containing a list of kernel references.
      -e EXCLUDE, --exclude EXCLUDE
                            File with list of kernel references to exclude. Gets updated with kernels as they are processed.
      -o OUT, --out OUT     Output directory.
    

Example usage

# Step 1:
kaggle-downloader competition-refs -o data/competition-refs.txt

# Step 2:
kaggle-downloader kernel-refs -c data/competition-refs.txt -e data/excluded-competition-refs.txt -o data/kernel-refs

# Step 3:
kaggle-downloader kernels -k data/kernel-refs -e data/excluded-kernel-refs.txt -o data/kernels

For developers

  1. Grab the latest release of Python.
  2. Install poetry.
  3. Run poetry install in the root of the repository.
  4. Set the Python interpreter in your IDE to the virtual environment that was created.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kaggle-downloader-1.0.2.tar.gz (5.5 kB view hashes)

Uploaded Source

Built Distribution

kaggle_downloader-1.0.2-py3-none-any.whl (6.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page