gokart

Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.

Documentation for the latest release is hosted on readthedocs.

About gokart

Here are some good things about gokart.

The following meta data for each Task is stored separately in a pkl file with hash value
- task output data
- imported all module versions
- task processing time
- random seed in task
- displayed log
- all parameters set as class variables in the task
Automatically rerun the pipeline if parameters of Tasks are changed.
Support GCS and S3 as a data store for intermediate results of Tasks in the pipeline.
The above output is exchanged between tasks as an intermediate file, which is memory-friendly
pandas.DataFrame type and column checking during I/O
Directory structure of saved files is automatically determined from structure of script
Seeds for numpy and random are automatically fixed
Can code while adhering to SOLID principles as much as possible
Tasks are locked via redis even if they run in parallel

All the functions above are created for constructing Machine Learning batches. Provides an excellent environment for reproducibility and team development.

Here are some non-goal / downside of the gokart.

Batch execution in parallel is supported, but parallel and concurrent execution of task in memory.
Gokart is focused on reproducibility. So, I/O and capacity of data storage can become a bottleneck.
No support for task visualize.
Gokart is not an experiment management tool. The management of the execution result is cut out as Thunderbolt.
Gokart does not recommend writing pipelines in toml, yaml, json, and more. Gokart is preferring to write them in Python.

Getting Started

Within the activated Python environment, use the following command to install gokart.

pip install gokart

If you use S3 or GCS as a data store, install the corresponding extras:

pip install gokart[s3]   # S3 support
pip install gokart[gcs]  # GCS support
pip install gokart[all]  # both S3 and GCS

Quickstart

Minimal Example

A minimal gokart tasks looks something like this:

import gokart

class Example(gokart.TaskOnKart):
    def run(self):
        self.dump('Hello, world!')

task = Example()
output = gokart.build(task)
print(output)

gokart.build return the result of dump by gokart.TaskOnKart. The example will output the following.

Hello, world!

Type-Safe Pipeline Example

We introduce type-annotations to make a gokart pipeline robust. Please check the following example to see how to use type-annotations on gokart. Before using this feature, ensure to enable mypy plugin feature in your project.

import gokart

# `gokart.TaskOnKart[str]` means that the task dumps `str`
class StrDumpTask(gokart.TaskOnKart[str]):
    def run(self):
        self.dump('Hello, world!')

# `gokart.TaskOnKart[int]` means that the task dumps `int`
class OneDumpTask(gokart.TaskOnKart[int]):
    def run(self):
        self.dump(1)

# `gokart.TaskOnKart[int]` means that the task dumps `int`
class TwoDumpTask(gokart.TaskOnKart[int]):
    def run(self):
        self.dump(2)

class AddTask(gokart.TaskOnKart[int]):
    # `a` requires a task to dump `int`
    a: gokart.TaskInstanceParameter[gokart.TaskOnKart[int]] = gokart.TaskInstanceParameter()
    # `b` requires a task to dump `int`
    b: gokart.TaskInstanceParameter[gokart.TaskOnKart[int]] = gokart.TaskInstanceParameter()

    def requires(self):
        return dict(a=self.a, b=self.b)

    def run(self):
        # loading by instance parameter,
        # `a` and `b` are treated as `int`
        # because they are declared as `gokart.TaskOnKart[int]`
        a = self.load(self.a)
        b = self.load(self.b)
        self.dump(a + b)


valid_task = AddTask(a=OneDumpTask(), b=TwoDumpTask())
# the next line will show type error by mypy
# because `StrDumpTask` dumps `str` and `AddTask` requires `int`
invalid_task = AddTask(a=OneDumpTask(), b=StrDumpTask())

This is an introduction to some of the gokart. There are still more useful features.

Please See Documentation .

Have a good gokart life.

Achievements

Gokart is a proven product.

It's actually been used by m3.inc for over 3 years
Natural Language Processing Competition by Nishika.inc 2nd prize : Solution Repository

Thanks

gokart is a wrapper for luigi. Thanks to luigi and dependent projects!

luigi

Metadata

Release files for gokart 1.12.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gokart 1.12.2
File	Size	Uploaded
gokart-1.12.2.tar.gz	86.5 kB	Jun 10, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for gokart 1.12.2
File	Interpreter	ABI	Platform	Reset
gokart-1.12.2-py3-none-any.whl 73.7 kB Jun 10, 2026	Python 3	none	any	Details

Total release size: 160.2 kB

Release files / gokart-1.12.2.tar.gz

Download URL	gokart-1.12.2.tar.gz
Size	86.5 kB
Tags	Source
SHA-256 checksum How to use checksums	`d50917e99f509045a88adc2d2e98baab72c72d47e521c74d21058890720c2b01`
BLAKE2b-256 checksum How to use checksums	`412a07613988133d0db61cc8952dd04c2371e604e4fe8ee95ba39c244bcc2ada`
Upload date	Jun 10, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / gokart-1.12.2-py3-none-any.whl

Download URL	gokart-1.12.2-py3-none-any.whl
Size	73.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`01748d3b7bfe60de54c24b88a13b3c188c7d0965868149a3f1cb1507026cbf68`
BLAKE2b-256 checksum How to use checksums	`80005038e6ea9a1ae1a3f265a9c8829bd8db1cfa6699cbb9f4d354f76a56d830`
Upload date	Jun 10, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`