✨qqtools✨

PyPI - Monthly Downloads Python version

A lightweight library, crafted and battle-tested daily by qq, to make PyTorch life a little easier.

It started from the frustration of PyG’s tightly coupled CUDA ecosystem—carefully matching CUDA versions, installing wheel builds from the official index, and repeatedly reinstalling dependencies like torch-scatter whenever anything changed. This project brings back a clean, one-line pip install ... experience, with no need to worry about CUDA compatibility.

I’ve gathered the repetitive parts of my day-to-day work and refined them into this slim utility library. It serves as a unified toolkit for handling data, training, and experiments, designed to keep projects moving fast with cleaner code and smoother workflows.

Built for me, shared for you.

What it includes

At its core, qqtools is a collection of small utilities I use around PyTorch projects:

data containers such as qDict and qData
dataset and dataloader helpers such as qDictDataset and qDictDataloader
small neural network helpers such as qMLP
a lightweight training framework, qpipeline
a command-line experiment queue for Linux, qexp
config and serialization helpers for YAML, JSON, pickle, and LMDB

At the core, it is still a practical toolbox for the repetitive parts around experiments.

Install

# Core install
pip install qqtools

# Full install
pip install qqtools[full]

# If you only want the experiment queue extras:
pip install qqtools[exp]

While some parts still work with torch==1.x, torch>=2.4 is recommended

qDict

qDict is mainly there for cleaner attribute access in batch-like code:

# Instead of dirty dict brackets:
# batch["input_ids"], batch["attention_mask"]

# Use clean attribute access:
batch = qt.qDict({"input_ids": input_ids, "attention_mask": attention_mask})
out = model(batch.input_ids)

Context scope and `qt.use_ctx`

qt.ctx provides a lightweight scoped context. Values set inside with qt.ctx(...) are visible only in that scope and its nested calls, and the outer state is restored automatically when the block exits.

Scope exit restores the previous key bindings. If you intentionally mutate a shared mutable object in place through the live context, that mutation is considered caller-managed behavior and may remain visible outside the block.

import qqtools as qt

with qt.ctx(dim=512):
    print(qt.ctx.dim)  # 512

print(qt.ctx.get("dim"))  # None

@qt.use_ctx is the simplest way to inject context values into a class constructor:

import qqtools as qt


@qt.use_ctx
class AttentionLayer:
    def __init__(self, dim=64, heads=8):
        self.dim = dim
        self.heads = heads


with qt.ctx(dim=512, heads=16):
    layer = AttentionLayer()
    print(layer.dim, layer.heads)  # 512 16

Manual constructor arguments still take precedence over injected context values.

qexp

qexp is a lightweight experiment queue for Linux hosts. It is built around a shared project root, can work on multi-machines with multi-GPUs.

Quick start:

qexp init --shared-root /mnt/share/myproject/.qexp --machine gpu-a
qexp submit --name demo1 -- python train.py -c config1.yaml
qexp submit --name demo2 -- python train.py -c config2.yaml
qexp submit --name demo3 -- python train.py -c config3.yaml
# 3 tasks will be queued and run sequentially

After init, qexp saves the current shared_root and machine as CLI context, so you usually do not need to repeat them on every command.

Python API:

from qqtools.plugins import qexp

task = qexp.submit(
    qexp.load_root_config("/mnt/share/myproject/.qexp", "gpu-a"),
    command=["python", "train.py", "--epochs", "10"],
    name="demo",
)
print(task.task_id)

Note: Run pip install qqtools[exp] before use qexp command.

qpipeline

qpipeline is a minimal training loop scaffold. It doesn't try to be a heavy framework. You write the project-specific model and task logic, and qpipeline handles the repetitive boilerplate: config-driven startup, train/val loops, metric aggregation, and checkpointing.

A tight training entry:

import torch
from qqtools.plugins.qpipeline import prepare_cmd_args, qPipeline
from qqtools.nn import qMLP

class MyTask:
    def __init__(self, args):
        # Your custom data logic goes here
        self.train_loader, self.val_loader = build_loaders(args)

    def batch_forward(self, model, batch):
        return {"pred": model(batch.x)}

    def batch_loss(self, out, batch):
        loss = torch.nn.functional.mse_loss(out["pred"], batch.y)
        return {"loss": (loss, len(batch.y))}

    def batch_metric(self, out, batch):
        mae = (out["pred"] - batch.y).abs().mean()
        return {"mae": (mae, len(batch.y))}

    def post_metric_to_err(self, result):
        return result["mae"]

class MyPipeline(qPipeline):
    @staticmethod
    def prepare_model(args):
        return qMLP([16, 8, 1])

    @staticmethod
    def prepare_task(args):
        return MyTask(args)

if __name__ == "__main__":
    args = prepare_cmd_args()
    pipe = MyPipeline(args, train=True)
    pipe.fit()

Because qpipeline enforces a stable entry contract, it pairs perfectly with qexp for queued execution:

qexp submit -- python entry.py --config configs/train.yaml

For one-off runtime config edits, qpipeline also supports dotted CLI overrides after normal parser handling:

python entry.py \
  --config configs/train.yaml \
  --task.dataloader.eval_batch_size 32 \
  --task.val_split val_ood \
  --runner.fast_dev_run

Configuration follows a standard YAML structure. See qConfig.md for details.

Plugin modules

Under src/qqtools/plugins/, there are also:

qchem - tools for reading and processing quantum chemistry outputs
qpipeline - a training pipeline framework built on top of the core torch utilities
qhyperconnect - an implementation of Hyper-Connection for PyTorch

Test

tox

qexp schema-5

qexp now uses the breaking Experiment Group, Task, and Attempt runtime. Initialize a fresh .qexp root; existing Batch-era roots are rejected rather than migrated.

qexp init --shared-root /path/to/project/.qexp --machine gpu1
qexp submit --group sweep -- python train.py --config a.yaml
qexp batch-submit --group sweep --file runs.yaml
qexp group pause sweep
qexp task retry TASK_ID
qexp task retry TASK_ID --acknowledge-duplicate-risk
qexp clean --task-id TASK_ID --dry-run
qexp clean --older-than-days 30 --limit 100

Cleanup waits for required machines to acknowledge removal of matching local GPU reservations, process manifests, and logs before deleting shared Task and Attempt records. Required machines are the Task home machine, historical Attempt machines, and the machine that prepared cleanup. Pending operations report waiting_ack and the remaining machine names. Cleanup blocks retry, claim, cancel, and offer, and its tombstone permanently reserves the Task ID.

batch-submit is only a bulk-input command and does not create a public Batch identity.

Release files for qqtools 1.2.31

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for qqtools 1.2.31
File	Size	Uploaded
qqtools-1.2.31.tar.gz	211.3 kB	Jul 24, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for qqtools 1.2.31
File	Interpreter	ABI	Platform	Reset
qqtools-1.2.31-py3-none-any.whl 262.5 kB Jul 24, 2026	Python 3	none	any	Details

Total release size: 473.8 kB

Release files / qqtools-1.2.31.tar.gz

Download URL	qqtools-1.2.31.tar.gz
Size	211.3 kB
Tags	Source
SHA-256 checksum How to use checksums	`7ec240a381e39b6319ea90ef79cb798ef21c02057b2b24278670dd60732a3fc8`
BLAKE2b-256 checksum How to use checksums	`69e8af1db7632ba2584957260692608d06b80539d9673ea1db3814a17fdfb22f`
Upload date	Jul 24, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.13.14`

Release files / qqtools-1.2.31-py3-none-any.whl

Download URL	qqtools-1.2.31-py3-none-any.whl
Size	262.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`c0580271745ae1f3ac365f3d2b8b366bceff40272132e40be027c2aba3d75fb2`
BLAKE2b-256 checksum How to use checksums	`e55cff9a75c8a203403625d48b6d36d67b0abc1c7e2514e89152d537fdf2132b`
Upload date	Jul 24, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.13.14`