Skip to main content

Semi-supervised Adaptive Learning Across Domains

Project description

🥗 salad

**S**\ emi-supervised **A**\ daptive **L**\ earning **A**\ cross **D**\ omains

.. figure:: img/domainshift.png

``salad`` is a library to easily setup experiments using the current
state-of-the art techniques in domain adaptation. It features several of
recent approaches, with the goal of being able to run fair comparisons
between algorithms and transfer them to real-world use cases. The
toolbox is under active development and will extended when new
approaches are published.

Contribute on Github: ``_

Currently implements the following techniques (in ``salad.solver``)

- VADA (``VADASolver``),
`arxiv:1802.08735 <>`__
- DIRT-T (``DIRTTSolver``),
`arxiv:1802.08735 <>`__
- Self-Ensembling for Visual Domain Adaptation
`arxiv:1706.05208 <>`__
- Associative Domain Adaptation (``AssociativeSolver``),
`arxiv:1708.00938 <>`__
- Domain Adversarial Training (``DANNSolver``),
`jmlr:v17/15-239.html <>`__
- Generalizing Across Domains via Cross-Gradient Training
`arxiv:1708.00938 <>`__
- Adversarial Dropout Regularization (``AdversarialDropoutSolver``),
` <>`__

Implements the following features (in ``salad.layers``):

- Weights Ensembling using Exponential Moving Averages or Stored
- WalkerLoss and Visit Loss
(`arxiv:1708.00938 <>`__)
- Virtual Adversarial Training
(`arxiv:1704.03976 <>`__)

Coming soon:

- Deep Joint Optimal Transport (``DJDOTSolver``),
`arxiv:1803.10081 <>`__
- Translation based approaches

📊 Benchmarking Results

One of salad's purposes is to constantly track the state of the art of a variety of domain
adaptation algorithms. The latest results can be reproduced by the files in the ``scripts/``

.. figure:: img/benchmarks.svg

💻 Installation

Requirements can be found in ``requirement.txt`` and can be installed

.. code:: bash

pip install -r requirements.txt

Install the package via

.. code:: bash

pip install torch-salad

For the latest development version, install via

.. code:: bash

pip install git+

📚 Using this library

Along with the implementation of domain adaptation routines, this
library comprises code to easily set up deep learning experiments in

This section will be extended upon pre-release.

Quick Start

To get started, the ``scripts/`` directory contains several python scripts
for both running replication studies on digit benchmarks and studies on
a different dataset (toy example: adaptation to noisy images).

.. code:: bash

$ cd scripts
$ python --log ./log --teach --source svhn --target mnist

Refer to the help pages for all options:

.. code::

usage: [-h] [--gpu GPU] [--cpu] [--njobs NJOBS] [--log LOG]
[--epochs EPOCHS] [--checkpoint CHECKPOINT]
[--learningrate LEARNINGRATE] [--dryrun]
[--source {mnist,svhn,usps,synth,synth-small}]
[--target {mnist,svhn,usps,synth,synth-small}]
[--sourcebatch SOURCEBATCH] [--targetbatch TARGETBATCH]
[--seed SEED] [--print] [--null] [--adv] [--vada]
[--dann] [--assoc] [--coral] [--teach]

Domain Adaptation Comparision and Reproduction Study

optional arguments:
-h, --help show this help message and exit
--gpu GPU Specify GPU
--cpu Use CPU Training
--njobs NJOBS Number of processes per dataloader
--log LOG Log directory. Will be created if non-existing
--epochs EPOCHS Number of Epochs (Full passes through the unsupervised
training set)
--checkpoint CHECKPOINT
Checkpoint path
--learningrate LEARNINGRATE
Learning rate for Adam. Defaults to Karpathy's
constant ;-)
--dryrun Perform a test run, without actually training a
--source {mnist,svhn,usps,synth,synth-small}
Source Dataset. Choose mnist or svhn
--target {mnist,svhn,usps,synth,synth-small}
Target Dataset. Choose mnist or svhn
--sourcebatch SOURCEBATCH
Batch size of Source
--targetbatch TARGETBATCH
Batch size of Target
--seed SEED Random Seed
--adv Train a model with Adversarial Domain Regularization
--vada Train a model with Virtual Adversarial Domain
--dann Train a model with Domain Adversarial Training
--assoc Train a model with Associative Domain Adaptation
--coral Train a model with Deep Correlation Alignment
--teach Train a model with Self-Ensembling

Reasons for using solver abstractions

The chosen abstraction style organizes experiments into a subclass of

Quickstart: MNIST Experiment

As a quick MNIST experiment:

.. code:: python

from salad.solvers import Solver

class MNISTSolver(Solver):

def __init__(self, model, dataset, **kwargs):

self.model = model
super().__init__(dataset, **kwargs)

def _init_optims(self, lr = 1e-4, **kwargs):

opt = torch.optim.Adam(self.model.parameters(), lr = lr)

def _init_losses(self):

For a simple tasks as MNIST, the code is quite long compared to other
PyTorch examples `TODO <#>`__.

💡 Domain Adaptation Problems

Legend: Implemented (✓), Under Construction (🚧)

📷 Vision

- Digits: MNIST ↔ SVHN ↔ USPS ↔ SYNTH (✓)
- `VisDA 2018 Openset and Detection <>`__
- Synthetic (GAN) ↔ Real (🚧)
- CIFAR ↔ STL (🚧)
- ImageNet to
`iCubWorld <>`__ (🚧)

🎤 Audio

- `Mozilla Common Voice Dataset <>`__ (🚧)

፨ Neuroscience

- White Noise ↔ Gratings ↔ Natural Images (🚧)
- `Deep Lab Cut Tracking <>`__ (🚧)

🔗 References to open source software

Part of the code in this repository is inspired or borrowed from
original implementations, especially:


Excellent list of domain adaptation ressources: -

👤 Contact

Maintained by `Steffen Schneider <>`__. Work is part
of my thesis project at the `Bethge Lab <>`__. This
README is also available as a webpage at
` <>`__. We
welcome issues and pull requests `to the official github
repository <>`__.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

torch-salad-0.2.0a0.tar.gz (50.3 kB view hashes)

Uploaded source

Built Distribution

torch_salad-0.2.0a0-py3-none-any.whl (82.7 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page