Skip to main content

slp

Python Version

slp is a framework for fast and reproducible development of multimodal models, with emphasis on NLP models.

It started as a collection of scripts and code I wrote / collected during my PhD and it evolves accordingly.

As such, the framework is opinionated and it follows a convention over configuration approach.

A heavy emphasis is put on:

  • Enforcing best practices and reproducibility of experiments
  • Making common things fast at the top-level and not having to go through extensive configuration options
  • Remaining extendable. Extensions and modules for more use cases should be easy to add
  • Out of the box extensive logging and experiment management
  • Separating dirty / scratch code (at the script level) for quick changes and clean / polished code at the library level

This is currently in alpha release under active development, so things may break and new features will be added.

Dependencies

We use Pytorch (1.7) and the following libraries

Installation

You can use slp as an external library by installing from PyPI with

pip install slp

Or you can clone it from github

git clone git@github.com:georgepar/slp

We use poetry for dependency management

When you clone the repo run:

pip install poetry
poetry install

and a clean environment with all the dependencies will be created. You can access it with poetry shell.

Note: Wandb logging is enabled by default. You can either

  • Create an account and run wandb login when you clone the repo in a new machine to store the results in the online managed environment
  • Run wandb offline when you clone the repo to disable remote sync or use the --offline command line argument in your scripts
  • Use one of their self-hosted solutions

Create a new project based on slp

You can use the template at https://github.com/georgepar/cookiecutter-pytorch-slp to create a new project based on slp

pip install cookiecutter poetry
cookiecutter gh:georgepar/cookiecutter-pytorch-slp
# Follow the interactive configuration and a new folder with the project name you provided will appear
cd $PROJECT_NAME
poetry install  # Installs slp and all other dependencies

And you are good to go. Follow the instructions in the README of the new project you created. Happy coding

Contributing

You are welcome to open issues / PRs with improvements and bug fixes.

Since this is mostly a personal project based around workflows and practices that work for me, I don't guarantee I will accept every change, but I'm always open to discussion.

If you are going to contribute, please use the pre-commit hooks under hooks, otherwise the PR will not go through the CI. And never, ever touch requirements.txt by hand, it will automatically be exported from poetry

cat <<EOT >> .git/hooks/pre-commit
#!/usr/bin/env bash

bash hooks/export-requirements-txt
bash hooks/checks
EOT

chmod +x .git/hooks/pre-commit  # Keep an up-to-date requirements.txt and run Linting, typechecking and tests

ln -s $(pwd)/hooks/commit-msg .git/hooks/commit-msg  # Sign-off your commit

Cite

If you use this code for your research, please include the following citation

@ONLINE {,
    author = "Georgios Paraskevopoulos",
    title  = "slp",
    year   = "2020",
    url    = "https://github.com/georgepar/slp"
}

Roadmap

  • Optuna integration for hyperparameter tuning
  • Add dataloaders for popular multimodal datasets
  • Add multimodal architectures
  • Add RIM, DNC and Kanerva machine implementations
  • Write unit tests

Metadata

Release files for slp 1.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for slp 1.2.1
File Size Uploaded Provenance Actions
slp-1.2.1.tar.gz 94.7 kB Without provenance Details for slp-1.2.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for slp 1.2.1
File Interpreter ABI Platform Actions
slp-1.2.1-py3-none-any.whl Python 3 none any Details for slp-1.2.1-py3-none-any.whl

Total release size: 182.8 kB

Release files / slp-1.2.1.tar.gz

Download URL slp-1.2.1.tar.gz
Size 94.7 kB
Tags Source
SHA-256 checksum
How to use checksums
3e948218f4412b20eff96a831b309887eed3a07c6b506cd024f98a0c02dceb96
BLAKE2b-256 checksum
How to use checksums
2ad68e0fa57458188d40c397cbff205ab9c4ef8bb8cadccce8406a44ac6efb5b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.1.7 CPython/3.8.10 Linux/5.8.0-1036-azure

Release files / slp-1.2.1-py3-none-any.whl

Download URL slp-1.2.1-py3-none-any.whl
Size 88.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
9358a166e62e7cce686c21f9eb6d896c2495dca018bd0a19245036e09667fcad
BLAKE2b-256 checksum
How to use checksums
6327209653ec313c76ef801c3496fbcd9c64074c0a9920783f2343efe44a6b11
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.1.7 CPython/3.8.10 Linux/5.8.0-1036-azure

Security for slp 1.2.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page