modular models for efficient ML development
Project description
# modmod
modmod is a library for making *Mod*-ular *Mod*-els. The primary problem that
modmod solves is how to load models at runtime without instantiating them
multiple times; in that respect, it is essentially a dependency injection
system for models.
# Installation
To use modmod, just install it with your package manager in the usual way. If
you use [Pipenv](https://docs.pipenv.org/), you can copy/paste this:
```
pipenv install modmod
```
# Usage
There are two main pieces of modmod: Models and Pools.
A `Pool` is a container for models. A `Model` can be treated like an augmented
function which is a `Model` factory.
Here's an example of defining the simplest possible model:
```
from modmod.model import Model
class AddThings(Model):
def call(self, x: int, y: int) -> int:
return x + y
```
And here is how you would use it:
```
import modmod.pool
pool = modmod.pool.get()
adder = pool.get(AddThings)
z = adder(1, 2)
print(z) # prints 3
```
You can also take a shortcut to get the model:
```
adder = AddThings.get()
```
However, this should never be done inside a model, bceause it will use the
default pool and will have strange side effects if anyone tries to use your
model in a non-default pool.
## Models with initialization
Sometimes a model needs to be initialized to load in data or do other one-time
startup tasks. To do this, you just override the constructor and the `create`
method. Here's an example for stripping stopwords:
```
import nltk
from modmod.model import Model
class RemoveStopwords(Model):
def __init__(self, pool, config, stopwords):
super().__init__(pool, config)
self.stopwords = stopwords
@classmethod
def create(cls, pool, config):
nltk.download('stopwords')
stopwords = nltk.corpus.stopwords.words('english')
stopwords.append('')
stopwords.remove('not')
stopwords.remove('no')
return RemoveStopwords(pool, config, stopwords)
def call(self, words: List[str]) -> List[str]:
return list(filter(lambda w: w not in self.stopwords, words))
```
The `create` method is invoked when you call `RemoveStopwords.get()`.
## Configuring the pool
Every model gets configuration passed into them, and this comes from the pool.
So, if you need configuration, you need to configure the pool.
**Note:** the pool must be configured *before* you get any models, since
configuring it overwrites the existing pool.
To configure the default pool:
```
import modmod.pool
config = {'opt1': 2}
modmod.pool.configure(config)
```
## Non-default Pools
Sometimes you will want separate pools for separate tasks. One example of this
is for unit testing: you may want to test with multiple configurations of the
model. To do this, you can use separate pools.
The first step is to configure the pool:
```
import modmod.pool
poolname = 'my-pool'
config = {'opt1': 2}
modmod.pool.configure(config, poolname)
```
The second step is just to use the pool!
```
import modmod.pool
pool = modmod.pool.get('my-pool')
adder = pool.get(AddThings)
# Equivalent:
adder = AddThings.get('my-pool')
```
# Roadmap
We have a few initiatives on the roadmap. Each of these will be a version bump:
* [ ] Add support for data and model versioning, add support for model training
* [ ] Add hooks for profiling, debugging, caching
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
modmod-0.2.0.tar.gz
(9.1 kB
view hashes)
Built Distribution
modmod-0.2.0-py3-none-any.whl
(9.4 kB
view hashes)