Combine tabular data with text and images using Wide and Deep models in Pytorch
A flexible package to use Deep Learning with tabular data, text and images using wide and deep models.
Companion posts: infinitoml
pytorch-widedeep is based on Google's Wide and Deep Algorithm, Wide & Deep
Learning for Recommender Systems.
In general terms,
pytorch-widedeep is a package to use deep learning with
tabular data. In particular, is intended to facilitate the combination of text
and images with corresponding tabular data using wide and deep models. With
that in mind there are a number of architectures that can be implemented with
just a few lines of code. For details on the main components of those
architectures please visit the
Install using pip:
pip install pytorch-widedeep
Or install directly from github
pip install git+https://github.com/jrzaurin/pytorch-widedeep.git
# Clone the repository git clone https://github.com/jrzaurin/pytorch-widedeep cd pytorch-widedeep # Install in dev mode pip install -e .
Important note for Mac users: at the time of writing (Dec-2020) the latest
torch release is
1.7. This release has some
when running on Mac and the data-loaders will not run in parallel. In
python 3.8, the
multiprocessing library start method
This also affects the data-loaders (for any
torch version) and they will not
run in parallel. Therefore, for Mac users I recommend using
python 3.6 or
torch <= 1.6 (with the corresponding, consistent version of
torch 1.6). I do not want to force this
versioning in the
setup.py file since I expect that all these issues are
fixed in the future. Therefore, after installing
pytorch-widedeep via pip or
directly from github, downgrade
pip install pytorch-widedeep pip install torch==1.6.0 torchvision==0.7.0
None of these issues affect Linux users.
Binary classification with the adult
DeepDense and defaults settings.
</code></pre> <p>Building a wide (linear) and deep model with <code>pytorch-widedeep</code>:</p> <pre><code class="language-python"> import pandas as pd import numpy as np from sklearn.model_selection import train_test_split from pytorch_widedeep import Trainer from pytorch_widedeep.preprocessing import WidePreprocessor, TabPreprocessor from pytorch_widedeep.models import Wide, TabMlp, WideDeep from pytorch_widedeep.metrics import Accuracy # the following 4 lines are not directly related to ``pytorch-widedeep``. I # assume you have downloaded the dataset and place it in a dir called # data/adult/ df = pd.read_csv("data/adult/adult.csv.zip") df["income_label"] = (df["income"].apply(lambda x: ">50K" in x)).astype(int) df.drop("income", axis=1, inplace=True) df_train, df_test = train_test_split(df, test_size=0.2, stratify=df.income_label) # prepare wide, crossed, embedding and continuous columns wide_cols = [ "education", "relationship", "workclass", "occupation", "native-country", "gender", ] cross_cols = [("education", "occupation"), ("native-country", "occupation")] embed_cols = [ ("education", 16), ("workclass", 16), ("occupation", 16), ("native-country", 32), ] cont_cols = ["age", "hours-per-week"] target_col = "income_label" # target target = df_train[target_col].values # wide wide_preprocessor = WidePreprocessor(wide_cols=wide_cols, crossed_cols=cross_cols) X_wide = wide_preprocessor.fit_transform(df_train) wide = Wide(wide_dim=np.unique(X_wide).shape, pred_dim=1) # deeptabular tab_preprocessor = TabPreprocessor(embed_cols=embed_cols, continuous_cols=cont_cols) X_tab = tab_preprocessor.fit_transform(df_train) deeptabular = TabMlp( mlp_hidden_dims=[64, 32], column_idx=tab_preprocessor.column_idx, embed_input=tab_preprocessor.embeddings_input, continuous_cols=cont_cols, ) # wide and deep model = WideDeep(wide=wide, deeptabular=deeptabular) # train the model trainer = Trainer(model, objective="binary", metrics=[Accuracy]) trainer.fit( X_wide=X_wide, X_tab=X_tab, target=target, n_epochs=5, batch_size=256, val_split=0.1, ) # predict X_wide_te = wide_preprocessor.transform(df_test) X_tab_te = tab_preprocessor.transform(df_test) preds = trainer.predict(X_wide=X_wide_te, X_tab=X_tab_te) # save and load trainer.save_model("model_weights/model.t")
Of course, one can do much more. See the Examples folder, the documentation or the companion posts for a better understanding of the content of the package and its functionalities.
This library takes from a series of other libraries, so I think it is just fair to mention them here in the README (specific mentions are also included in the code).
TextProcessor class in this library uses the
Vocab. The code at
utils.fastai_transforms is a minor
adaptation of their code so it functions within this library. To my experience
Tokenizer is the best in class.
ImageProcessor class in this library uses code from the fantastic Deep
Learning for Computer
(DL4CV) book by Adrian Rosebrock.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size pytorch_widedeep-0.4.8-py3-none-any.whl (101.2 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size pytorch-widedeep-0.4.8.tar.gz (78.5 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for pytorch_widedeep-0.4.8-py3-none-any.whl