A python module to generate word embeddings from tiny data
Project description
nonce2vec
Welcome to Nonce2Vec!
This is the repo accompanying the paper "High-risk learning: acquiring new word vectors from tiny data" (Herbelot & Baroni, 2017). If you use this code, please cite the following:
@InProceedings{herbelot-baroni:2017:EMNLP2017,
author = {Herbelot, Aur\'{e}lie and Baroni, Marco},
title = {High-risk learning: acquiring new word vectors from tiny data},
booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
month = {September},
year = {2017},
address = {Copenhagen, Denmark},
publisher = {Association for Computational Linguistics},
pages = {304--309},
url = {https://www.aclweb.org/anthology/D17-1030}
}
NEW! We have now released v2.0 of Nonce2Vec which is packaged via pip and runs on gensim v3.4.0. This should make it way easier for you to replicate experiments.
Install
pip3 install nonce2vec
Download and extract the required resources
To download the nonces, chimeras and MEN datasets:
wget http://129.194.21.122/~kabbach/noncedef.chimeras.men.7z
To use the pretrained gensim model from Herbelot and Baroni (2017):
wget http://129.194.21.122/~kabbach/wiki_all.model.7z
Generate a pre-trained word2vec model
To generate a gensim.word2vec model from scratch, with :
Use a Wikipedia dump
To use the same Wikipedia dump as Herbelot and Baroni (2017):
wget http://129.194.21.122/~kabbach/wiki.all.utf8.sent.split.lower.7z
Else, to create a new Wikipedia dump from an earlier archive, check out WiToKit.
Train the background model
n2v train \
--data /absolute/path/to/wikipedia/dump \
--outputdir /absolute/path/to/dir/where/to/store/w2v/model \
--alpha 0.025 \
--neg 5 \
--window 5 \
--sample 1e-3 \
--epochs 5 \
--min-count 50 \
--size 400 \
--num-threads number_of_cpu_threads_to_use
--train-mode skipgram
Check the correlation with the MEN dataset
n2v check \
--data /absolute/path/to/MEN/MEN_dataset_natural_form_full
--model /absolute/path/to/gensim/word2vec/model
Test nonce2vec on the nonce definitional dataset
n2v test \
--on nonces \
--model /absolute/path/to/pretrained/w2v/model \
--data /absolute/path/to/nonce.definitions.299.test \
--alpha 1 \
--neg 3 \
--window 15 \
--sample 10000 \
--epochs 1 \
--min-count 1 \
--lambda 70 \
--sample-decay 1.9 \
--window-decay 5 \
--sum-filter random \
--sum-over-set \
--replication
Test nonce2vec on the chimeras dataset
n2v test \
--on chimeras \
--model /absolute/path/to/pretrained/w2v/model \
--data /absolute/path/to/chimeras.dataset.lx.tokenised.test.txt \
--alpha 1 \
--neg 3 \
--window 15 \
--sample 10000 \
--epochs 1 \
--min-count 1 \
--lambda 70 \
--sample-decay 1.9 \
--window-decay 5 \
--sum-filter random \
--sum-over-set \
--replication
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.