Word2Vec Trainer
A PyTorch implementation of Word2Vec built as a learning project to understand how word embeddings are trained.
About
I wanted to understand Word2Vec beyond the API, so I implemented the training pipeline myself and packaged it as a Python library. I built this project while learning about Word2Vec, Skip-gram, negative sampling, and word embeddings.
The goal was to implement the main components myself and understand how they work rather than treating Word2Vec as a black box.
The project includes:
- Text preprocessing
- Vocabulary creation
- Training pair generation
- Skip-gram model
- Negative sampling
- Model training with PyTorch
Project Structure
word2vec_trainer/
│
├── pyproject.toml
├── README.md
├── LICENSE
│
├── src/
│ └── word2vec_trainer/
│ ├── dataset_loader.py
│ ├── word2vec_model.py
│ ├── preprocessing.py
│ └── trainer.py
│
├── tests/
│ ├── test_dataset.py
│ ├── test_preprocessing.py
│ ├── test_model.py
│ └── test_trainer.py
│
└── examples/
└── basic_training.py
Training Pipeline
Text without commas or special characters
↓
Preprocessing
↓
Vocabulary
↓
Training Pairs
↓
Negative Sampling
↓
Skip-gram Model
↓
Learned Word Embeddings
Installation
Clone the repository:
git clone https://github.com/AbhishekBiswas12/word2vec_trainer.git
cd word2vec_trainer
Install in editable mode:
python -m pip install -e .
Usage
A basic training example is available in:
examples/basic_training.py
The public API is still evolving as the project develops.
Testing
Run the test suite with:
pytest
AI-Assisted Development
This project was built primarily as a learning exercise.
During development, I occasionally used ChatGPT and Google Colab's coding assistant when I got stuck with implementation details, debugging, or understanding concepts.
I used these tools as learning and development assistance while working to understand the underlying implementation.
Status
🚧 Work in progress
Future improvements include:
- Improve the public API
- Expand test coverage
- Add more examples
- Add embedding evaluation and visualization
- Benchmark different training configurations
- Publish the package to PyPI
References
- Efficient Estimation of Word Representations in Vector Space
- Distributed Representations of Words and Phrases and their Compositionality
- The Illustrated Word2Vec
License
MIT License. See LICENSE for details.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file word2vec_trainer_pytorch-0.1.0.tar.gz.
File metadata
- Download URL: word2vec_trainer_pytorch-0.1.0.tar.gz
- Upload date:
- Size: 10.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
404fb81af5e47fb6cd6c2440be4b393d6f6658a860b1c85d382a5722645bce07
|
|
| MD5 |
e4597432834b7a42756cea40abb6b33f
|
|
| BLAKE2b-256 |
6ec5b55712730eb55d5f0171e66872219c70bea70446e2b7c3545ff799f0201d
|
Provenance
The following attestation bundles were made for word2vec_trainer_pytorch-0.1.0.tar.gz:
Publisher:
pypi-publish.yml on AbhishekBiswas12/word2vec_trainer
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
word2vec_trainer_pytorch-0.1.0.tar.gz -
Subject digest:
404fb81af5e47fb6cd6c2440be4b393d6f6658a860b1c85d382a5722645bce07 - Sigstore transparency entry: 2581921740
- Sigstore integration time:
-
Permalink:
AbhishekBiswas12/word2vec_trainer@3f910bf8f821649ff72d00d3f99c9a34851d7e4c -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/AbhishekBiswas12
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
pypi-publish.yml@3f910bf8f821649ff72d00d3f99c9a34851d7e4c -
Trigger Event:
release
-
Statement type:
File details
Details for the file word2vec_trainer_pytorch-0.1.0-py3-none-any.whl.
File metadata
- Download URL: word2vec_trainer_pytorch-0.1.0-py3-none-any.whl
- Upload date:
- Size: 9.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
b81a4b0f93f64c63bedb0f5415700b7931ab0919189ea02f8cd53e2d59fbcc26
|
|
| MD5 |
e1666e1897a47ffa7313b3622360e5f4
|
|
| BLAKE2b-256 |
8a08effc14553e836163bc4dfdd5aacb8f90f3fd90ecea30701c9375d7338845
|
Provenance
The following attestation bundles were made for word2vec_trainer_pytorch-0.1.0-py3-none-any.whl:
Publisher:
pypi-publish.yml on AbhishekBiswas12/word2vec_trainer
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
word2vec_trainer_pytorch-0.1.0-py3-none-any.whl -
Subject digest:
b81a4b0f93f64c63bedb0f5415700b7931ab0919189ea02f8cd53e2d59fbcc26 - Sigstore transparency entry: 2581921756
- Sigstore integration time:
-
Permalink:
AbhishekBiswas12/word2vec_trainer@3f910bf8f821649ff72d00d3f99c9a34851d7e4c -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/AbhishekBiswas12
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
pypi-publish.yml@3f910bf8f821649ff72d00d3f99c9a34851d7e4c -
Trigger Event:
release
-
Statement type:
