prepare your dataset for finetuning LLMs
Project description
Dataset Preparation for Transformers Fine-tuning
The Dataset Prep Transformers package simplifies the process of preparing datasets for fine-tuning or training various large language models available in the Hugging Face Transformers library. Whether you're using a model from the Hugging Face repository or have your own dataset, this package streamlines the data integration for a seamless training experience.
Features
- Easily integrate your dataset with Hugging Face Transformers models for training or fine-tuning.
- Specify the model repository ID and dataset from the Hugging Face library to automatically fetch and configure the data.
- Seamlessly incorporate your custom dataset by providing it as input to the package.
Installation
You can install the package using pip:
pip install d4train
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file d4train-0.0.2.2.tar.gz.
File metadata
- Download URL: d4train-0.0.2.2.tar.gz
- Upload date:
- Size: 3.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
93b130b3b8229f358e57890b4c83adba95e351a4437f86db341faf0e5e5a9bff
|
|
| MD5 |
bba7bc511e44cc5a9c4f0ed7d18df240
|
|
| BLAKE2b-256 |
b5e83326c2cbc38cc1df84e029e676efa018d9b221d77b2d5534771ef071a116
|
File details
Details for the file d4train-0.0.2.2-py3-none-any.whl.
File metadata
- Download URL: d4train-0.0.2.2-py3-none-any.whl
- Upload date:
- Size: 3.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
fc14ba4a55ce6b8455e69896544077e79588c57f053d79ad403a78eb7959b0ca
|
|
| MD5 |
82e127a4efa63f96f4c3677056efd7ec
|
|
| BLAKE2b-256 |
180570590d6cfc1a51d6750cddf5ee7730f5ca261f0d5eba75f72214cd1b5aaf
|
