Pytorch-Dataset
A PyTorch Code Dataset for Cutting-Edge Fine-tuning
Installation
You can install the package using pip
pip install pytorch-dataset
Usage
Downloader that downloads and unzips each repository in an account
from pytorch import GitHubRepoDownloader
downloader = GitHubRepoDownloader(username="lucidrains", download_dir="lucidrains_repositories")
downloader.download_repositories()
Processor that cleans, formats, and submits the cleaned dataset to huggingface
from pytorch import CodeDatasetBuilder
code_builder = CodeDatasetBuilder("lucidrains_repositories")
code_builder.save_dataset("lucidrains_python_code_dataset")
code_builder.push_to_hub("lucidrains_python_code_dataset", organization="kye")
License
MIT
1 maintainer
Metadata
1 maintainer
Release files for pytorch-dataset 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| pytorch_dataset-0.0.4.tar.gz | 4.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| pytorch_dataset-0.0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 10.5 kB
Release files / pytorch_dataset-0.0.4.tar.gz
| Download URL | pytorch_dataset-0.0.4.tar.gz |
|---|---|
| Size | 4.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
376f68737a66b4cb67b4446ab8c415206e22596f6023e8bac8537669ff2b1b65
|
|
BLAKE2b-256 checksum How to use checksums |
128c0ccc030c799d5e8f93cd4cc4116061649575938a14237ef0765ec0cb895a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
Release files / pytorch_dataset-0.0.4-py3-none-any.whl
| Download URL | pytorch_dataset-0.0.4-py3-none-any.whl |
|---|---|
| Size | 5.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
41b98a61c71ca72d772c9d783ff851b8de14f4773fe1579ec9952cfa7daeed6a
|
|
BLAKE2b-256 checksum How to use checksums |
743058bc378d038083c0b0499b5f23285faf17ffbd2aa367d76dd5310244238c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
