Pytorch Dataset - Pytorch
Project description
Pytorch-Dataset
A PyTorch Code Dataset for Cutting-Edge Fine-tuning
Installation
You can install the package using pip
pip install pytorch-dataset
Usage
Downloader that downloads and unzips each repository in an account
from pytorch import GitHubRepoDownloader
downloader = GitHubRepoDownloader(username="lucidrains", download_dir="lucidrains_repositories")
downloader.download_repositories()
Processor that cleans, formats, and submits the cleaned dataset to huggingface
from pytorch import CodeDatasetBuilder
code_builder = CodeDatasetBuilder("lucidrains_repositories")
code_builder.save_dataset("lucidrains_python_code_dataset")
code_builder.push_to_hub("lucidrains_python_code_dataset", organization="kye")
License
MIT
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pytorch_dataset-0.0.4.tar.gz
(4.6 kB
view hashes)
Built Distribution
Close
Hashes for pytorch_dataset-0.0.4-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 41b98a61c71ca72d772c9d783ff851b8de14f4773fe1579ec9952cfa7daeed6a |
|
| MD5 | 78f3644d7a2f24c23a2882367d27a306 |
|
| BLAKE2b-256 | 743058bc378d038083c0b0499b5f23285faf17ffbd2aa367d76dd5310244238c |
