Last released Apr 4, 2024
System to ease incremental training of a Huggingface transformer model from a large S3-based dataset
Last released Mar 20, 2024
Lazy-loading HF Datasets sourced from AWS S3 buckets and a chunking text document tokenizer.
Supported by