gcsfs
GCSFS is a Python library that provides a familiar, file-system-like interface to Google Cloud Storage (GCS). Built on top of fsspec, it allows you to interact with cloud buckets as if they were local directories, making it a favorite for data scientists and engineers.
Getting Started
Installation
Install via pip or conda:
# Using pip
pip install gcsfs
# OR using conda
conda install -c conda-forge gcsfs
Basic Usage
import gcsfs
# Initialize the filesystem
fs = gcsfs.GCSFileSystem(project='my-google-project')
# List files in a bucket
files = fs.ls('my-bucket')
# Read a file directly into a string/bytes
with fs.open('my-bucket/data.txt', 'rb') as f:
content = f.read()
Specialized Bucket Support
GCSFS now automatically supports advanced Google Cloud Storage features through its ExtendedFileSystem implementation.
1. Hierarchical Namespace (HNS)
Hierarchical Namespace (HNS) replaces the traditional "flat" GCS structure with true logical directories.
- Atomic Renames: Moving or renaming a directory is an
O(1)metadata operation. No more slow "copy-then-delete" for large folders. - High Performance: Offers up to 8x higher initial Queries Per Second (QPS) for read/write operations.
- AI/ML Ready: Ideal for heavy checkpointing and managing millions of small files.
2. Rapid Buckets (Zonal Storage)
Rapid Buckets are zonal storage resources designed for ultra-low latency and maximum throughput.
- Zonal Co-location: Place your data in the same zone as your GPU/TPU clusters to minimize network lag.
- True Appends: Unlike standard GCS objects, you can append data to existing objects in Rapid buckets without a full rewrite.
- Streaming I/O: Optimized for high-speed model loading and real-time logging.
Integration & Auth
GCSFS plays nicely with the rest of the Python data ecosystem.
Authentication Modes
- Default: Uses your local gcloud credentials or environment service accounts.
- Cloud: Explicitly use Google Metadata service (
token='cloud'). - Anonymous: Access public data without a login (
token='anon'). - Service Account: Pass the path to your JSON key file (
token='path/to/key.json').
[!TIP] Note on Async: GCSFS is built on
aiohttp. If you are building high-concurrency applications, you can use the asynchronous API by passingasynchronous=Trueto theGCSFileSystemconstructor.
Support
Work on this repository is supported in part by:
"Anaconda, Inc. - Advancing AI through open source."
3 maintainers
Metadata
3 maintainers
Release files for gcsfs 2026.10.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| gcsfs-2026.10.0.tar.gz | 1.1 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| gcsfs-2026.10.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.2 MB
Release files / gcsfs-2026.10.0.tar.gz
| Download URL | gcsfs-2026.10.0.tar.gz |
|---|---|
| Size | 1.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
caa7e34d7618905a70bc7b3d7cf84d745cfa25d9817fd4d7f6211c3ce1350a19
|
|
BLAKE2b-256 checksum How to use checksums |
ad59a1074622f998fd67c35a08509aedf75e18ec11889f5256aa89c631138587
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 1, 2026.
Transparency logRelease files / gcsfs-2026.10.0-py3-none-any.whl
| Download URL | gcsfs-2026.10.0-py3-none-any.whl |
|---|---|
| Size | 88.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5211ef51fab3cb30bc9fa4628c0bc5a26f9908c1bdfb6864778454a682373a7b
|
|
BLAKE2b-256 checksum How to use checksums |
855ba74dc26273a07af686f815ae2069c03e83d4717aae1a4149d1f13f0907d7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 1, 2026.
Transparency log
