gcsfs
GCSFS is a Python library that provides a familiar, file-system-like interface to Google Cloud Storage (GCS). Built on top of fsspec, it allows you to interact with cloud buckets as if they were local directories, making it a favorite for data scientists and engineers.
Getting Started
Installation
Install via pip or conda:
# Using pip
pip install gcsfs
# OR using conda
conda install -c conda-forge gcsfs
Basic Usage
import gcsfs
# Initialize the filesystem
fs = gcsfs.GCSFileSystem(project='my-google-project')
# List files in a bucket
files = fs.ls('my-bucket')
# Read a file directly into a string/bytes
with fs.open('my-bucket/data.txt', 'rb') as f:
content = f.read()
Specialized Bucket Support
GCSFS now automatically supports advanced Google Cloud Storage features through its ExtendedFileSystem implementation.
1. Hierarchical Namespace (HNS)
Hierarchical Namespace (HNS) replaces the traditional "flat" GCS structure with true logical directories.
- Atomic Renames: Moving or renaming a directory is an
O(1)metadata operation. No more slow "copy-then-delete" for large folders. - High Performance: Offers up to 8x higher initial Queries Per Second (QPS) for read/write operations.
- AI/ML Ready: Ideal for heavy checkpointing and managing millions of small files.
2. Rapid Buckets (Zonal Storage)
Rapid Buckets are zonal storage resources designed for ultra-low latency and maximum throughput.
- Zonal Co-location: Place your data in the same zone as your GPU/TPU clusters to minimize network lag.
- True Appends: Unlike standard GCS objects, you can append data to existing objects in Rapid buckets without a full rewrite.
- Streaming I/O: Optimized for high-speed model loading and real-time logging.
Integration & Auth
GCSFS plays nicely with the rest of the Python data ecosystem.
Authentication Modes
- Default: Uses your local gcloud credentials or environment service accounts.
- Cloud: Explicitly use Google Metadata service (
token='cloud'). - Anonymous: Access public data without a login (
token='anon'). - Service Account: Pass the path to your JSON key file (
token='path/to/key.json').
[!TIP] Note on Async: GCSFS is built on
aiohttp. If you are building high-concurrency applications, you can use the asynchronous API by passingasynchronous=Trueto theGCSFileSystemconstructor.
Support
Work on this repository is supported in part by:
"Anaconda, Inc. - Advancing AI through open source."
3 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file gcsfs-2026.8.1.tar.gz.
File metadata
- Download URL: gcsfs-2026.8.1.tar.gz
- Upload date:
- Size: 1.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
738da7dcc74817c7668d032f932598482a23cce1b061c537e54278be44d2fff8
|
|
| MD5 |
e46332b5ad6430280baed9dfb42f39b9
|
|
| BLAKE2b-256 |
d008cb32ace6483b8016e8954f009f133799d299dc1bb35b310ced04bd46a925
|
Provenance
The following attestation bundles were made for gcsfs-2026.8.1.tar.gz:
Publisher:
release.yml on fsspec/gcsfs
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
gcsfs-2026.8.1.tar.gz -
Subject digest:
738da7dcc74817c7668d032f932598482a23cce1b061c537e54278be44d2fff8 - Sigstore transparency entry: 2791584209
- Sigstore integration time:
-
Permalink:
fsspec/gcsfs@73929a1b18ae3ef7b56b586a5403240cd827e0cf -
Branch / Tag:
refs/tags/2026.8.1 - Owner: https://github.com/fsspec
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@73929a1b18ae3ef7b56b586a5403240cd827e0cf -
Trigger Event:
push
-
Statement type:
File details
Details for the file gcsfs-2026.8.1-py3-none-any.whl.
File metadata
- Download URL: gcsfs-2026.8.1-py3-none-any.whl
- Upload date:
- Size: 91.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
66e6316690b17e1887a4f4bc0902eee53f5b3fd860cc3f0377d65c5be3313b1f
|
|
| MD5 |
4dca2f674d9fc05091435ca74d14fa7a
|
|
| BLAKE2b-256 |
04e11db9ecfb9f22a71624413ee158b933c18538aacfde77acbb7413cea21a6e
|
Provenance
The following attestation bundles were made for gcsfs-2026.8.1-py3-none-any.whl:
Publisher:
release.yml on fsspec/gcsfs
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
gcsfs-2026.8.1-py3-none-any.whl -
Subject digest:
66e6316690b17e1887a4f4bc0902eee53f5b3fd860cc3f0377d65c5be3313b1f - Sigstore transparency entry: 2791584282
- Sigstore integration time:
-
Permalink:
fsspec/gcsfs@73929a1b18ae3ef7b56b586a5403240cd827e0cf -
Branch / Tag:
refs/tags/2026.8.1 - Owner: https://github.com/fsspec
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@73929a1b18ae3ef7b56b586a5403240cd827e0cf -
Trigger Event:
push
-
Statement type:
