Install requirements and run code in virtual environments from the comfort of your own GIL
Project description
httpfs-sync
Overview
httpfs-sync
is an implementation of a filesystem interface for the Python fsspec
library that uses synchronous requests based on urllib3
. This serves as an alternative
to the default asynchronous (based on aiohttp
) filesystem implementation provided by
fsspec
. This project is ideal for environments where asynchronous operations are not
feasible or desired, allowing users to interact with HTTP-based filesystems using a
synchronous API.
If you're dealing with deadlocks potentially involving the default HTTP 'filesystem' in
fsspec
, the implementation here may help with verification of behavior: just use the classes
provided here as a drop-in replacement for HTTPFileSystem
and compare behaviors.
Features
- Synchronous HTTP requests for file system operations.
- Fully compatible with the
fsspec
API, making it easy to drop in as a replacement. - Uses
urllib3
for robust, production-ready HTTP communications.
Installation
To install httpfs-sync
, you can use pip:
pip install httpfs-sync
Usage
Here's a simple example of how to use httpfs-sync to read a file from an HTTP server:
import fsspec
from httpfs_sync.core import SyncHTTPFileSystem
# enable use of SyncHTTPFileSystem and associated files for both http and https
SyncHTTPFileSystem.overwrite_async_registration()
# Use the registered file system to access a remote file
with fsspec.open('http://example.com/data.txt', 'r') as f:
data = f.read()
print(data)
License
httpfs-sync is distributed under the MIT License, allowing the code to be used in commercial applications as well as open-source projects.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for httpfs_sync-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6ed226c9a95de5112b2055a6c0edb750a00b67a936d42a91c9dcda306d35da0 |
|
MD5 | 1758c134e25cf1af9488a8c88799a9b2 |
|
BLAKE2b-256 | 6563a1f634dd3c14cc6e6791284bbc8d750b0205df96891098f6b49e46cba914 |