Install requirements and run code in virtual environments from the comfort of your own GIL
Project description
httpfs-sync
Overview
httpfs-sync
is an implementation of a filesystem interface for the Python fsspec
library that uses synchronous requests based on urllib3
. This serves as an alternative
to the default asynchronous (based on aiohttp
) filesystem implementation provided by
fsspec
. This project is ideal for environments where asynchronous operations are not
feasible or desired, allowing users to interact with HTTP-based filesystems using a
synchronous API.
If you're dealing with deadlocks potentially involving the default HTTP 'filesystem' in
fsspec
, the implementation here may help with verification of behavior: just use the classes
provided here as a drop-in replacement for HTTPFileSystem
and compare behaviors.
Features
- Synchronous HTTP requests for file system operations.
- Fully compatible with the
fsspec
API, making it easy to drop in as a replacement. - Uses
urllib3
for robust, production-ready HTTP communications.
Installation
To install httpfs-sync
, you can use pip:
pip install httpfs-sync
Usage
Here's a simple example of how to use httpfs-sync to read a file from an HTTP server:
import fsspec
from httpfs_sync.core import SyncHTTPFileSystem
# enable use of SyncHTTPFileSystem and associated files for both http and https
SyncHTTPFileSystem.overwrite_async_registration()
# Use the registered file system to access a remote file
with fsspec.open('http://example.com/data.txt', 'r') as f:
data = f.read()
print(data)
License
httpfs-sync is distributed under the MIT License, allowing the code to be used in commercial applications as well as open-source projects.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for httpfs_sync-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 223c1bf44ace63f6f5b3a07678174133a701314c8bdf88a53f7fb7d71de8e521 |
|
MD5 | 86e5f53fc456d2afbecf77f5105830a0 |
|
BLAKE2b-256 | 6b42c1e2acce42bb8597b215313de8dacbe07d90928c195dcaf7218fa461c543 |