Skip to main content

Versioning Caching kinda-Proxy for decoupling external responses

Project description

Buffy

Versioning Caching kinda-Proxy for decoupling external HTTP responses

Maintainer: tim.bleimehl@dzd-ev.de

status: alpha (WIP - do not use productive yet)

What is this?

Buffy is a server/client framework to buffer/cache your http requests. It decouples your dependency on external webservers that are not under your control. It manages downloads from slow and unreliable webservers in the background:

  • Resume broken downloads
  • Retry corrupted downloads
  • Versioning of changing downloads

You can ignore any issues with external webservers and just focus on your application.

At the moment there is only a python client library. But the server has a REST API that can be consumed from any coding language. You are welcome to write a client in your language.

Example use

from buffy.buffypyclient import BuffyPyClient

# connect to buffy server
c = BuffyPyClient(ssl=False)

# create a request
req = c.create_request(
    url="https://ftp.ncbi.nlm.nih.gov/pubmed/baseline/pubmed22n0003.xml.gz"
)

# save requested file
req.download_response_content_to("/tmp/pubmed22n0003.xml.gz")

# or alternatively stream requested file
with open("/tmp/pubmed22n0003_2.xml.gz", "wb") as f:
    for chunk in req.download_response_content():
        f.write(chunk)

This is all it takes to request a file. Next time the webserver at ftp.ncbi.nlm.nih.go should be down the buffy client will just serve you the cached answer. Should your Buffy server be down, the client will fall back to direct downloading the request from the source.

Setup Server

todo

Python Client API

todo

Server REST API

todo

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

DZDBuffy-0.0.18.tar.gz (190.1 kB view details)

Uploaded Source

Built Distribution

DZDBuffy-0.0.18-py3-none-any.whl (30.1 kB view details)

Uploaded Python 3

File details

Details for the file DZDBuffy-0.0.18.tar.gz.

File metadata

  • Download URL: DZDBuffy-0.0.18.tar.gz
  • Upload date:
  • Size: 190.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.6

File hashes

Hashes for DZDBuffy-0.0.18.tar.gz
Algorithm Hash digest
SHA256 49e9fcb47ccfb9d83b5a6702e8840b2a1bb985c3ab08302ff4ff0a8a03403b35
MD5 af742f0eb73e39a34faecca3d90ecfff
BLAKE2b-256 f0c9a5a41738f9f4dea08a71f675c068467fb9dae73a5b27dcdd6d5bf78e2dd2

See more details on using hashes here.

File details

Details for the file DZDBuffy-0.0.18-py3-none-any.whl.

File metadata

  • Download URL: DZDBuffy-0.0.18-py3-none-any.whl
  • Upload date:
  • Size: 30.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.6

File hashes

Hashes for DZDBuffy-0.0.18-py3-none-any.whl
Algorithm Hash digest
SHA256 c44fad48857368c38a3b9b9a092ecbc3e70bcdddb02026a73797be337a48ae9d
MD5 b75368bac1b37f0ab4bcc31509b0b3ce
BLAKE2b-256 2199f27083cef918c653d33be778122e5c97ba0331224135d5fe95f7c80c2846

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page