Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

HTTP downloader that fix wget issues

Project description

ReGet: retransmit less

Inspired by lack of functionality of wget this tool solves two annoying issues

  • wget -N -O somefile doesn’t work as you may expect
  • there is no way to atomically replace target file with updated version

Okay, second problem easily solved like this

#!/bin/bash

TMPFILE=$(mktemp -q)

(wget -qO "$TMPFILE" "$1" && mv "$TMPFILE" "$2") || rm "$TMPFILE"

But what about first issue? It needs much more complicated shell-magic, introduces more troubles and requires additional HTTP request to target server.

Features

  • Built on top of requests.
  • Keep response metadata for next run.
  • Send only needed headers from previous run.
  • Atomic update of destination file.

Installation

$ pip install reget

Usage

As console tool:

$ reget http://example.org /tmp/example

will download server response to /tmp/example and create /tmp/example.json with headers needed for next run.

As python package

>>> import reget
>>> reget.get('http://example.org', '/tmp/example')
<Response [200]>
>>> reget.get('http://example.org', '/tmp/example')
<Response [304]>

Note that subsequent calls with the same url save_path arguments will respect 304 Not Modified.

Known issues

  • Noticable startup time

TODO

  • Option to use single directory for response metadata.
  • Accept multiple <url> <save_path> from file/stdin.

Project details


Release history Release notifications

This version
History Node

0.5

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
reget-0.5-py2.py3-none-any.whl (5.8 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Jul 20, 2015
reget-0.5.tar.gz (3.4 kB) Copy SHA256 hash SHA256 Source None Jul 20, 2015

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page