HTTP downloader that fix wget issues
ReGet: retransmit less
Inspired by lack of functionality of wget this tool solves two annoying issues
- wget -N -O somefile doesn’t work as you may expect
- there is no way to atomically replace target file with updated version
Okay, second problem easily solved like this
#!/bin/bash TMPFILE=$(mktemp -q) (wget -qO "$TMPFILE" "$1" && mv "$TMPFILE" "$2") || rm "$TMPFILE"
But what about first issue? It needs much more complicated shell-magic, introduces more troubles and requires additional HTTP request to target server.
- Built on top of requests.
- Keep response metadata for next run.
- Send only needed headers from previous run.
- Atomic update of destination file.
$ pip install reget
As console tool:
$ reget http://example.org /tmp/example
will download server response to /tmp/example and create /tmp/example.json with headers needed for next run.
As python package
>>> import reget >>> reget.get('http://example.org', '/tmp/example') <Response > >>> reget.get('http://example.org', '/tmp/example') <Response >
Note that subsequent calls with the same url save_path arguments will respect 304 Not Modified.
- Noticable startup time
- Option to use single directory for response metadata.
- Accept multiple <url> <save_path> from file/stdin.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|File Name & Hash SHA256 Hash Help||Version||File Type||Upload Date|
(5.8 kB) Copy SHA256 Hash SHA256
|py2.py3||Wheel||Jul 20, 2015|
(3.4 kB) Copy SHA256 Hash SHA256
|–||Source||Jul 20, 2015|