Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Grab from a remote site page all resources that a browser will probably download visiting the page

Project Description

Quick info

Let say you need to use the HTTP load testing and benchmarking utility siege on a web page and you also want to use the --internet option, to simulate at best the behavior of a web browser.

When a web browser load a page, it also load all the resources inside that page:

  • Images
  • JavaScript files
  • CSS
  • Media resources

So you need a list of all URLs taken from that page.

This utility (its name mean “You Will Download”) will simply create this list for you.

You simply need to redirect the utility output to a file, then use also the siege --file option.

Usage

$ youlldownload http://host.com/section/page

Using with siege:

$ youlldownload http://host.com/section/page > list.txt
$ siege -i -f list.txt [other options]

Taken resouces

  • from script tags we’ll take the src URL
  • from link tags with rel equals to stylesheet we’ll take the href url
  • from img tags we’ll take the src URL
  • from object tags we’ll take the data URL
  • from embed tags we’ll take the src URL
  • from style tags we’ll take the URL inside if the tag is using an “@import url” directive
  • from iframe tags we’ll take the src URL
  • from source tags inside video we’ll take the src URL

Also: CSS sources are deeply analyzed for found additional resources inside them (like background images, fonts, …).

Authors

This product was developed by RedTurtle Technology team.

Changelog

0.4 (2015-11-06)

  • Fix setup.py to properly create a package [ale-rt]

0.3 (2015-05-28)

  • Remove duplicated URLs from final report [keul]
  • Do not include same version of an URL with anchors [keul]
  • Inspect also resources from CSS (backgroun images, fonts, …) [keul]
  • Script was not properly working outside homepage if a “base” tag was not provided [keul]

0.2 (2014-04-02)

  • Added support for src attribute of iframe tag [keul]
  • Added support for src attribute of source tag (HTML 5 video element) [keul]
  • Do not break if base tag is not present [keul]

0.1 (2013-01-30)

  • initial release
Release History

Release History

This version
History Node

0.4

History Node

0.3

History Node

0.2

History Node

0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
YoullDownload-0.4.tar.gz (9.0 kB) Copy SHA256 Checksum SHA256 Source Nov 9, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting