Skip to main content

Crawl rendered JavaScript templates from a local server.

Project description

https://travis-ci.org/svenkreiss/localcrawl.svg?branch=master

You will need PhantomJS. On a Mac:

brew install PhantomJS

Example Run Command

localcrawl --start _build/html/index.html --out _crawled/ --depth 3

Mustache Example

This can be used to convert templated files to HTML files (e.g. for validation with html5validator).

Input:

<html>
<head>
  <title>Mustache Test</title>
</head>
<body>
  <div id="output"></div>

  <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
  <script>
    var data = {
      item: 'Fork',
      price: function() { return (1.10 * 1.08).toFixed(2); },
    };
    var html = Mustache.render('{{item}}: <b>${{price}}</b>', data);
    document.getElementById('output').innerHTML = html;
  </script>
</body>
</html>

The crawled output includes the output from processing the template (Fork: <b>$1.19</b>):

<html><head>
  <title>Mustache Test</title>
</head>
<body>
  <div id="output">Fork: <b>$1.19</b></div>

  <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
  <script>
    var data = {
      item: 'Fork',
      price: function() { return (1.10 * 1.08).toFixed(2); },
    };
    var html = Mustache.render('{{item}}: <b>${{price}}</b>', data);
    document.getElementById('output').innerHTML = html;
  </script>


</body></html>

Should play nice with:

JavaScript template engines / JS frameworks:

Static site generators:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

localcrawl-0.1.0.tar.gz (4.1 kB view hashes)

Uploaded Source

Built Distribution

localcrawl-0.1.0-py2.py3-none-any.whl (6.7 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page