Command line tool that crawls a local webserver with a headless browser and outputs static html files. Works well with html5validator to validate HTML5 from dynamic content with Javascript.
Run localcrawl --help:
PhantomJS is required. It is pre-installed on TravisCI. On a Mac run brew install PhantomJS.
Example Command
localcrawl --start _build/html/index.html --out _crawled/ --depth 3
Mustache Example
This can be used to convert templated files to HTML files (e.g. for validation with html5validator).
Input:
<html>
<head>
<title>Mustache Test</title>
</head>
<body>
<div id="output"></div>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
<script>
var data = {
item: 'Fork',
price: function() { return (1.10 * 1.08).toFixed(2); },
};
var html = Mustache.render('{{item}}: <b>${{price}}</b>', data);
document.getElementById('output').innerHTML = html;
</script>
</body>
</html>
The crawled output includes the output from processing the template (Fork: <b>$1.19</b>):
<html><head>
<title>Mustache Test</title>
</head>
<body>
<div id="output">Fork: <b>$1.19</b></div>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mustache.js/2.2.1/mustache.min.js"></script>
<script>
var data = {
item: 'Fork',
price: function() { return (1.10 * 1.08).toFixed(2); },
};
var html = Mustache.render('{{item}}: <b>${{price}}</b>', data);
document.getElementById('output').innerHTML = html;
</script>
</body></html>
Should play nice with:
JavaScript template engines / JS frameworks:
Static site generators:
Metadata
Release files for localcrawl 0.2.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| localcrawl-0.2.3.tar.gz | 5.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| localcrawl-0.2.3-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details |
Total release size: 11.2 kB
Release files / localcrawl-0.2.3.tar.gz
| Download URL | localcrawl-0.2.3.tar.gz |
|---|---|
| Size | 5.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9b222f2ee69860a7b4c96ecb7f98aaf71d24cee55d1441b737c59a5811dc17a5
|
|
BLAKE2b-256 checksum How to use checksums |
1a9653ac58886509486ea2181677d33f809e0a2b5644b5a59bb5f80cf3e58631
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / localcrawl-0.2.3-py2.py3-none-any.whl
| Download URL | localcrawl-0.2.3-py2.py3-none-any.whl |
|---|---|
| Size | 6.0 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
2f7c629bc110bf3f3b0c8773ab87f8c11c5adc85bcb2d2a1000a1aad382f4c7f
|
|
BLAKE2b-256 checksum How to use checksums |
5a94dab8a257e468a5f91b5bc3ade9cd92d6c5c33c97989a3643f2213ff3b16e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |