A simple web crawler.
Project description
A Tiny Web Crawler
A web crawler written in python.
Install Requirements
pip install validators beautifulsoup4 lxml
Python version: Python 3.6.3 :: Anaconda, Inc.
Run
python crawler.py
Starts crawling from root_url until max_link urls are crawled. Saves the output to out.json.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tiny_web_crawler-0.1.tar.gz
(4.1 kB
view hashes)
Built Distribution
Close
Hashes for tiny_web_crawler-0.1-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 071199a9f7f5277329e5c3d82ba3a14475651f31c6e2352d7841efdaa00b82f1 |
|
| MD5 | bf291092f5516403a49000d836fd67fa |
|
| BLAKE2b-256 | 63b5edf5a5e9ba29901f7f29353ae5caeccf7a9925b3e2015c8dcce58d8694b4 |
