A simple package for crawling bol.com
Project description
bolcom_crawler
This is a really simple crawler that makes use of Scrapy to crawl bol.com.
Usage
The Crawler
instance has two functions you can use, crawl_products
and crawl_category
. See an example below.
from bol_crawler.crawler import Crawler
crawler = Crawler()
# to crawl products
products = crawler.crawl_products(
[
'https://www.bol.com/nl/p/lg-34gl750-b-ultragear-gaming-monitor/9200000115819731',
]
)
# to crawl a category
products = crawler.crawl_category(
[
'https://www.bol.com/nl/l/gaming-toetsenborden/N/18214/', 0 # the 0 value is how often you want to go to the next page. 0 is just crawling the first page
]
)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bol-crawler-0.0.1.tar.gz
(1.3 kB
view hashes)
Built Distribution
Close
Hashes for bol_crawler-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 147b9a04b8cb06f8a920c7ef0d3fd9e0e1caae9cc8cd2e186c2ad9691adbad27 |
|
MD5 | 18dec37c0ff05a04191f047b96177e31 |
|
BLAKE2b-256 | 8b044d2a8dc29303392a37d87b467b543e3217efbcff243def962dc17fdb5338 |