Ruten Seller Product Parser
This is a repository that offers a ProductCrawler class to crawl Ruten web pages for the product information in json format.
from ruten_crawler import ProductCrawler
product_crawler = ProductCrawler(seller_id = "hambergurs")
results = product_crawler.get_crawl_result()
Installation
To install this verson from PyPI, type:
pip install rutencrawler
To get the newest one from this repo (note that we are in the alpha stage, so there may be frequent updates), type:
pip install git+git://github.com/jn8029/ruten_crawler.git
Overview
class ProductCrawler class handles the whole web crawling logic. It takes optional arguments of sleep_time and sleep_at_each_iteration
class ProductPageParser handles the product page information extraction. Currently the parser only extracts shipping information, urls for images and the title of the product. More info can be extracted and the logic can be added here.
class ProdcutListParser handles the parsing of product list page. The main function is to extract a list of product urls at each page, and then the urls are then used to parse product information with ProductPageParser
To-do
- add more error-proof exception handlers in ProductCrawler due to the multi-threaded nature of the process.
- add more product info extraction features in ProductCrawler, e.g. price, remaining time, description, etc.
Metadata
Release files for ruten-crawler 0.0.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ruten_crawler-0.0.6.tar.gz | 3.7 kB | Without provenance | Details for ruten_crawler-0.0.6.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ruten_crawler-0.0.6-py3-none-any.whl | Python 3 | none | any | Details for ruten_crawler-0.0.6-py3-none-any.whl |
Total release size: 11.7 kB
Release files / ruten_crawler-0.0.6.tar.gz
| Download URL | ruten_crawler-0.0.6.tar.gz |
|---|---|
| Size | 3.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f314316b279d602405ebda7a934cedbd35a24bd6ae44534a023e6f0581ad7721
|
|
BLAKE2b-256 checksum How to use checksums |
52ed7a432f85f561bb10d7c4a617d2e92270602cbafbc1c8fb2d481ee89fad63
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.5.4
|
Release files / ruten_crawler-0.0.6-py3-none-any.whl
| Download URL | ruten_crawler-0.0.6-py3-none-any.whl |
|---|---|
| Size | 8.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4b538f333e6484b231efa19ab157111e37dd0124f389e676b625f9509d4d0f67
|
|
BLAKE2b-256 checksum How to use checksums |
3adff081e5b889299889807e3d06e9f703ad91cbd889962ec3f6a2f1cc2faa0f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.5.4
|
Security for ruten-crawler 0.0.6
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.