Skip to main content

Ruten Seller Product Parser

PyPI version PyPI license

This is a repository that offers a ProductCrawler class to crawl Ruten web pages for the product information in json format.

from ruten_crawler import ProductCrawler
product_crawler = ProductCrawler(seller_id = "hambergurs")
results = product_crawler.get_crawl_result()

Installation

To install this verson from PyPI, type:


pip install rutencrawler

To get the newest one from this repo (note that we are in the alpha stage, so there may be frequent updates), type:


pip install git+git://github.com/jn8029/ruten_crawler.git

Overview

class ProductCrawler class handles the whole web crawling logic. It takes optional arguments of sleep_time and sleep_at_each_iteration

class ProductPageParser handles the product page information extraction. Currently the parser only extracts shipping information, urls for images and the title of the product. More info can be extracted and the logic can be added here.

class ProdcutListParser handles the parsing of product list page. The main function is to extract a list of product urls at each page, and then the urls are then used to parse product information with ProductPageParser

To-do

  • add more error-proof exception handlers in ProductCrawler due to the multi-threaded nature of the process.
  • add more product info extraction features in ProductCrawler, e.g. price, remaining time, description, etc.

Metadata

Release files for ruten-crawler 0.0.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ruten-crawler 0.0.6
File Size Uploaded Provenance Actions
ruten_crawler-0.0.6.tar.gz 3.7 kB Without provenance Details for ruten_crawler-0.0.6.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ruten-crawler 0.0.6
File Interpreter ABI Platform Actions
ruten_crawler-0.0.6-py3-none-any.whl Python 3 none any Details for ruten_crawler-0.0.6-py3-none-any.whl

Total release size: 11.7 kB

Release files / ruten_crawler-0.0.6.tar.gz

Download URL ruten_crawler-0.0.6.tar.gz
Size 3.7 kB
Tags Source
SHA-256 checksum
How to use checksums
f314316b279d602405ebda7a934cedbd35a24bd6ae44534a023e6f0581ad7721
BLAKE2b-256 checksum
How to use checksums
52ed7a432f85f561bb10d7c4a617d2e92270602cbafbc1c8fb2d481ee89fad63
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.5.4

Release files / ruten_crawler-0.0.6-py3-none-any.whl

Download URL ruten_crawler-0.0.6-py3-none-any.whl
Size 8.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
4b538f333e6484b231efa19ab157111e37dd0124f389e676b625f9509d4d0f67
BLAKE2b-256 checksum
How to use checksums
3adff081e5b889299889807e3d06e9f703ad91cbd889962ec3f6a2f1cc2faa0f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.5.4

Security for ruten-crawler 0.0.6

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.0.6 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page