kompas-scraper

Title

Kompas Scraper

Descrption

With kompas Scraper you can scrape search results and extract the contents produced by the search result.
In kompas scraper it will scrape the data present of the website and give json data which contains the details of the contents on the website.
The scraper gives information of the title of the article, article content, links, publish date etc.

JSON sample data

{
    "Scraper_repeat_data_0": [
        {
            "trenLatest_item": [
                {
                    "link": "https://health.kompas.com/read/2021/11/10/120000368/punya-gejala-mirip-ini-beda-bronkitis-dan-pneumonia",
                    "text": "\n\n\n\n\n\n\nHEALTH\n10/11/2021, 12:00 WIB\n\nPunya Gejala Mirip, Ini Beda Bronkitis dan Pneumonia\n\n\n"
                },
                {
                    "link": "https://health.kompas.com/read/2021/11/10/103100268/apa-beda-nyeri-dada-karena-gerd-dan-serangan-jantung-",
                    "text": "\n\n\n\n\n\n\nHEALTH\n10/11/2021, 10:31 WIB\n\nApa Beda Nyeri Dada karena GERD dan Serangan Jantung?\n\n\n"
                },
                {
                    "link": "https://health.kompas.com/read/2021/11/10/090100968/6-ciri-ciri-hamil-anak-kembar",
                    "text": "\n\n\n\n\n\n\nHEALTH\n10/11/2021, 09:01 WIB\n\n6 Ciri-ciri Hamil Anak Kembar\n\n\n"
                }
            ]
        }

Download full sample data

Run Scraper

from kompas_scraper import *
link="https://www.kompas.com/food"
data=run_kompas_scraper(link)

How it works?

It takes URL of kompas page with a search keyword to scrape the data.
It generates the json data which contains the information of the kompas search result.

Examples

Below are some of the examples of URLs using which you can scrape:

Queries/ Feedback

If you have some queries or feedback please contact us at following
Telegram
Email

Metadata

Release files for kompas-scraper 1.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for kompas-scraper 1.0.1
File	Size	Uploaded	Provenance	Actions
kompas_scraper-1.0.1.tar.gz	2.9 kB	Nov 12, 2021	Without provenance	Details for kompas_scraper-1.0.1.tar.gz

Release files / kompas_scraper-1.0.1.tar.gz

Download URL	kompas_scraper-1.0.1.tar.gz
Size	2.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`3b5337371f3a08e38ba7d73bdcc24550ea9b56943b876290a421a9a197ff3c3d`
BLAKE2b-256 checksum How to use checksums	`0e9817d1c1d75c1dc5e4447c24ea5778a357bcfc7997a612ad25f162ef20da91`
Upload date	Nov 12, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.5.0 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.6`

Security for kompas-scraper 1.0.1

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.