Skip to main content

The ultimate library for data scientist to scrape data from https://www.lefaso.net

Project description

lefaso-net-scraper

PyPI version

Description

lefaso-net-scraper is a robust and versatile Python library designed to efficiently extract articles from the popular online news source in Burkina Faso, www.lefaso.net. This powerful scraping tool allows users to effortlessly collect article content and user comments on lefaso.net.

Important

This scraper, like other scrapers, is based on the structure of the target website. Changes to the website's structure can affect the scraper. We use automated workflows to detect these issues frequently, but we cannot catch all of them. Please report any issues you encounter and use the latest version.

JSON/dictionary Fields

Field Description
article_topic Category or subject of the article.
article_title The main headline or title of the article.
article_published_date Date when the article was published.
article_origin Source or platform where the article was published.
article_url Web link to the article.
article_content Full text or body of the article.
article_comments Feedback or responses from readers.

Installation

  • Using pip
pip install --upgrade  lefaso-net-scraper
  • Using poetry
poetry add lefaso-net-scraper
poetry update lefaso-net-scraper  # to update the package

Usage

# coding: utf-8

from lefaso_net_scraper import LefasoNetScraper

section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
data = scraper.run()
  • Settings Pagination range
# coding: utf-8

from lefaso_net_scraper import LefasoNetScraper

section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
scraper.set_pagination_range(start=20, stop=100)
data = scraper.run()
  • Save data to csv
# coding: utf-8

from lefaso_net_scraper import LefasoNetScraper
import pandas as pd

section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
data = scraper.run()
df = pd.DataFrame.from_records(data)
df.to_csv('path/to/df.csv')

We ❤ open source

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lefaso_net_scraper-0.3.7.tar.gz (16.3 kB view details)

Uploaded Source

Built Distribution

lefaso_net_scraper-0.3.7-py3-none-any.whl (17.6 kB view details)

Uploaded Python 3

File details

Details for the file lefaso_net_scraper-0.3.7.tar.gz.

File metadata

  • Download URL: lefaso_net_scraper-0.3.7.tar.gz
  • Upload date:
  • Size: 16.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for lefaso_net_scraper-0.3.7.tar.gz
Algorithm Hash digest
SHA256 9974e3ac277672e73671342e7d0d112684280c893a0e1680807bb693d490ca84
MD5 26bfd03b7d33870d7b61cc35ed4b313e
BLAKE2b-256 d05d0846279e1bc28df45d9c524288f8a6829e0ce0f9214f0e0601f4cd3c7121

See more details on using hashes here.

File details

Details for the file lefaso_net_scraper-0.3.7-py3-none-any.whl.

File metadata

  • Download URL: lefaso_net_scraper-0.3.7-py3-none-any.whl
  • Upload date:
  • Size: 17.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for lefaso_net_scraper-0.3.7-py3-none-any.whl
Algorithm Hash digest
SHA256 1a88ea30bac7241f5522e27ddde66721f3bb71178dd66e2c0f19f99a3e5e397f
MD5 660ad922ca639217020a44efa22516f5
BLAKE2b-256 7d2075d9a1ea9349016008f792cfece459cad8e6ed806f151a3ae1cac61d25a1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page