lefaso-net-scraper 0.4.0

The ultimate library for data scientist to scrape data from https://www.lefaso.net

lefaso-net-scraper

Description

lefaso-net-scraper is a robust and versatile Python library designed to efficiently extract articles from the popular online news source in Burkina Faso, www.lefaso.net. This powerful scraping tool allows users to effortlessly collect article content and user comments on lefaso.net.

Important

This scraper, like other scrapers, is based on the structure of the target website. Changes to the website's structure can affect the scraper. We use automated workflows to detect these issues frequently, but we cannot catch all of them. Please report any issues you encounter and use the latest version.

JSON/dictionary Fields

Field	Description
article_topic	Category or subject of the article.
article_title	The main headline or title of the article.
article_published_date	Date when the article was published.
article_origin	Source or platform where the article was published.
article_url	Web link to the article.
article_content	Full text or body of the article.
article_comments	Feedback or responses from readers.

Installation

• Using pip

pip install --upgrade  lefaso-net-scraper

# For jupiter support
pip install --upgrade  lefaso-net-scraper[notebook]

• Using poetry

poetry add lefaso-net-scraper

# For jupiter support
poetry add lefaso-net-scraper[notebook]

Usage

# coding: utf-8

from lefaso_net_scraper import LefasoNetScraper

section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
data = scraper.run()

• Settings Pagination range

# coding: utf-8

from lefaso_net_scraper import LefasoNetScraper

section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
scraper.set_pagination_range(start=20, stop=100)
data = scraper.run()

• Save data to csv

# coding: utf-8

from lefaso_net_scraper import LefasoNetScraper
import pandas as pd

section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
data = scraper.run()
df = pd.DataFrame.from_records(data)
df.to_csv('path/to/df.csv')

We ❤ open source