The ultimate library for data scientist to scrape data from https://www.lefaso.net
Project description
lefaso-net-scraper
Description
lefaso-net-scraper is a robust and versatile Python library designed to efficiently extract articles from the popular online news source in Burkina Faso, www.lefaso.net. This powerful scraping tool allows users to effortlessly collect article content and user comments on lefaso.net.
Important
This scraper, like other scrapers, is based on the structure of the target website. Changes to the website's structure can affect the scraper. We use automated workflows to detect these issues frequently, but we cannot catch all of them. Please report any issues you encounter and use the latest version.
JSON/dictionary Fields
Field | Description |
---|---|
article_topic |
Category or subject of the article. |
article_title |
The main headline or title of the article. |
article_published_date |
Date when the article was published. |
article_origin |
Source or platform where the article was published. |
article_url |
Web link to the article. |
article_content |
Full text or body of the article. |
article_comments |
Feedback or responses from readers. |
Installation
- Using pip
pip install --upgrade lefaso-net-scraper
- Using poetry
poetry add lefaso-net-scraper
poetry update lefaso-net-scraper # to update the package
Usage
# coding: utf-8
from lefaso_net_scraper import LefasoNetScraper
section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
data = scraper.run()
- Settings Pagination range
# coding: utf-8
from lefaso_net_scraper import LefasoNetScraper
section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
scraper.set_pagination_range(start=20, stop=100)
data = scraper.run()
- Save data to csv
# coding: utf-8
from lefaso_net_scraper import LefasoNetScraper
import pandas as pd
section_url = 'https://lefaso.net/spip.php?rubrique473'
scraper = LefasoNetScraper(section_url)
data = scraper.run()
df = pd.DataFrame.from_records(data)
df.to_csv('path/to/df.csv')
We ❤ open source
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file lefaso_net_scraper-0.3.7.tar.gz
.
File metadata
- Download URL: lefaso_net_scraper-0.3.7.tar.gz
- Upload date:
- Size: 16.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9974e3ac277672e73671342e7d0d112684280c893a0e1680807bb693d490ca84 |
|
MD5 | 26bfd03b7d33870d7b61cc35ed4b313e |
|
BLAKE2b-256 | d05d0846279e1bc28df45d9c524288f8a6829e0ce0f9214f0e0601f4cd3c7121 |
File details
Details for the file lefaso_net_scraper-0.3.7-py3-none-any.whl
.
File metadata
- Download URL: lefaso_net_scraper-0.3.7-py3-none-any.whl
- Upload date:
- Size: 17.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a88ea30bac7241f5522e27ddde66721f3bb71178dd66e2c0f19f99a3e5e397f |
|
MD5 | 660ad922ca639217020a44efa22516f5 |
|
BLAKE2b-256 | 7d2075d9a1ea9349016008f792cfece459cad8e6ed806f151a3ae1cac61d25a1 |