Skip to main content

Scraper Ignite

A simple web scraper project creator that automatically generates a basic project structure including extractors, parsers, pipelines, configuration, and more.

Getting Started

This repository contains a utility package that, once installed, provides a CLI command to create a web scraper project structure automatically. The command is registered as a console script named scraper-ignite.

Prerequisites

Setup and Usage

  1. Clone the Repository

    Clone this repository to your local machine.

  2. Install the Package

    From the root of the repository, install the package locally:

    pip install scraper-ignite
    
  3. Run the Project Setup Command

    Execute the following command to generate the scraper project structure:

    scraper-ignite --project <your_project_name> # defaults to scraper_app
    
  4. Next Steps

    After generating the project, follow these steps:

    • Navigate into the generated project directory:

      cd scraper_app
      
    • Create a virtual environment:

      python -m venv venv
      
    • Activate the virtual environment:

      • Windows:
        venv\Scripts\activate
        
      • Mac/Linux:
        source venv/bin/activate
        
    • Install the project dependencies:

      pip install -r requirements.txt
      
    • Run the scraper:

      python run.py
      

License

This project is licensed under the MIT License.

Author

Nahom D
Email: nahom@nahom.eu.org

Metadata

Release files for scraper-ignite 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for scraper-ignite 0.1.1
File Size Uploaded Provenance Actions
scraper_ignite-0.1.1.tar.gz 5.1 kB Without provenance Details for scraper_ignite-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for scraper-ignite 0.1.1
File Interpreter ABI Platform Actions
scraper_ignite-0.1.1-py3-none-any.whl Python 3 none any Details for scraper_ignite-0.1.1-py3-none-any.whl

Total release size: 10.3 kB

Release files / scraper_ignite-0.1.1.tar.gz

Download URL scraper_ignite-0.1.1.tar.gz
Size 5.1 kB
Tags Source
SHA-256 checksum
How to use checksums
b3a6d90299f22559811b224cfe52628dc755367f8558cd3515ea8423d69ad284
BLAKE2b-256 checksum
How to use checksums
858084ce0e2830cc7743d4b94b85a982f0dddd41d0f015eed493f99f9d081a03
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.13.1

Release files / scraper_ignite-0.1.1-py3-none-any.whl

Download URL scraper_ignite-0.1.1-py3-none-any.whl
Size 5.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e7aca18575055aaa54500b5474452d8970136d19930b48cc8d7ad3bec3938253
BLAKE2b-256 checksum
How to use checksums
c8fb0036dde4ef33a5dd15fac256fae6bdb1968b09365fa31fbeae05940f7aa3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.13.1

Security for scraper-ignite 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page