Skip to main content

A simple package for scraping websites and generating responses using Google's Generative AI.

Project description

ai-scraper

A simple package for scraping websites and generating responses using Google's Generative AI.

Features

  • Scrapes a webpage using Selenium with headless Chrome.
  • Passes scraped HTML into a Generative AI model along with custom instructions.
  • Returns the AI-generated response.

Installation

You can install the package via pip (after publishing to PyPI):

bash
pip install ai-scraper

Usage example

from ai_scraper import scrape

response_text = scrape(
    google_api_key="YOUR_GOOGLE_API_KEY",
    url="https://webscraper.io/test-sites/e-commerce/allinone",
    instructions="Return all products names, in JSON."
)
print(response_text)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ai_gemini_scraper-0.1.2.tar.gz (3.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ai_gemini_scraper-0.1.2-py3-none-any.whl (4.1 kB view details)

Uploaded Python 3

File details

Details for the file ai_gemini_scraper-0.1.2.tar.gz.

File metadata

  • Download URL: ai_gemini_scraper-0.1.2.tar.gz
  • Upload date:
  • Size: 3.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for ai_gemini_scraper-0.1.2.tar.gz
Algorithm Hash digest
SHA256 d86c4dccca0f4d6e8c03f545892a7d6b1ea435a3aaf978660b68db8f167bd488
MD5 ce8dba9bbb4f8030bfba7b6cc33b92d7
BLAKE2b-256 78edd329d06a4afae48bd74def5615b4b43472acf76dbd6b50b09acf335f88a4

See more details on using hashes here.

File details

Details for the file ai_gemini_scraper-0.1.2-py3-none-any.whl.

File metadata

File hashes

Hashes for ai_gemini_scraper-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 a8c5c68f3dcaa564e08234338619d48ce32dae9d2a20d076fe8c6af2c72d3836
MD5 ae8af75c284d917a89b99167a1a07457
BLAKE2b-256 e11231c3f8743b80556d5b5013a943a1edd5fb576c7a5e7e1cd7548c41677245

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page