This package contains implementation of a web scraper with Gemini Pro integrated into it.
Project description
gemini-pro-web-scraper
Ever wondered about scraping a website without running a single line of code? Well, Gemini Pro Web Scraper is the tool to do so. This tool automatically scrapes the data you want from a website of your choice.
Source Code
The source code of the application Gemini Pro Web Scraper is available in Source Code.
Installation
pip install gemini-pro-web-scraper
How to Use the Application?
Pre-requisites:
- Python installed in your device.
- .env file in the same directory as <GEMINI_PRO_WEB_SCRAPER_DIRECTORY> and has the value of GEMINI_API_KEY.
First, open a Terminal or Command Prompt window and run the following command.
cd <GEMINI_PRO_WEB_SCRAPER_DIRECTORY>
python3 main.py
Note: Replace <GEMINI_PRO_WEB_SCRAPER_DIRECTORY> with the path to the directory of the application Gemini Pro Web Scraper.
Then, the application will start with something looking like in the screenshot below.
You will then be asked to input the following values.
- Temperature - between 0 and 1 inclusive
- Top P - between 0 and 1 inclusive
- Top K - at least 1
- Max output tokens - at least 1
The following screenshot shows what is displayed after inputting the mentioned values.
You will be required to input the following pieces of information.
- The URL of the website you want to scrape (e.g., https://sandbox.oxylabs.io/products).
- What the URL entered in step 1 contains (e.g., games for https://sandbox.oxylabs.io/products).
- The number of elements you want to scrape.
- The details of each element you want to scrape (i.e., the name and the corresponding CSS selector for each element).
- The name of the file you want the code to be in (without the extension).
Once you enter the values mentioned above, the file containing the code will be created inside "scrapers" directory. Moreover, the CSV file containing the scraped data will be generated inside "csvs" directory. Then, you will be asked whether you still want to continue unit testing or not. If you enter 'Y', you will be redirected to an application window like in screenshot above. Else, you will exit the application.
The Python file generated which contains the web scraping code looks like below.
Below is how the generated CSV file looks like.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gemini_pro_web_scraper-1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b7f6a960549ac94cf714aac50ca958645780cc8fcecc5fc49a9636c37c01793e |
|
MD5 | 9a3e11149ec11967e0565effe6b0fcb0 |
|
BLAKE2b-256 | c9f451aa11997339414301f93cca868818ea8ac169a6956361b891a653a68657 |