Skip to main content

A handy application of web scrapping along with Named entity recognition.

Project description

Jeeva - A sample module for webscrapping

## Description

The package is designed to perform Google searches, extract information from web pages, and conduct Named Entity Recognition (NER) using the SpaCy library. The core functionality revolves around a NERProcessor class that utilizes SpaCy for extracting entities such as names, emails, phone numbers, product names, and organizations from text. Additionally, the package includes functions for performing Google searches and extracting information from URLs.

## Features

  1. NERProcessor: - Performs Named Entity Recognition (NER) on a given text. - Extracts entities such as names, emails, phone numbers, product names, and organizations. - Allows filtering entities based on specific types (e.g., person, email, phone, product, organization).

  2. Google Search: - Conducts a Google search based on a user-provided query. - Retrieves a specified number of search results.

  3. Extract Information from URL: - Extracts text content from a given URL. - Utilizes the NERProcessor for Named Entity Recognition if provided. - Handles cases where the NERProcessor is not provided, returning raw text.

  4. Main Script: - Integrates the functionalities of the package into a user-friendly command-line interface. - Allows users to input a query, specify the type of entities to extract, and displays the results in a structured format.

  5. Contact Author Information: - visit jeevanantham-portfolio.web.app

  6. Flexible NER Type Filtering: - Allows users to filter entities based on specific types or extract all available entities if no type is specified.

This package is useful for scenarios where users want to automate Google searches, extract information from web pages, and perform basic Named Entity Recognition on text data. The flexible NER filtering provides customization options based on the user’s specific needs.

## Installation

bash pip install jeeva

## Usage

### NERProcessor

from jeeva import NERProcessor

# Create an instance of NERProcessor ner_processor = NERProcessor()

# Perform Named Entity Recognition on a text entities = ner_processor.perform_ner(“Your sample text here”)

# Access the extracted entities print(“Names:”, entities[“names”]) print(“Emails:”, entities[“emails”]) print(“Phone Numbers:”, entities[“phone_numbers”]) print(“Product Names:”, entities[“product_names”]) print(“Organizations:”, entities[“organizations”])

### Google Search

from jeeva import google_search

# Perform a Google search links = google_search(“Your query here”, num_results=10)

### Extract Information from URL from jeeva import extract_info_from_url # Example usage when NERProcessor is provided entities = extract_info_from_url(”https://example.com”, ner_processor, ner_type=”person”)

# Example usage when NERProcessor is not provided text_content = extract_info_from_url(”https://example.com”, ner_processor=None) print(text_content)

### Main Script

# Example usage of the main script from jeeva import main

user_query = input(“Enter your query: “) user_ner_type = input(“Enter the NER type (person, email, phone, product, organization) or leave blank for all: “) main(user_query, ner_type=user_ner_type)

## Contact

For any issues or inquiries, please contact:

## License

This project is free and open source as it is limited in features. To perform web scraping with NLP tasks with highest accuracy of 99.82% contact me. Th original module contains about 300 varieties of trained NLP categoriztion, advanced web scrapping techniques to scrap information from any site. contact - www.jeevanantham-portfolio.web.app

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jeeva-0.1.0.tar.gz (4.0 kB view details)

Uploaded Source

File details

Details for the file jeeva-0.1.0.tar.gz.

File metadata

  • Download URL: jeeva-0.1.0.tar.gz
  • Upload date:
  • Size: 4.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.7.8

File hashes

Hashes for jeeva-0.1.0.tar.gz
Algorithm Hash digest
SHA256 e88b1c3f04b8c4785c934afff73b68339efe32247f568b6503df21637573aeab
MD5 ecf96f53bb8a4d2f50820fbd434a650e
BLAKE2b-256 3043c6b231084abd37b4f891f04fe1300146eaa47a666bd66144424bb3f039c8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page