pip install --upgrade pip
pip install build setuptools wheel twine

python -m build

pip install --upgrade pip pip install build twine python -m twine upload dist/*

first we need to download the docker

SearchNow

SearchNow is an open-source Python framework that enables LLMs and AI agents to search the web and extract clean webpage content using SearXNG and Trafilatura.

It can be used as an open-source alternative to commercial search APIs such as Tavily Search and Brave Search in your AI applications.

Features

• 🔍 Search the web using SearXNG
• 📄 Extract clean webpage content using Trafilatura
• ⚡ Async API for high-performance applications
• 🤖 Designed for LLMs, AI Agents, and RAG pipelines
• 🌐 Fetch and parse any webpage with a single function
• 🆓 Fully open source

Installation

pip install searchnow

Or install the latest version from GitHub.

Requirements

SearchNow requires a running SearXNG instance.

Run SearXNG using Docker

docker run -d \
  --name searxng \
  -p 8080:8080 \
  searxng/searxng

After starting the container, your search endpoint will be available at:

http://localhost:8080

Usage

Search the Web

import asyncio
from searchnow import FetchResult

async def main():
    fetch = FetchResult()

    result = await fetch.fetch_data("NVIDIA NeMo Framework")

    print(result)

asyncio.run(main())

Fetch Content from a URL

import asyncio
from searchnow import FetchResult

async def main():
    fetch = FetchResult()

    url_data = await fetch.fetch_url(
        url="https://github.com/bigscience-workshop/petals"
    )

    print(url_data)

asyncio.run(main())

API

fetch_data(query)

Searches the web and returns cleaned search results.

result = await fetch.fetch_data("NVIDIA NeMo Framework")

fetch_url(url)

Downloads a webpage and extracts its readable content.

url_data = await fetch.fetch_url(
    url="https://github.com/bigscience-workshop/petals"
)

Use Cases

• AI Agents
• Retrieval-Augmented Generation (RAG)
• LLM Tool Calling
• Research Assistants
• Web Knowledge Retrieval
• Autonomous Agents

Why SearchNow?

Many LLM applications rely on paid search providers such as:

• Tavily
• Brave Search
• SerpAPI

SearchNow provides an open-source alternative by combining:

• SearXNG for web search
• Trafilatura for webpage extraction

This gives developers complete control over their search infrastructure without vendor lock-in.

Tech Stack

• Python
• SearXNG
• Trafilatura
• AsyncIO
• HTTPX

Contributing

Contributions are welcome!

Feel free to submit issues, feature requests, or pull requests.

License

MIT License