agent-search 0.0.8

AgentSearch: An open source framework and dataset for webscale local search.

AgentSearch [ΨΦ]: A powerful search agent client and search engine.

This framework facilitates seamless integration with the AgentSearch dataset or hosted search APIs (e.g. Search Engines) and with RAG-specialized LLM's (e.g. Search Agents).

Features of AgentSearch

• Search Agent: Seamless integration with any RAG-specialized LLM, such as Sensei-7B.
• Customizable Search: Deploy your own local search engine with the AgentSearch dataset. Or, introduce your own custom datasets to meet your specific needs.
• API Endpoint: Connect with SciPhi and other search provider APIs for easy deployment.

Quickstart Guide for AgentSearch

Install the AgentSearch Client

pip install agent-search

# Get a free API key at https://www.sciphi.ai/signup
export SCIPHI_API_KEY=$MY_SCIPHI_API_KEY

# Requires SCIPHI_API_KEY in env
from agent_search import SciPhi

client = SciPhi()

# Perform a search
search_response = client.search(query='Quantum Field Theory', search_provider='agent-search')
print(search_response)
# [{ 'score': '.89', 'url': 'https://...', 'metadata': {...} }

# Generate a RAG response
rag_response = client.get_search_rag_response(query='latest news', search_provider='bing', llm_model='SciPhi/Sensei-7B-V1')
print(rag_response)
# { 'response': '...', 'other_queries': '...', 'search_results': '...' }

---

Community & Support

• Engage with the community on Discord.
• For tailored inquiries or feedback, please email us.

Local AgentSearch Setup and Initialization

For self-hosting, follow the steps below.

Prerequisites

Ensure Docker and Postgres are installed on your system.

• Download Docker here.
• Download Postgres here.

Steps:

1. Launch Postgres Database:

   • Start the Postgres service on your system:

     # Command to start Postgres, adjust based on your system's configuration
     sudo service postgresql start
     

   • This step ensures that the Postgres database is running and ready to be populated.

2. Relational Database Population:

   • Command to populate the Postgres database:

     python -m agent_search.scripts.populate_postgres_from_hf run
     

   • This script populates a postgres database with the parameters from config.ini, adaptable to custom datasets. For help directly istalling the full 4TB postgres database, contact our team.

3. Start Qdrant Service with Docker:

   • Run the Qdrant service in Docker:

     docker run -p 6333:6333 -p 6334:6334 \
         -v $(pwd)/qdrant_storage:/qdrant/storage:z \
         qdrant/qdrant
     

   • For Qdrant installation guidance, see Qdrant Documentation.

4. Vector Database Population:

   • Populate the Vector database:

     python -m agent_search.scripts.populate_qdrant_from_postgres run --delete_existing=True
     

   • This step prepares a qdrant database with the parameters from config.ini. For direct installation assistance, contact our team.

5. Run the Server:

   • Launch the AgentSearch server:

     python -m agent_search.app.server
     

Additional Notes

• Execute all commands from the root directory of the AgentSearch project.
• Replace query in the run command with your search query.
• Check back soon for our User Guide.