Skip to main content

Web Search Proxy implementation

Project description

Unique Search Proxy

A unified web search proxy API that provides a consistent interface for multiple search backends. Built with FastAPI and designed for seamless integration with AI applications.

Overview

This service acts as an abstraction layer over different search providers, allowing clients to switch between search engines without changing their integration code. Currently supports:

Engine Description
Google Custom Search Direct integration with Google's Custom Search JSON API
Vertex AI (Gemini) AI-powered search using Google's Gemini models with grounding capabilities

Quick Start

Prerequisites

  • Python 3.12+
  • uv for dependency management
  • Google Cloud credentials (for Vertex AI)
  • Google Custom Search API key and Engine ID (for Google Search)

Installation

# Install dependencies
uv sync

# Copy and configure environment variables
cp .env.example .env

Environment Variables

# Google Custom Search
GOOGLE_SEARCH_API_KEY=your-api-key
GOOGLE_SEARCH_API_ENDPOINT=https://www.googleapis.com/customsearch/v1
GOOGLE_SEARCH_ENGINE_ID=your-engine-id

# Vertex AI
VERTEXAI_SERVICE_ACCOUNT_CREDENTIALS=path/to/credentials.json

Running the Service

Development:

uv run python -m unique_search_proxy.web.app

Docker (from published package — hash-verified):

CI generates a hash-pinned requirements.txt from uv.lock and passes it into the Docker build. Dependencies are installed with --require-hashes, then the package itself is installed with --no-deps. To reproduce locally:

uv export --locked --package unique-search-proxy --no-dev --no-emit-project \
  -o deploy/requirements.txt
docker build --build-arg PACKAGE_VERSION=0.2.0 -t search-proxy deploy/

Every transitive dependency is verified against its sha256 hash from the lockfile.

Docker (from local source — no registry required):

Build a wheel first, copy it into deploy/, then reference it:

uv build --wheel --out-dir deploy/
docker build \
  --build-arg LOCAL_WHEEL=unique_search_proxy-0.2.0-py3-none-any.whl \
  -t search-proxy deploy/

Running the container:

docker run --rm -p 8080:8080 search-proxy

# With custom environment variables
docker run --rm -p 8080:8080 -e WORKERS=8 -e LOG_LEVEL=debug search-proxy

API Documentation

FastAPI provides automatic interactive API documentation:

URL Description
/docs Swagger UI - interactive API explorer
/redoc ReDoc - alternative documentation
/openapi.json OpenAPI schema

API Reference

Health Check

GET /health

Response:

{
  "status": "healthy"
}

Search

POST /search
Content-Type: application/json

Request Body:

Field Type Required Description
search_engine string No "google" or "vertexai" (default: "google")
query string Yes The search query
kwargs object No Engine-specific parameters

Response:

{
  "results": [
    {
      "url": "https://example.com/article",
      "title": "Article Title",
      "snippet": "A brief description of the content...",
      "content": ""
    }
  ]
}

Search Engine Configuration

Google Custom Search

Uses Google's Custom Search JSON API for traditional web search results.

Parameters (kwargs):

Parameter Type Default Description
cx string env default Custom Search Engine ID (overrides env)
fetchSize int 10 Number of results to fetch
timeout int 10 Request timeout in seconds

Example:

{
  "search_engine": "google",
  "query": "latest AI developments",
  "kwargs": {
    "fetchSize": 20,
    "timeout": 15
  }
}

Vertex AI (Gemini)

Leverages Google's Gemini models with web grounding for AI-enhanced search results. This engine:

  1. Uses Gemini to search and synthesize information from the web
  2. Generates structured results with citations
  3. Optionally resolves shortened/redirect URLs to final destinations

Parameters (kwargs):

Parameter Type Default Description
modelName string "gemini-2.5-flash" Gemini model to use
entrepriseSearch bool false Use Enterprise Web Search
systemInstruction string (built-in) Custom system prompt
resolveUrls bool true Resolve redirect URLs

Example:

{
  "search_engine": "vertexai",
  "query": "Compare the top 3 cloud providers for ML workloads",
  "kwargs": {
    "modelName": "gemini-2.5-flash",
    "resolveUrls": true
  }
}

Project Structure

connectors/unique_search_proxy/
├── unique_search_proxy/          # Python package (published to PyPI)
│   ├── __init__.py
│   └── web/                      # Web search API sub-module
│       ├── __init__.py
│       ├── app.py                # FastAPI application
│       ├── settings.py           # Global settings
│       └── core/                 # Search engine implementations
│           ├── schema.py         # Shared schemas
│           ├── google_search/    # Google Custom Search backend
│           └── vertexai/         # Vertex AI (Gemini) backend
├── tests/                        # Test suite
├── deploy/                       # Container build artifacts
│   ├── Dockerfile                # Hash-verified install or local wheel
│   └── entrypoint.sh
└── pyproject.toml

The package uses a sub-module hierarchy (web/) to support future extensions (e.g. internal/ search) that can be deployed as separate containers from the same package.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                       FastAPI App                           │
│                      /search endpoint                       │
└─────────────────────────┬───────────────────────────────────┘
                          │
                    ┌─────▼─────┐
                    │  Factory  │
                    └─────┬─────┘
                          │
          ┌───────────────┼───────────────┐
          │                               │
    ┌─────▼─────┐                   ┌─────▼─────┐
    │  Google   │                   │ Vertex AI │
    │  Search   │                   │  (Gemini) │
    └───────────┘                   └───────────┘

The service uses a factory pattern to register and resolve search engines, making it easy to add new backends.

Error Handling

All errors return a consistent format:

{
  "status": "failed",
  "error": "Error description"
}
Status Code Description
400 Validation error (invalid request)
500 Internal server error

Production Deployment

The service includes a production-ready deploy/entrypoint.sh that uses Uvicorn:

Variable Default Description
HOST 0.0.0.0 Bind address
PORT 8080 Listen port
WORKERS 4 Uvicorn workers
TIMEOUT 120 Keep-alive timeout
LOG_LEVEL info Logging verbosity

Development

# Run with hot reload
uv run uvicorn unique_search_proxy.web.app:app --reload --port 2349

# Format code
uv run ruff format .

# Lint
uv run ruff check .

# Run tests
uv run pytest

# Type check
uv run basedpyright

License

Proprietary - Unique AG

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

unique_search_proxy-0.2.1.tar.gz (10.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

unique_search_proxy-0.2.1-py3-none-any.whl (18.7 kB view details)

Uploaded Python 3

File details

Details for the file unique_search_proxy-0.2.1.tar.gz.

File metadata

  • Download URL: unique_search_proxy-0.2.1.tar.gz
  • Upload date:
  • Size: 10.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.11 {"installer":{"name":"uv","version":"0.10.11","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for unique_search_proxy-0.2.1.tar.gz
Algorithm Hash digest
SHA256 2267fde5c68df2627fb4d50451e4f3f667ee085d717d773edc5ff6af86e2ec4a
MD5 7fd1416f0dfd4f20ab0f30466db4bf14
BLAKE2b-256 820cca56dc0bd13c0643fa2565ff4eebbc859c0c276eb9b73b127b26d798a65a

See more details on using hashes here.

File details

Details for the file unique_search_proxy-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: unique_search_proxy-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 18.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.11 {"installer":{"name":"uv","version":"0.10.11","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for unique_search_proxy-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 0a1b585c90b061db5593944a0b50a0f2ec398da2485a4a291c70ef131d3eacd4
MD5 4e391d26f2d1710c39cd6bccaefff786
BLAKE2b-256 c27bca508583b255d515bb9760a6e2e1ee6f57952e5e8b8e8bbb669c7a356902

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page