Skip to main content

Web Search Proxy implementation

Project description

Unique Search Proxy

A unified web search proxy API that provides a consistent interface for multiple search backends. Built with FastAPI and designed for seamless integration with AI applications.

Overview

This service acts as an abstraction layer over different search providers, allowing clients to switch between search engines without changing their integration code. Currently supports:

Engine Description
Google Custom Search Direct integration with Google's Custom Search JSON API
Vertex AI (Gemini) AI-powered search using Google's Gemini models with grounding capabilities

Quick Start

Prerequisites

  • Python 3.12+
  • uv for dependency management
  • Google Cloud credentials (for Vertex AI)
  • Google Custom Search API key and Engine ID (for Google Search)

Installation

# Install dependencies
uv sync

# Copy and configure environment variables
cp .env.example .env

Environment Variables

# Google Custom Search
GOOGLE_SEARCH_API_KEY=your-api-key
GOOGLE_SEARCH_API_ENDPOINT=https://www.googleapis.com/customsearch/v1
GOOGLE_SEARCH_ENGINE_ID=your-engine-id

# Vertex AI
VERTEXAI_SERVICE_ACCOUNT_CREDENTIALS=path/to/credentials.json

Running the Service

Development:

uv run python -m unique_search_proxy.web.app

Docker (from published package — hash-verified):

CI generates a hash-pinned requirements.txt from uv.lock and passes it into the Docker build. Dependencies are installed with --require-hashes, then the package itself is installed with --no-deps. To reproduce locally:

uv export --locked --package unique-search-proxy --no-dev --no-emit-project \
  -o deploy/requirements.txt
docker build --build-arg PACKAGE_VERSION=0.2.0 -t search-proxy deploy/

Every transitive dependency is verified against its sha256 hash from the lockfile.

Docker (from local source — no registry required):

Build a wheel first, copy it into deploy/, then reference it:

uv build --wheel --out-dir deploy/
docker build \
  --build-arg LOCAL_WHEEL=unique_search_proxy-0.2.0-py3-none-any.whl \
  -t search-proxy deploy/

Running the container:

docker run --rm -p 8080:8080 search-proxy

# With custom environment variables
docker run --rm -p 8080:8080 -e WORKERS=8 -e LOG_LEVEL=debug search-proxy

API Documentation

FastAPI provides automatic interactive API documentation:

URL Description
/docs Swagger UI - interactive API explorer
/redoc ReDoc - alternative documentation
/openapi.json OpenAPI schema

API Reference

Health Check

GET /health

Response:

{
  "status": "healthy"
}

Search

POST /search
Content-Type: application/json

Request Body:

Field Type Required Description
search_engine string No "google" or "vertexai" (default: "google")
query string Yes The search query
kwargs object No Engine-specific parameters

Response:

{
  "results": [
    {
      "url": "https://example.com/article",
      "title": "Article Title",
      "snippet": "A brief description of the content...",
      "content": ""
    }
  ]
}

Search Engine Configuration

Google Custom Search

Uses Google's Custom Search JSON API for traditional web search results.

Parameters (kwargs):

Parameter Type Default Description
cx string env default Custom Search Engine ID (overrides env)
fetchSize int 10 Number of results to fetch
timeout int 10 Request timeout in seconds

Example:

{
  "search_engine": "google",
  "query": "latest AI developments",
  "kwargs": {
    "fetchSize": 20,
    "timeout": 15
  }
}

Vertex AI (Gemini)

Leverages Google's Gemini models with web grounding for AI-enhanced search results. This engine:

  1. Uses Gemini to search and synthesize information from the web
  2. Generates structured results with citations
  3. Optionally resolves shortened/redirect URLs to final destinations

Parameters (kwargs):

Parameter Type Default Description
modelName string "gemini-2.5-flash" Gemini model to use
entrepriseSearch bool false Use Enterprise Web Search
systemInstruction string (built-in) Custom system prompt
resolveUrls bool true Resolve redirect URLs

Example:

{
  "search_engine": "vertexai",
  "query": "Compare the top 3 cloud providers for ML workloads",
  "kwargs": {
    "modelName": "gemini-2.5-flash",
    "resolveUrls": true
  }
}

Project Structure

connectors/unique_search_proxy/
├── unique_search_proxy/          # Python package (published to PyPI)
│   ├── __init__.py
│   └── web/                      # Web search API sub-module
│       ├── __init__.py
│       ├── app.py                # FastAPI application
│       ├── settings.py           # Global settings
│       └── core/                 # Search engine implementations
│           ├── schema.py         # Shared schemas
│           ├── google_search/    # Google Custom Search backend
│           └── vertexai/         # Vertex AI (Gemini) backend
├── tests/                        # Test suite
├── deploy/                       # Container build artifacts
│   ├── Dockerfile                # Hash-verified install or local wheel
│   └── entrypoint.sh
└── pyproject.toml

The package uses a sub-module hierarchy (web/) to support future extensions (e.g. internal/ search) that can be deployed as separate containers from the same package.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                       FastAPI App                           │
│                      /search endpoint                       │
└─────────────────────────┬───────────────────────────────────┘
                          │
                    ┌─────▼─────┐
                    │  Factory  │
                    └─────┬─────┘
                          │
          ┌───────────────┼───────────────┐
          │                               │
    ┌─────▼─────┐                   ┌─────▼─────┐
    │  Google   │                   │ Vertex AI │
    │  Search   │                   │  (Gemini) │
    └───────────┘                   └───────────┘

The service uses a factory pattern to register and resolve search engines, making it easy to add new backends.

Error Handling

All errors return a consistent format:

{
  "status": "failed",
  "error": "Error description"
}
Status Code Description
400 Validation error (invalid request)
500 Internal server error

Production Deployment

The service includes a production-ready deploy/entrypoint.sh that uses Uvicorn:

Variable Default Description
HOST 0.0.0.0 Bind address
PORT 8080 Listen port
WORKERS 4 Uvicorn workers
TIMEOUT 120 Keep-alive timeout
LOG_LEVEL info Logging verbosity

Development

# Run with hot reload
uv run uvicorn unique_search_proxy.web.app:app --reload --port 2349

# Format code
uv run ruff format .

# Lint
uv run ruff check .

# Run tests
uv run pytest

# Type check
uv run basedpyright

License

Proprietary - Unique AG

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

unique_search_proxy-0.2.0.tar.gz (10.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

unique_search_proxy-0.2.0-py3-none-any.whl (18.7 kB view details)

Uploaded Python 3

File details

Details for the file unique_search_proxy-0.2.0.tar.gz.

File metadata

  • Download URL: unique_search_proxy-0.2.0.tar.gz
  • Upload date:
  • Size: 10.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.11 {"installer":{"name":"uv","version":"0.10.11","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for unique_search_proxy-0.2.0.tar.gz
Algorithm Hash digest
SHA256 ff76358480eb25435d36f830c36aa97f4aa3fdc94f85970b6bb305faa1cb6142
MD5 026ae98c6e311609c873a9efca1067e1
BLAKE2b-256 755314fa796057033ba644ce32db3b8fec06f7f4689d68dc4e6ee955c9966e75

See more details on using hashes here.

File details

Details for the file unique_search_proxy-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: unique_search_proxy-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 18.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.11 {"installer":{"name":"uv","version":"0.10.11","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for unique_search_proxy-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 de55f0f95770f96a99dc79114802f601f434c7a33d10c9162f9217fdfc628112
MD5 fec456bd3753ba49e5b647a77ef4c529
BLAKE2b-256 b69f7f4ef4c2c2019bb688c17abe2e1196a64e1360ff973b7b00864b309d800f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page