aipip 0.1.0

AI Provider Interaction Platform - A flexible framework for interacting with various AI provider APIs.

AIPIP: AI Provider Interaction Platform

Vision

This project, AIPIP (AI Provider Interaction Platform), aims to build a flexible and extensible Python platform for interacting with various AI provider APIs. While the initial focus is on text generation using Large Language Models (LLMs), the architecture is designed to accommodate future expansion into other modalities like image generation, audio processing, streaming responses, and more complex multi-modal interactions as AI capabilities evolve.

Architecture Overview

The platform follows a modular, service-oriented architecture to promote decoupling, testability, and maintainability.

Note on Inspiration: This architecture draws inspiration from projects like aisuite, which also provide a unified interface to multiple AI providers. However, we have chosen a custom layered approach with a distinct Service Layer to better support the goal of building a broader application platform. This separation allows for more flexibility in integrating diverse functionalities (e.g., complex evaluation workflows, data analysis, problem generation) on top of the core provider interactions, leading to better separation of concerns and maintainability as the application scope grows beyond simple API calls.

1. Configuration (config/):

   • Uses Pydantic models for defining structured and validated configurations.
   • Centralized loading mechanism (e.g., from environment variables, .env files, or dedicated config files) to manage API keys, provider settings, model parameters, etc.
   • Secure handling of sensitive information like API keys.

2. Provider Abstraction (providers/interfaces/):

   • Defines abstract base classes (ABCs) or interfaces for different types of AI interactions (e.g., TextProviderInterface, ImageProviderInterface).
   • These interfaces enforce a common set of methods (e.g., generate_completion, generate_image) that specific provider implementations must adhere to.

3. Provider Implementations (providers/clients/):

   • Concrete classes implementing the provider interfaces for specific vendors (e.g., AnthropicClient, GoogleClient, OpenAIClient implementing TextProviderInterface).
   • Each client class encapsulates the logic for interacting with a specific provider's SDK/API.
   • Clients receive their necessary configuration (API key, etc.) via dependency injection during initialization, making them stateless regarding configuration loading.

4. Provider Registry/Factory (providers/registry.py):

   • A central component responsible for instantiating and managing provider client objects based on the loaded application configuration.
   • Provides a way for other parts of the application (like services) to request and obtain initialized provider instances without needing to know the instantiation details.

5. Service Layer (services/):

   • Contains modules with specific business logic (e.g., TextGenerationService).
   • Services depend on the Provider Registry to get the necessary provider clients via their interfaces.
   • Encapsulates workflows and orchestrates calls to providers. Services like TextGenerationService may offer methods for single calls (generate) or batch/comparative calls (generate_batch).

6. Application Entry Points (cli/, api/, or separate apps):

   • Entry points for interacting with the aipip library (e.g., the example aipip.cli.run_text_generation script).
   • Separate applications (like the planned evaluation_app) can be built on top of the aipip library by importing its services.

7. Utilities (utils/):

   • Shared helper functions and classes used across different parts of the application.

8. Testing (tests/):

   • Comprehensive unit and integration tests for all components, facilitated by the decoupled architecture and dependency injection.

9. Tool Calling / Function Calling:

   • Support for provider-specific tool/function calling mechanisms will be integrated.
   • The common TextProviderInterface will likely include methods or parameters to pass tool schemas and receive tool invocation requests from the LLM.
   • Provider client implementations (providers/clients/) will handle the specific API interactions for tool use.
   • Services (services/) can then orchestrate multi-turn conversations involving tool execution, potentially drawing inspiration from patterns like aisuite's automatic execution flow in later phases.

Proposed Directory Structure

.
├── src/
│   └── aipip/              # Main package source code
│       ├── __init__.py
│       ├── config/
│       │   ├── __init__.py
│       │   ├── models.py
│       │   └── loader.py
│       ├── providers/
│       │   ├── __init__.py
│       │   ├── interfaces/
│       │   │   ├── __init__.py
│       │   │   └── text_provider.py
│       │   ├── clients/
│       │   │   ├── __init__.py
│       │   │   ├── anthropic_client.py
│       │   │   ├── google_client.py
│       │   │   └── openai_client.py
│       │   └── registry.py
│       ├── services/
│       │   ├── __init__.py
│       │   └── text_generation_service.py # Provides core generation logic
│       ├── utils/
│       │   ├── __init__.py
│       │   └── helpers.py
│       └── cli/
│           ├── __init__.py
│           └── run_text_generation.py # Example CLI using the service
├── evaluation_app/         # Example application using the aipip library
│   ├── __init__.py
│   ├── run_evaluation.py
│   └── ...               # App-specific logic, prompts, etc.
├── tests/
│   ├── integration/
│   ├── providers/
│   └── ...               # Unit and integration tests
├── .gitignore
├── LICENSE
├── README.md
├── pyproject.toml        # Build system & project metadata
└── main.py               # Optional: Example top-level script (if needed)

Roadmap & Current Status

This README outlines the target architecture. We will migrate functionality from the old structure progressively.

Phase 1: Core Text Generation Setup (COMPLETE)

• Configuration System: Define Pydantic models (config/models.py) and loading mechanism (config/loader.py).
• Text Provider Interface: Define TextProviderInterface (providers/interfaces/text_provider.py).
• Provider Implementations: anthropic_client.py, google_client.py, openai_client.py classes implementing the interface.
• Provider Registry: Implement ProviderRegistry (providers/registry.py) to instantiate and provide clients.
• Text Generation Service: Create an initial TextGenerationService (services/text_generation_service.py) using the registry.
• Basic CLI Entry Point: Create a simple CLI script (cli/run_text_generation.py) to test the structure.
• Unit Tests: Add basic unit tests for config loading, registry, and provider clients (using mocks).
• Integration Tests: Add basic integration tests (tests/integration/) for providers.

Phase 2: Enhance Core & Build Example Applications

• Evaluation Application (evaluation_app/): Design and implement a separate application using aipip (and potentially generate_batch) to run logic problems against different providers/models, collect results, and potentially perform basic analysis.
• Token Counting: Add token counting capabilities to the provider interface and clients (or use external library like tiktoken), potentially exposing it via the service layer.
• Batch Generation: Enhance TextGenerationService with a generate_batch method to run the same input against multiple models/configurations for a provider.
• Tool Calling Support: Implement basic tool/function calling capabilities in the text provider interface and clients, and update the TextGenerationService to handle them.
• Refactor: Adapt prompt generation/result parsing logic into reusable utilities (aipip/utils/) or parts of the evaluation_app as needed.
• Add comprehensive tests for new core features and the evaluation application.

Phase 3: Future Enhancements & Applications (Examples)

• Logic Solution Analysis Application: A separate app using aipip to analyze/compare the quality of solutions generated for logic problems.
• Problem Generation Service/Application: Using aipip to generate new logic problems or other evaluation data.
• Image Generation Provider Interface & Implementations
• Audio Processing Provider Interface & Implementations
• Streaming Support in Providers & Services
• Advanced Error Handling, Retries, and Rate Limiting
• Adapting to Evolving Standards: Monitor and adapt provider clients and interfaces to support emerging standards for structured context communication (e.g., Anthropic's Model Context Protocol - MCP) as they gain adoption.
• Asynchronous Provider Implementations (asyncio)
• Web API (e.g., using FastAPI)
• User Interface
• Deployment Setup (Docker, CI/CD)
• Advanced Tool Calling: Implement more sophisticated tool handling (e.g., automatic execution flows).

(This list will be updated as the project progresses)

Setup & Usage

1. Prerequisites: Python 3.9+
2. Installation: Follow the Local Development Setup instructions to install the package editable along with development dependencies (pip install -e '.[dev]').
3. API Keys: Create a .env file in the project root and add your API keys:

   # .env (ensure this file is in .gitignore)
   ANTHROPIC_API_KEY="your_anthropic_key"
   GOOGLE_API_KEY="your_google_key"
   OPENAI_API_KEY="your_openai_key"
   

   Alternatively, export these as environment variables.
4. Basic CLI Usage:

   # Example: Anthropic (using --prompt, requires max_tokens)
   python -m aipip.cli.run_text_generation --provider anthropic --prompt "Haiku about clouds" --model claude-3-haiku-20240307 --max-tokens 30
   
   # Example: Google (using --messages)
   python -m aipip.cli.run_text_generation --provider google --messages user "What is AGI?"
   
   # Example: OpenAI (using --prompt and --temperature)
   python -m aipip.cli.run_text_generation --provider openai --prompt "Tell me about the Zen of Python" --temperature 0.8
   
   # Example: OpenAI (using --messages)
   python -m aipip.cli.run_text_generation --provider openai --messages user "What is the capital of France?" assistant "Paris" user "Is it sunny there?"
   

Local Development Setup

It is highly recommended to use a virtual environment for local development to isolate project dependencies.

1. Create a virtual environment:

   python -m venv .venv
   

2. Activate the environment:

   • macOS/Linux:

     source .venv/bin/activate
     

   • Windows (Command Prompt/PowerShell):

     .\.venv\Scripts\activate
     

   (Your terminal prompt should now show (.venv))

3. Install the package in editable mode with development dependencies:

   pip install -e '.[dev]'
   

   The -e flag installs the package in "editable" mode, meaning changes to the source code in src/ will be reflected immediately without needing to reinstall. The [dev] part installs the extra dependencies listed under [project.optional-dependencies.dev] in pyproject.toml (like pytest).

4. Run tests: With the virtual environment activated, you can run tests using pytest:

   pytest
   

5. Deactivate the environment when you're finished:

   deactivate
   

Testing Strategy

This project uses pytest as the testing framework.

• Tests are located in the tests/ directory.
• The structure of tests/ should mirror the structure of src/aipip/ where applicable (e.g., tests for src/aipip/config/ go into tests/config/).
• The goal is to achieve good test coverage through a combination of:
  • Unit Tests: Testing individual functions, classes, or methods in isolation.
  • Integration Tests: Testing the interaction between different components (e.g., a service interacting with a provider client).
• Focus on testing the core logic, public interfaces, and expected behaviors (including edge cases and error handling) of the package components.
• Tests can be run using the pytest command after setting up the local development environment (see "Local Development Setup" section).

Running Tests

Tests can be run using pytest after setting up the local development environment.

• Run all tests (Unit & Integration):

  pytest
  

• Run only unit tests: (Faster, no network/API keys needed)

  pytest -m "not integration"
  

• Run only integration tests: (Requires network and API keys in .env or environment)

  pytest -m integration
  

• See output during tests: To see print() statements (like the responses from integration tests), use the -s flag:

  pytest -s -m integration # Run integration tests and show output
  pytest -s # Run all tests and show output
  

Integration Tests (tests/integration/)

• Integration tests verify the interaction with live provider APIs.
• They are marked with the integration marker (configured in pyproject.toml).
• Prerequisites: Requires valid API keys for the providers being tested. These keys should be stored in a .env file in the project root (and this file should be in .gitignore) or exported as environment variables (e.g., ANTHROPIC_API_KEY, GOOGLE_API_KEY, OPENAI_API_KEY). The tests load the .env file automatically and will skip if the required key for a specific provider test is not found.
• Cost & Time: Be aware that running these tests incurs API costs and takes longer than unit tests.
• Purpose: Verify connectivity, authentication, and basic request/response compatibility with the live APIs.

Handling Upstream API Changes

This library relies on the official Python SDKs provided by the respective AI vendors (e.g., anthropic, google-generativeai, openai). Changes to these upstream SDKs or their underlying APIs can impact aipip.

Strategy:

1. Dependency Management: We specify version ranges for provider SDKs in pyproject.toml (e.g., openai>=1.0,<2.0) to prevent automatically pulling in potentially breaking major version updates. Minor/patch updates from providers will be tested before updating the lower bound.
2. Interface Stability: The core TextProviderInterface aims for stability. Common parameters are defined explicitly. Provider-specific parameters are handled via **kwargs passed directly to the client implementation, allowing flexibility without constant interface changes.
3. Client Implementation Responsibility: Each concrete client class (e.g., AnthropicClient, GoogleClient, OpenAIClient) is responsible for adapting to changes in its specific upstream SDK. This involves updating:
   • Parameter mapping (from interface calls to SDK calls).
   • Method calls to the SDK.
   • Response parsing.
4. Testing: Our unit tests for each client (e.g., test_openai_client.py) use mocking to simulate the provider SDK. These tests are crucial for detecting when an SDK update breaks our client's implementation, as the mocks or the expected call signatures/responses will no longer align.
5. Monitoring & Maintenance: We will monitor provider announcements and SDK releases. When breaking changes occur in an upstream SDK, the corresponding aipip client implementation and its tests will be updated, and a new version of aipip will be released.

This approach allows aipip to provide a consistent interface while managing the inevitable evolution of the underlying provider APIs and SDKs.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

(Contribution guidelines will be added later)

Releasing to PyPI

This project uses PyPI's trusted publishing for automated releases.

The release process is triggered automatically by pushing a Git tag that matches the version pattern v*.*.* (e.g., v0.1.0, v1.2.3).

The .github/workflows/publish-to-pypi.yml GitHub Actions workflow handles:

1. Building the source distribution and wheel.
2. Uploading the package to PyPI using the trusted publisher configuration.

No manual API token configuration is required in GitHub secrets.