dolphin-logger 0.2.0

A proxy server to log LLM requests and responses.

dolphin-logger

A proxy server that logs your LLM conversations to create datasets from your interactions with any Chat LLM. Supports multiple LLM backends including OpenAI, Anthropic, Google, and Ollama with full OpenAI API compatibility.

Quick Start

1. Install: pip install . (or pip install dolphin-logger when published)
2. Initialize: dolphin-logger init
3. Configure: Edit ~/.dolphin-logger/config.json with your API keys
4. Run: dolphin-logger (starts server on http://localhost:5001)
5. Use: Point your LLM client to http://localhost:5001 instead of the original API

Features

• Maintains OpenAI API compatibility
• Supports multiple LLM backends through configuration:
  • OpenAI (e.g., gpt-4.1)
  • Anthropic native SDK (e.g., claude-3-opus)
  • Anthropic via OpenAI-compatible API
  • Google (e.g., gemini-pro)
  • Ollama (local models e.g., codestral, dolphin)
• Configuration-based model definition using config.json
• Dynamic API endpoint selection based on the requested model in the /v1/chat/completions endpoint.
• Provides a /v1/models endpoint listing all configured models.
• Provides a /health endpoint for server status and configuration load checks.
• Supports both streaming and non-streaming responses.
• Automatic request logging to JSONL format.
• Logging suppression for specific tasks (requests starting with "### Task:").
• Error handling with detailed responses.
• Request/response logging with thread-safe implementation.
• Support for API keys via environment variables for enhanced security.
• Command-line interface for server operation, log uploading, and configuration management.

Installation

1. Clone the repository:

   git clone https://github.com/your-username/dolphin-logger.git # Replace with actual URL
   cd dolphin-logger
   

2. Install the package: It's recommended to install in a virtual environment.

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   pip install .
   

Configuration

Dolphin Logger uses a config.json file to define available LLM models and their settings.

1. Initial Setup (First-Time Users):

Run the init command to create the necessary configuration directory and a default configuration file:

dolphin-logger init

This will:

• Create the ~/.dolphin-logger/ directory if it doesn't exist.
• Copy a config.json.example file from the package to ~/.dolphin-logger/config.json.
• If ~/.dolphin-logger/config.json already exists, it will not be overwritten.

2. Configuration File Location:

The active configuration file is always expected at: ~/.dolphin-logger/config.json

You can check this path using the CLI:

dolphin-logger config --path

3. Editing config.json:

Open ~/.dolphin-logger/config.json and customize it with your desired models, API endpoints, and API keys. A config.json.example is included in the root of this repository for reference.

Example config.json structure:

{
  "huggingface_repo": "cognitivecomputations/dolphin-logger",
  "models": [
    {
      "provider": "anthropic",
      "providerModel": "claude-3-7-sonnet-latest",
      "model": "claude",
      "apiKey": "ENV:ANTHROPIC_API_KEY"
    },
    {
      "provider": "openai",
      "providerModel": "gpt-4.1",
      "model": "gpt",
      "apiBase": "https://api.openai.com/v1/",
      "apiKey": "ENV:OPENAI_API_KEY"
    },
    {
      "provider": "openai",
      "providerModel": "gemini-2.5-pro-preview-05-06",
      "model": "gemini",
      "apiBase": "https://generativelanguage.googleapis.com/v1beta/",
      "apiKey": "ENV:GOOGLE_API_KEY"
    },
    {
      "provider": "ollama",
      "providerModel": "codestral:22b-v0.1-q5_K_M",
      "model": "codestral"
    },
    {
      "provider": "ollama",
      "providerModel": "dolphin3",
      "model": "dolphin"
    }
  ]
}

Configuration fields:

• provider: The provider type:
  • "openai" for OpenAI-compatible APIs
  • "anthropic" for native Anthropic SDK (recommended for Claude models)
  • "ollama" for local Ollama models
• providerModel: The actual model name to send to the provider's API
• model: The model name that clients will use when making requests to the proxy
• apiBase: The base URL for the API. For Ollama, this defaults to http://localhost:11434/v1 if not specified. For Anthropic (using the native SDK via provider: "anthropic"), this field is not used.
• apiKey: The API key for authentication. Not needed for Ollama. This can be the actual key string or a reference to an environment variable.

Using Environment Variables for API Keys (Recommended for Security):

To avoid hardcoding API keys in your config.json, you can instruct Dolphin Logger to read them from environment variables:

• In the apiKey field for a model, use the prefix ENV: followed by the name of the environment variable. For example: "apiKey": "ENV:MY_OPENAI_API_KEY"
• Dolphin Logger will then look for an environment variable named MY_OPENAI_API_KEY and use its value.
• If the specified environment variable is not set at runtime, a warning will be logged during startup, and the API key for that model will be treated as missing (effectively None). This might lead to authentication errors if the provider requires a key.

Benefits:

• Enhanced Security: Keeps sensitive API keys out of configuration files, which might be accidentally committed to version control.
• Flexibility: Allows different API keys for different environments (development, staging, production) without changing config.json. Ideal for Docker deployments and CI/CD pipelines.

Example config.json entry:

    {
      "provider": "openai",
      "providerModel": "gpt-4-turbo",
      "model": "gpt-4-turbo-secure",
      "apiBase": "https://api.openai.com/v1",
      "apiKey": "ENV:OPENAI_API_KEY" 
    }

In this case, you would set the OPENAI_API_KEY environment variable in your system before running Dolphin Logger.

Note for Anthropic models:

• Using the "anthropic" provider is recommended as it uses the official Anthropic Python SDK
• This provides better performance and reliability compared to using Claude through an OpenAI-compatible API

Note for Ollama models:

• If apiBase is not specified for an Ollama provider, it defaults to http://localhost:11434/v1.
• No API key is required for local Ollama models.

4. Validate Configuration (Optional): After editing, you can validate your config.json:

dolphin-logger config --validate

This will check for JSON syntax errors and basic structural issues.

Command-Line Interface (CLI)

Dolphin Logger is managed via the dolphin-logger command-line tool.

dolphin-logger [command] [options]

Available Commands:

• server (default)

  • Starts the LLM proxy server.
  • This is the default action if no command is specified.
  • Example:

    dolphin-logger
    # or explicitly
    dolphin-logger server
    

  • The server will run on port 5001 by default (configurable via the PORT environment variable).

• upload

  • Uploads collected .jsonl log files from ~/.dolphin-logger/logs/ to a configured Hugging Face Hub dataset repository.
  • Example:

    dolphin-logger upload
    

  • See the "Uploading Logs" section for prerequisites and details.

• init

  • Initializes the Dolphin Logger configuration.
  • Creates the ~/.dolphin-logger/ directory if it doesn't exist.
  • Copies a default config.json.example to ~/.dolphin-logger/config.json if no config.json is present. This file serves as a template for your actual configuration.
  • Example:

    dolphin-logger init
    

• config

  • Manages or inspects the configuration.
  • --path: Shows the expected absolute path to the config.json file.

    dolphin-logger config --path
    

  • --validate: Attempts to load and validate the config.json file, checking for JSON correctness and basic structure. It will also report on API keys resolved from environment variables.

    dolphin-logger config --validate
    

Using the Proxy Server

Once the server is running (using dolphin-logger or dolphin-logger server):

1. List available models: You can check the available models by calling the /v1/models endpoint:

   curl http://localhost:5001/v1/models 
   

   This will return a list of models as defined in your ~/.dolphin-logger/config.json.

2. Make chat completion requests: Use the proxy as you would the OpenAI API, but point your client's base URL to http://localhost:5001 (or your configured port). Include the model name (as defined in the model field in your config.json) in your request.

   cURL example using a model named "claude":

   curl http://localhost:5001/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer dummy-token" \
     -d '{
       "model": "claude",
       "messages": [{"role": "user", "content": "Hello from Claude!"}],
       "stream": false
     }'
   

   Python OpenAI SDK example:

   from openai import OpenAI
   
   # Point to your local dolphin-logger proxy
   client = OpenAI(
       base_url="http://localhost:5001/v1",
       api_key="dummy-key"  # Not validated by proxy
   )
   
   response = client.chat.completions.create(
       model="claude",  # Use model name from your config
       messages=[{"role": "user", "content": "Hello!"}]
   )
   print(response.choices[0].message.content)
   

   Using with popular tools:

   • Cursor/Continue.dev: Set API base URL to http://localhost:5001/v1
   • LangChain: Use openai_api_base="http://localhost:5001/v1"
   • Any OpenAI-compatible client: Point base URL to your proxy

3. Check Server Health: Verify server status and configuration load:

   curl http://localhost:5001/health
   

   Expected response (healthy):

 {
   "status": "ok",
   "message": "Server is healthy, configuration loaded."
 }
 ```
 *If configuration issues exist (e.g., no models loaded):*
 ```json
 {
   "status": "error",
   "message": "Server is running, but configuration might have issues (e.g., no models loaded)."
 }
 ```

## Environment Variables

The proxy primarily uses the following environment variables:

- `PORT`: Sets the port on which the proxy server will listen (default: `5001`).
- **API Keys (Dynamic)**: Any environment variable that you reference in your `config.json` using the `ENV:` prefix for `apiKey` fields (e.g., if you have `"apiKey": "ENV:OPENAI_API_KEY"`, then `OPENAI_API_KEY` becomes a relevant environment variable).
- `HF_TOKEN` (for uploading logs): A Hugging Face Hub token with write access to the target dataset repository is required when using the `dolphin-logger upload` command.

## Logging

- All proxied requests and their corresponding responses to `/v1/chat/completions` are automatically logged.
- Logs are stored in date-specific `.jsonl` files (one line per JSON object).
- Log files are named with UUIDs to ensure uniqueness (e.g., `~/.dolphin-logger/logs/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.jsonl`).
- Logging is thread-safe.
- **Log Suppression:** If the first user message in a request starts with `### Task:`, that specific request/response pair will not be logged. This is useful for excluding specific interactions (e.g., meta-prompts or system commands) from your dataset.

## Uploading Logs

The `dolphin-logger upload` command facilitates uploading your collected logs to a Hugging Face Hub dataset.

**Prerequisites:**
- **Hugging Face Token:** Ensure the `HF_TOKEN` environment variable is set with a Hugging Face token. This token must have write access to the target dataset repository.
- **Target Repository:** The default target dataset is `cognitivecomputations/dolphin-logger`. This can be changed by modifying the `DATASET_REPO_ID` variable within the `dolphin_logger/upload.py` file if needed.

**Process:**
1.  Run: `dolphin-logger upload`
2.  The tool will:
 *   Locate all `.jsonl` files in `~/.dolphin-logger/logs/`.
 *   Create a new branch in the target Hugging Face dataset (e.g., `upload-logs-YYYYMMDD-HHMMSS`).
 *   Commit the log files to this new branch.
 *   Attempt to create a Pull Request (Draft) from this new branch to the dataset's main branch.
 *   Print the URL of the created Pull Request.
3.  You will need to visit the URL to review and merge the Pull Request on Hugging Face Hub.

## Troubleshooting

**Common Issues and Solutions:**

1. **"Configuration file not found" error:**
- Run `dolphin-logger init` to create the default configuration
- Check that `~/.dolphin-logger/config.json` exists with `dolphin-logger config --path`

2. **"Template config file not found" error:**
- Ensure you've installed the package properly with `pip install .`
- Verify the `config.json.example` file exists in the project root

3. **API authentication errors:**
- Verify your API keys are correctly set in environment variables
- Check that environment variable names match those specified in config (e.g., `ENV:OPENAI_API_KEY` requires `OPENAI_API_KEY` to be set)
- Use `dolphin-logger config --validate` to check API key resolution

4. **Server won't start / Port already in use:**
- Check if another process is using port 5001: `lsof -i :5001`
- Set a different port: `PORT=5002 dolphin-logger`
- Kill existing processes if needed

5. **Models not appearing in `/v1/models` endpoint:**
- Validate your configuration: `dolphin-logger config --validate`
- Check that your config.json has a properly formatted "models" array
- Restart the server after configuration changes

6. **Ollama models not working:**
- Ensure Ollama is running: `ollama list`
- Check that the model names in your config match available Ollama models
- Verify Ollama is accessible at `http://localhost:11434`

7. **Logs not being created:**
- Check that requests don't start with "### Task:" (these are suppressed by default)
- Verify the `~/.dolphin-logger/logs/` directory exists and is writable
- Look for error messages in the server output

**Getting Help:**
- Enable verbose logging with detailed error messages
- Check the server console output for specific error details
- Validate your configuration with `dolphin-logger config --validate`
- Ensure all required environment variables are set

## Error Handling

The proxy includes comprehensive error handling:
- Preserves original error messages from upstream APIs when available.
- Provides detailed error information in JSON format for debugging.
- Maintains appropriate HTTP status codes for different error types.

## Project Structure

The `dolphin-logger` codebase is organized into several modules within the `src/dolphin_logger/` directory:

- `cli.py`: Handles command-line argument parsing and dispatches to appropriate functions for different commands (`server`, `upload`, `init`, `config`).
- `server.py`: Contains the Flask application setup, route definitions (`/health`, `/v1/models`, and the main proxy route), and the main server running logic.
- `core_proxy.py`: Implements the core logic for proxying requests. This includes selecting the target API based on configuration, and separate handlers for Anthropic SDK requests and general REST API requests.
- `config.py`: Manages configuration loading (from `~/.dolphin-logger/config.json`), creation of default configuration, and resolution of API keys from environment variables.
- `logging_utils.py`: Provides utilities for managing log files (daily rotation, UUID naming) and determining if a request should be logged.
- `upload.py`: Contains the logic for finding log files and uploading them to a Hugging Face Hub dataset.
- `main.py`: The primary entry point for the `dolphin-logger` script, which calls the main CLI function from `cli.py`.
- `__init__.py`: Makes `dolphin_logger` a Python package and defines the package version.

## Testing

The project includes a suite of unit and functional tests to ensure reliability and prevent regressions.

**Tools Used:**
- `pytest`: For test discovery and execution.
- `pytest-mock`: For mocking dependencies in unit tests.
- Standard Python `unittest.mock` and `subprocess` modules.

**Running Tests:**
1.  Ensure you have installed the development dependencies. If a `dev` extra is defined in `pyproject.toml` (e.g., `pip install .[dev]`), use that. Otherwise, install test tools manually:
 ```bash
 pip install pytest pytest-mock requests
 ```
2.  Navigate to the root directory of the project.
3.  Run the tests:
 ```bash
 python -m pytest tests/
 ```
 Or more simply:
 ```bash
 pytest tests/
 ```

**Test Environment Notes:**
- **Unit Tests (`tests/test_*.py` excluding `test_functional.py`):** These are self-contained and mock external dependencies like file system operations, API calls, and environment variables. They test individual modules in isolation.
- **Functional Tests (`tests/test_functional.py`):**
 - These tests start an actual instance of the Dolphin Logger server on a free port using a subprocess.
 - They use a temporary, isolated configuration directory and log directory for each test session, ensuring they **do not interfere with your user-level `~/.dolphin-logger` setup.**
 - The functional tests primarily verify the server's behavior with configurations that point to **non-existent backend services.** This allows testing of the proxy's routing, error handling, and logging mechanisms when upstream services are unavailable, without requiring actual LLM API keys or running local LLMs during the test execution.

## License

MIT