promptmask 0.1.0

A local-first privacy layer for Large Language Models.

PromptMask

A local-first privacy layer for Large Language Models.

Cloud AI is smart but sacrifices privacy.
Local AI keeps your secret but is dumb.
What if we can combine the advantages of both sides?

PromptMask ensures your private data never leaves your machines. It redacts and un-redacts sensitive data locally, so that only anonymized data is sent to third-party AI services.

Table of Contents

• Table of Contents
• How It Works
• Quickstart
  • Prerequisites
  • For General Users: local OpenAI-compatible API Gateway
  • For Python Developers: OpenAIMasked
• Configuration
• Advanced Usage: PromptMask Class
• Web Server: WebUI & API
• Development & Contribution
• License

How It Works

The core principle is to use a trusted (local) model as a "privacy filter" for a powerful, remote model. The process is fully automated.

Quickstart

Prerequisites

Ensure you have a local LLM running with an OpenAI-compatible API endpoint. Ollama is a popular and straightforward option. By default, PromptMask will attempt to connect to http://localhost:11434/v1.

Other options to run a local OpenAI-compatible LLM API include llama.cpp and vLLM.

For General Users: local OpenAI-compatible API Gateway

You can point any existing tool or application at the local gateway. It's the seamless way to add PromptMask layer without coding in Python.

1. Install promptmask-web via pip:

   pip install "promptmask[web]"
   

2. Run the web server:

   promptmask-web
   

   The gateway is now running at http://localhost:8000.

3. Use the gateway endpoint: Simply replace the official OpenAI API base URL with the local gateway's URL in your tool of choice.

   For example, using curl:

   curl http://localhost:8000/gateway/v1/chat/completions \
     -H "Authorization: Bearer $YOUR_OPENAI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "gpt-99-ultra",
       "messages": [
         {
           "role": "user",
           "content": "My name is Ho Shih-Chieh and my appointment ID is Y1a2e87. I booked a dental appointment on Oct 26, but I have to cancel for a meeting. Please help me write a cancellation request email in French."
         }
       ]
     }'
   

   Your sensitive data (Ho Shih Chieh, Y1a2e87) will be redacted before being sent to OpenAI, and then restored in the final response.

   If you are using other cloud AI providers, such as Google Gemini, you need to add web.upstream_oai_api_base to your config file (more detail on configuration section)

   [web]
   upstream_oai_api_base = "https://generativelanguage.googleapis.com/v1beta/openai"
   

For Python Developers: OpenAIMasked

The OpenAIMasked class is a drop-in replacement for the standard openai.OpenAI client.

1. Install the base package:

   pip install promptmask
   

2. Mask the OpenAI client in your code: The adapter automatically handles masking/unmasking for standard and streaming requests.

   Simply replace the standard openai.OpenAI client as follows:

   # from openai import OpenAI
   from promptmask import OpenAIMasked
   # client = OpenAI()
   client = OpenAIMasked()
   

   Full example:

   from promptmask import OpenAIMasked
   
   # This client has the same interface as openai.OpenAI, but with automatic privacy redaction.
   client = OpenAIMasked(base_url="https://api.cloud-ai-service.example.com/v1") # reads OPENAI_API_KEY from environment variables by default.
   
   # --- Standard Request ---
   response = client.chat.completions.create(
       model="gpt-100-pro",
       messages=[
           {"role": "user", "content": "My user ID is johndoe and my phone number is 4567890. Please help me write an application letter."}
       ]
   )
   # The response content is automatically unmasked.
   print(response.choices[0].message.content)
   
   # --- Streaming Request ---
   stream = client.chat.completions.create(
       model="gpt-101-turbo-mini",
       stream=True,
       messages=[
           {"role": "user", "content": "My patient, Jensen Huang (Patient ID: P123456789), is taking metformin and is experiencing nausea. What are the common side effects and management strategies?"}
       ]
   )
   
   # The stream chunks are unmasked on-the-fly.
   for chunk in stream:
       print(chunk.choices[0].delta.content or "", end="")
   

Configuration

To customize, create a promptmask.config.user.toml file in your working directory. For example, to change the categories of data to mask:

# promptmask.config.user.toml

[llm_api]
# Specify a particular local model to use for masking
model = "qwen2.5:7b"

# Define what data is considered sensitive.
[sensitive]
# Override the default one
include = "personal ID and passwords"

Check promptmask.config.default.toml for a full config file example.

Environment variables can also be used to override specific settings:

• LOCALAI_API_BASE: The Base URL for your local LLM's API (e.g., http://192.168.1.234:11434/v1).
• LOCALAI_API_KEY: The API key for your local LLM, if required.

Configuration Priority Hierarchy

PromptMask is configured through a hierarchy of sources, from highest to lowest priority:

0. LOCALAI_API_BASE and LOCALAI_API_KEY environment variables.
1. A dict passed directly to the PromptMask constructor (config parameter).
2. A path to a TOML file (config_file parameter).
3. A promptmask.config.user.toml file in the current working directory.
4. The packaged promptmask.config.default.toml.

Advanced Usage: PromptMask Class

For more granular control, you can import the PromptMask class directly to perform masking and unmasking as separate steps.

import asyncio # PromptMask also runs syncrounously
from promptmask import PromptMask

async def main():
    masker = PromptMask()

    original_text = "Please process the visa application for Jensen Huang, passport number A12345678."

    # 1. Mask the text
    masked_text, mask_map = await masker.async_mask_str(original_text)

    print(f"Masked Text: {masked_text}")
    # Expected output (may vary): Masked Text: Please process the visa application for ${PERSON_NAME}, passport number ${PASSPORT_NUMBER}.
    
    print(f"Mask Map: {mask_map}")
    # Expected output: Mask Map: {"Jensen Huang": "${PERSON_NAME}", "A12345678": "${PASSPORT_NUMBER}"}

    # (Imagine sending masked_text to a remote API and getting a response)
    remote_response_text = "The visa application for ${PERSON_NAME} with passport ${PASSPORT_NUMBER} is now under review."

    # 2. Unmask the response
    unmasked_response = masker.unmask_str(remote_response_text, mask_map)
    print(f"Unmasked Response: {unmasked_response}")
    # Expected output: Unmasked Response: The visa application for Jensen Huang with passport A12345678 is now under review.

if __name__ == "__main__":
    asyncio.run(main())

Web Server: WebUI & API

When you run promptmask-web with the installed promptmask[web], a full-featured web service is launched at http://localhost:8000. It includes:

• A simple Web UI
  • to try out features including masking/unmasking and the gateway
  • A Configuration Manager to view and hot-reload settings.
• Interactive API documentation (via Swagger UI) at http://localhost:8000/docs
  • API Gateway at /gateway/v1/chat/completions to take care of your privacy seamlessly.
  • Direct Masking/Unmasking API Endpoints (details on API documentation).
  • Edit configuration via /config endpoint.

Development & Contribution

Contributions are welcome.

1. Clone the repository:

   git clone https://github.com/cxumol/promptmask.git
   cd promptmask
   

2. Install in editable mode with development dependencies:

   pip install -e ".[dev,web]"
   

3. Run tests:

   pytest
   

4. Lint and format code (optional): ruff for linting and formatting, if you want.

   ruff check .
   ruff format .
   

Please open an issue or submit a pull request for any bugs or feature proposals.

License

PromptMask is distributed under the MIT License. See LICENSE for more information.