Skip to main content

AI discovers APIs. Code syncs data. No adapters to write.

Project description

Liquid

AI discovers APIs. Code syncs data. No adapters to write.

Tests License Python Version


Point Liquid at any URL. AI discovers the API, proposes field mappings to your data model, and generates a deterministic adapter. After human approval, sync runs on schedule with zero LLM calls.

URL  ──→  AI discovers API  ──→  Human verifies mapping  ──→  Deterministic sync
          (once)                  (one-time review)             (forever, no LLM)

The Problem

Connecting to external APIs requires custom code per service. 50 services = 50 adapters. Each with unique endpoints, auth flows, pagination, and data models. Writing and maintaining them doesn't scale.

The Solution

from liquid import Liquid, SyncConfig
from liquid._defaults import InMemoryVault, CollectorSink

client = Liquid(llm=my_llm, vault=InMemoryVault(), sink=CollectorSink())

# 1. AI discovers the API (once)
schema = await client.discover("https://api.shopify.com")

# 2. AI proposes field mappings → human reviews
review = await client.propose_mappings(schema, {"amount": "float", "date": "datetime"})
review.approve_all()

# 3. Create adapter config
config = await client.create_adapter(
    schema=schema,
    auth_ref="vault/shopify",
    mappings=review.finalize(),
    sync_config=SyncConfig(endpoints=["/orders"], schedule="0 */6 * * *"),
)

# 4. Deterministic sync — no AI, runs forever
result = await client.sync(config)
print(f"Synced {result.records_delivered} records")

How Discovery Works

Liquid tries the cheapest method first, falls through on failure:

Priority Strategy When it works AI needed?
1 MCP Service publishes an MCP server No
2 OpenAPI Has /openapi.json or /swagger.json No
3 GraphQL Has /graphql with introspection No
4 REST Heuristic REST API without spec Yes (once)
5 Browser No API at all — capture network traffic Yes (once)

Key Features

Progressive Discovery — MCP → OpenAPI → GraphQL → REST → Browser. Cheapest first.

Selective Re-mapping — When APIs change, repair_adapter() diffs schemas and re-maps only broken fields. Working mappings stay untouched.

Safe Transforms — Field transforms like value * -1 or value.lower() are evaluated via AST whitelisting. No eval(), no injection risk.

Pluggable Pagination — Cursor, offset, page number, link header. Each is a strategy, not a switch/case.

Learning System — Corrections improve future proposals. Connect Shopify for the 51st time → mapping is instant.

Auth Classification — Detects OAuth (Tier A), app registration (Tier B), or manual credentials (Tier C). Returns structured escalation info.

Installation

pip install liquid               # core
pip install liquid[mcp]          # + MCP server discovery
pip install liquid[browser]      # + Playwright browser discovery

Architecture

┌─────────────┐    ┌──────────────┐    ┌────────────────┐    ┌─────────────┐
│  Discovery   │──→│  Auth Setup  │──→│  Field Mapping  │──→│ Sync Engine │
│  (AI, once)  │   │ (AI + human) │   │ (AI + human)    │   │ (code, loop)│
└─────────────┘    └──────────────┘    └────────────────┘    └─────────────┘

Liquid is a library, not a framework. You control when to discover, how to present mappings, where to store configs, and what to do with synced data.

Extension Points (Protocols)

Protocol Purpose You provide
Vault Credential storage Postgres, AWS Secrets Manager, etc.
LLMBackend AI provider Claude, GPT, Llama, any LLM
DataSink Where data goes Database, queue, webhook, file
KnowledgeStore Shared mappings Redis, central registry, or disabled

Auto-Repair on API Changes

When an API breaks your adapter:

result = await client.repair_adapter(config, target_model, auto_approve=True)
# Re-discovers → diffs schemas → selectively re-maps broken fields
# Returns updated AdapterConfig or MappingReview for human review

Liquid vs Alternatives

Liquid Airbyte Nango Custom code
New service discover(url) Write connector YAML Write TypeScript sync Write adapter from scratch
AI involvement Discovery only, then deterministic None AI-generated code None
Auth handling Classifies & escalates Per-connector Managed OAuth Manual
When API changes repair_adapter() Update connector Update sync code Debug & fix
Runtime LLM calls Zero Zero Zero N/A
Self-hosted Yes (library) Yes (platform) Yes (platform) Yes
License AGPL-3.0 ELv2 AGPL-3.0 Yours

Documentation

Contributing

We welcome contributions! Check out our contributing guide and browse good first issues.

License

AGPL-3.0. Commercial licenses available — contact us.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

liquid_api-0.2.0.tar.gz (57.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

liquid_api-0.2.0-py3-none-any.whl (40.6 kB view details)

Uploaded Python 3

File details

Details for the file liquid_api-0.2.0.tar.gz.

File metadata

  • Download URL: liquid_api-0.2.0.tar.gz
  • Upload date:
  • Size: 57.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for liquid_api-0.2.0.tar.gz
Algorithm Hash digest
SHA256 394d5239f20fd9b1af0afacee7120aa2ebd6336e2fd0596637f462f6dca31c03
MD5 c1a26f2e7996a5f62016d4c19c84a4a6
BLAKE2b-256 a127425c1ce91a4b4e4dc21bb2d063e8dba971dcfa47bf9ede5b9522da179184

See more details on using hashes here.

File details

Details for the file liquid_api-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: liquid_api-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 40.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for liquid_api-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 a7a4edb756e8df1e8057498adbe15c657013084f20976cb958bde94088bbcff1
MD5 005a429fa84f4f1145317bf992409b47
BLAKE2b-256 6206396b68789c915423cf21bb28d1899d10e5d70ebc6c17187c3948c2360bca

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page