Skip to main content

AI discovers APIs. Code syncs data. No adapters to write.

Project description

Liquid

AI discovers APIs. Code syncs data. No adapters to write.

Tests License Python Version


Point Liquid at any URL. AI discovers the API, proposes field mappings to your data model, and generates a deterministic adapter. After human approval, sync runs on schedule with zero LLM calls.

URL  ──→  AI discovers API  ──→  Human verifies mapping  ──→  Deterministic sync
          (once)                  (one-time review)             (forever, no LLM)

The Problem

Connecting to external APIs requires custom code per service. 50 services = 50 adapters. Each with unique endpoints, auth flows, pagination, and data models. Writing and maintaining them doesn't scale.

The Solution

from liquid import Liquid, SyncConfig
from liquid._defaults import InMemoryVault, CollectorSink

client = Liquid(llm=my_llm, vault=InMemoryVault(), sink=CollectorSink())

# 1. AI discovers the API (once)
schema = await client.discover("https://api.shopify.com")

# 2. AI proposes field mappings → human reviews
review = await client.propose_mappings(schema, {"amount": "float", "date": "datetime"})
review.approve_all()

# 3. Create adapter config
config = await client.create_adapter(
    schema=schema,
    auth_ref="vault/shopify",
    mappings=review.finalize(),
    sync_config=SyncConfig(endpoints=["/orders"], schedule="0 */6 * * *"),
)

# 4. Deterministic sync — no AI, runs forever
result = await client.sync(config)
print(f"Synced {result.records_delivered} records")

How Discovery Works

Liquid tries the cheapest method first, falls through on failure:

Priority Strategy When it works AI needed?
1 MCP Service publishes an MCP server No
2 OpenAPI Has /openapi.json or /swagger.json No
3 GraphQL Has /graphql with introspection No
4 REST Heuristic REST API without spec Yes (once)
5 Browser No API at all — capture network traffic Yes (once)

Key Features

Progressive Discovery — MCP → OpenAPI → GraphQL → REST → Browser. Cheapest first.

Selective Re-mapping — When APIs change, repair_adapter() diffs schemas and re-maps only broken fields. Working mappings stay untouched.

Safe Transforms — Field transforms like value * -1 or value.lower() are evaluated via AST whitelisting. No eval(), no injection risk.

Pluggable Pagination — Cursor, offset, page number, link header. Each is a strategy, not a switch/case.

Learning System — Corrections improve future proposals. Connect Shopify for the 51st time → mapping is instant.

Auth Classification — Detects OAuth (Tier A), app registration (Tier B), or manual credentials (Tier C). Returns structured escalation info.

Installation

pip install liquid               # core
pip install liquid[mcp]          # + MCP server discovery
pip install liquid[browser]      # + Playwright browser discovery

Architecture

┌─────────────┐    ┌──────────────┐    ┌────────────────┐    ┌─────────────┐
│  Discovery   │──→│  Auth Setup  │──→│  Field Mapping  │──→│ Sync Engine │
│  (AI, once)  │   │ (AI + human) │   │ (AI + human)    │   │ (code, loop)│
└─────────────┘    └──────────────┘    └────────────────┘    └─────────────┘

Liquid is a library, not a framework. You control when to discover, how to present mappings, where to store configs, and what to do with synced data.

Extension Points (Protocols)

Protocol Purpose You provide
Vault Credential storage Postgres, AWS Secrets Manager, etc.
LLMBackend AI provider Claude, GPT, Llama, any LLM
DataSink Where data goes Database, queue, webhook, file
KnowledgeStore Shared mappings Redis, central registry, or disabled

Auto-Repair on API Changes

When an API breaks your adapter:

result = await client.repair_adapter(config, target_model, auto_approve=True)
# Re-discovers → diffs schemas → selectively re-maps broken fields
# Returns updated AdapterConfig or MappingReview for human review

Liquid vs Alternatives

Liquid Airbyte Nango Custom code
New service discover(url) Write connector YAML Write TypeScript sync Write adapter from scratch
AI involvement Discovery only, then deterministic None AI-generated code None
Auth handling Classifies & escalates Per-connector Managed OAuth Manual
When API changes repair_adapter() Update connector Update sync code Debug & fix
Runtime LLM calls Zero Zero Zero N/A
Self-hosted Yes (library) Yes (platform) Yes (platform) Yes
License AGPL-3.0 ELv2 AGPL-3.0 Yours

Documentation

Contributing

We welcome contributions! Check out our contributing guide and browse good first issues.

License

AGPL-3.0. Commercial licenses available — contact us.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

liquid_api-0.3.0.tar.gz (59.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

liquid_api-0.3.0-py3-none-any.whl (41.3 kB view details)

Uploaded Python 3

File details

Details for the file liquid_api-0.3.0.tar.gz.

File metadata

  • Download URL: liquid_api-0.3.0.tar.gz
  • Upload date:
  • Size: 59.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for liquid_api-0.3.0.tar.gz
Algorithm Hash digest
SHA256 38b59e54f2539fe64eeca53a695c0cf085ad5b4759c849310c74ba1a94a14c3a
MD5 a575c74940f3ca48f4c8180e90ee1c13
BLAKE2b-256 3a1acc682bb3e69a4526aac2e2c77c723322ce8ffa3a05371d50ff57c786c80d

See more details on using hashes here.

File details

Details for the file liquid_api-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: liquid_api-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 41.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for liquid_api-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c884bec28e4f082ea1d220f19f8039990c133028ae0141315c77de29578c2351
MD5 57acef9e279dd26c52f00d59bd0a6090
BLAKE2b-256 70a31beb271c6e84f5d56c74e72b5c48ea90e88aa272ca149894d4a0704fa277

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page