dlt-community-sources 0.7.0

Community-maintained dlt sources for various APIs

dlt-community-sources

Community-maintained dlt sources for various APIs. Install only what you need via extras.

Available Sources

Source	Extra	Description
App Store Connect	app-store-connect	Apple App Store Connect API
Twilio	twilio	Twilio SMS, Calls, Usage Records
Meta Ads	meta-ads	Meta (Facebook) Ads campaigns, ad sets, ads, and insights
NextDNS	nextdns	NextDNS logs and analytics
Yahoo Ads Search	yahoo-ads-search	Yahoo Japan Ads Search API (SS) with MCC support

Installation

pip install dlt-community-sources[app-store-connect]

Or with uv:

uv add dlt-community-sources[app-store-connect]

Usage

import dlt
from dlt_community_sources.twilio import twilio_source

pipeline = dlt.pipeline(
    pipeline_name="twilio",
    destination="duckdb",
    dataset_name="twilio_data",
)

source = twilio_source(
    account_sid="your_account_sid",
    api_key_sid="your_api_key_sid",
    api_key_secret="your_api_key_secret",
    resources=["messages", "calls"],
)

load_info = pipeline.run(source)
print(load_info)

See each source's README for detailed usage and authentication options.

Features

All sources share these common features:

• Incremental loading where applicable
• Automatic token/auth refresh
• Rate limit handling with exponential backoff
• Graceful permission handling (skips inaccessible resources)
• Works with any dlt destination

Development

# Fork the repository first, then:
git clone https://github.com/YOUR_USERNAME/dlt-community-sources.git
cd dlt-community-sources
uv sync --group dev
uv run pytest -v
uv run ruff check .

AI-assisted development

This repository is set up for AI coding assistants with two layers of AI context:

• Project rules (.ai/rules.md) — coding conventions, testing patterns, and structure specific to this repo
• dltHub AI workbench — dlt ecosystem knowledge, MCP server, and guided skills for pipeline development

Both are synced to tool-specific locations (Claude Code, Cursor, Codex, etc.) via a single command:

bash scripts/sync-ai-rules.sh

The dltHub workbench provides an MCP server for source search (9,700+ APIs), pipeline inspection, and data exploration — plus skills that guide you through the full pipeline development workflow. See CONTRIBUTING.md for details.

Edit .ai/ only for project rules, then re-run the sync script. CI checks that files are in sync.

Contributing

See CONTRIBUTING.md.

License

MIT