dlt-community-sources

CI PyPI License: MIT Python 3.12+

Community-maintained dlt sources for various APIs. Install only what you need via extras.

Available Sources

Source Extra Description
App Store Connect app-store-connect Apple App Store Connect API
Twilio twilio Twilio SMS, Calls, Usage Records
Meta Ads meta-ads Meta (Facebook) Ads campaigns, ad sets, ads, and insights
NextDNS nextdns NextDNS logs and analytics
Microsoft Ads microsoft-ads Microsoft Advertising (Bing Ads) with certificate auth support
TikTok Ads tiktok-ads TikTok Marketing API (Business API)
Yahoo Ads Search yahoo-ads-search Yahoo Japan Ads Search API (SS) with MCC support

Installation

pip install dlt-community-sources[app-store-connect]

Or with uv:

uv add dlt-community-sources[app-store-connect]

Usage

import dlt
from dlt_community_sources.twilio import twilio_source

pipeline = dlt.pipeline(
    pipeline_name="twilio",
    destination="duckdb",
    dataset_name="twilio_data",
)

source = twilio_source(
    account_sid="your_account_sid",
    api_key_sid="your_api_key_sid",
    api_key_secret="your_api_key_secret",
    resources=["messages", "calls"],
)

load_info = pipeline.run(source)
print(load_info)

See each source's README for detailed usage and authentication options.

Features

All sources share these common features:

  • Incremental loading where applicable
  • Automatic token/auth refresh
  • Rate limit handling with exponential backoff
  • Works with any dlt destination

Error handling

Sources distinguish two kinds of resources:

  • Auxiliary resources (metadata: campaigns, ads, audiences, ...) skip expected client errors (HTTP 400/403/404) per endpoint — some APIs return 4xx for valid requests on accounts without certain features. Every skip is logged with the response body.
  • Primary resources (the fact data a source exists for: report, insights) never skip. Failures raise PrimaryResourceTerminalError (request rejected — retrying will not help; 4xx other than 408/429) or PrimaryResourceTransientError (timeout, rate limit, provider-side job failure), so a load never "succeeds" with silently missing data. Both mix in dlt's TerminalException / TransientException. Note that dlt wraps resource exceptions in PipelineStepFailed / ResourceExtractionError, so retry helpers that inspect only one level (such as dlt.pipeline.helpers.retry_load) cannot see the classification — use contains_terminal_exception, which walks the exception chain:
from tenacity import Retrying, retry_if_exception, stop_after_attempt
from dlt_community_sources._utils import contains_terminal_exception

for attempt in Retrying(
    stop=stop_after_attempt(3),
    retry=retry_if_exception(lambda e: not contains_terminal_exception(e)),
    reraise=True,
):
    with attempt:
        pipeline.run(source)

To load a subset of resources, either pass the source-specific resources argument or use dlt's native selection: source.with_resources("report").

Development

# Fork the repository first, then:
git clone https://github.com/YOUR_USERNAME/dlt-community-sources.git
cd dlt-community-sources
uv sync --group dev
uv run pytest -v
uv run ruff check .

AI-assisted development

This repository is set up for AI coding assistants with two layers of AI context:

  • Project rules (.ai/rules.md) — coding conventions, testing patterns, and structure specific to this repo
  • dltHub AI workbench — dlt ecosystem knowledge, MCP server, and guided skills for pipeline development

Both are synced to tool-specific locations (Claude Code, Cursor, Codex, etc.) via a single command:

bash scripts/sync-ai-rules.sh

The dltHub workbench provides an MCP server for source search (9,700+ APIs), pipeline inspection, and data exploration — plus skills that guide you through the full pipeline development workflow. See CONTRIBUTING.md for details.

Edit .ai/ only for project rules, then re-run the sync script. CI checks that files are in sync.

Contributing

See CONTRIBUTING.md.

License

MIT

Project links

Data verified by PyPI on Aug 7, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ota1200 from gravatar.com ota1200

Credits

Author: ota2000

GitHub Statistics

Data verified by PyPI on Aug 7, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (MIT)

Requires

Python >=3.12

Provides Extra

app-store-connect meta-ads microsoft-ads nextdns tiktok-ads twilio yahoo-ads-search

Tags

api data-loading data-pipeline dlt elt etl

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dlt_community_sources-0.7.20.tar.gz (57.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

dlt_community_sources-0.7.20-py3-none-any.whl (78.3 kB view details)

Uploaded Python 3

File details

Details for the file dlt_community_sources-0.7.20.tar.gz.

File metadata

  • Download URL: dlt_community_sources-0.7.20.tar.gz
  • Upload date:
  • Size: 57.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for dlt_community_sources-0.7.20.tar.gz
Algorithm Hash digest
SHA256 5973e3f61b2f4963813bd0a769e077b864760156e239e8515cc819f2ed894b71
MD5 0219abfdb90bfdf25f9eac009ac47b25
BLAKE2b-256 68845377ff4ac9b1daff854f5f0937a711f6e9af2b07bf391936fa683f6c2426

See more details on using hashes here.

Provenance

The following attestation bundles were made for dlt_community_sources-0.7.20.tar.gz:

Publisher: publish.yaml on ota2000/dlt-community-sources

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file dlt_community_sources-0.7.20-py3-none-any.whl.

File metadata

File hashes

Hashes for dlt_community_sources-0.7.20-py3-none-any.whl
Algorithm Hash digest
SHA256 f449c5bc044d371bef1d5c63215f00b86cb3f0be1a9d84ef9737e60d3494f23a
MD5 ce0564b12883cb14c513a85471b83a6c
BLAKE2b-256 e144e1a60852b6e1b2b5a49f2d71b4a6ad8dfc05b1cf673b1c9f92b339a8b2f6

See more details on using hashes here.

Provenance

The following attestation bundles were made for dlt_community_sources-0.7.20-py3-none-any.whl:

Publisher: publish.yaml on ota2000/dlt-community-sources

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.8.0

2 files

0.7.21

2 files

This release

0.7.20 This release

2 files

0.7.19

2 files

0.7.18

2 files

0.7.17

2 files

0.7.16

2 files

0.7.15

2 files

0.7.14

2 files

0.7.13

2 files

0.7.12

2 files

0.7.11

2 files

0.7.10

2 files

0.7.9

2 files

0.7.8

2 files

0.7.7

2 files

0.7.6

2 files

0.7.5

2 files

0.7.4

2 files

0.7.3

2 files

0.7.2

2 files

0.7.1

2 files

0.7.0

2 files

0.6.4

2 files

0.6.3

2 files

0.6.2

2 files

0.6.1

2 files

0.6.0

2 files

0.5.2

2 files

0.5.1

2 files

0.5.0

2 files

0.4.1

2 files

0.4.0

2 files

0.3.0

2 files

0.2.2

2 files

0.2.1

2 files

0.1.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page