dagster-xquik

dagster-xquik provides a Dagster resource for read-only X/Twitter data workflows through the Xquik REST API.

Use it when Dagster assets or jobs need to:

• Search public tweets
• Fetch a tweet by ID
• Search users
• Fetch a user profile by ID or handle
• Fetch user timeline tweets
• Fetch regional trends

Installation

dagster-xquik is pending its first release through the Dagster community integrations release flow. Until that release is published, install it from a local checkout:

cd libraries/dagster-xquik
pip install -e .

After the first release is published, install the package from PyPI:

pip install dagster-xquik

Usage

Set XQUIK_API_KEY in the environment, then pass it through Dagster's EnvVar.

from dagster import AssetExecutionContext, EnvVar, asset, Definitions
from dagster_xquik import XquikResource


@asset
def recent_xquik_tweets(
    context: AssetExecutionContext,
    xquik: XquikResource,
) -> list[dict[str, object]]:
    response = xquik.get_client().search_tweets(q="xquik", limit=10)
    tweets = response.get("data", [])
    context.log.info("Fetched %s tweets", len(tweets))
    return tweets


defs = Definitions(
    assets=[recent_xquik_tweets],
    resources={
        "xquik": XquikResource(api_key=EnvVar("XQUIK_API_KEY")),
    },
)

Resource

from dagster import EnvVar
from dagster_xquik import XquikResource

xquik = XquikResource(api_key=EnvVar("XQUIK_API_KEY"))
client = xquik.get_client()

client.search_tweets(q="dagster", limit=25)
client.get_tweet("1912345678901234567")
client.search_users(q="dagster")
client.get_user("@dagster")
client.get_user_tweets("@dagster", limit=20)
client.get_trends(woeid=1, count=10)

XquikResource accepts:

Local Development

make install
make test
make ruff
make check
make build