Skip to main content

interloper-sql

SQL for interloper: ibis_view, portable views authored as Ibis expressions, and behind the postgres extra a SQL database destination, SQLDestination, a DatabaseDestination over SQLAlchemy Core with the SQLConnection that holds the database URL.

The base package depends on Ibis only, so a package that defines ibis_views pulls in no database driver. interloper-sql[postgres] adds SQLAlchemy and psycopg[binary]; the destination and its connection exist only with it.

One backend serves every SQLAlchemy dialect whose driver is installed. PostgreSQL works with the extra; MySQL, SQLite, Redshift, SQL Server and the rest need their own driver on top.

Setup

The connection is a single SQLAlchemy URL:

Database URL Driver to install
PostgreSQL postgresql+psycopg://user:password@host:5432/db interloper-sql[postgres]
MySQL / MariaDB mysql+pymysql://user:password@host:3306/db pymysql
SQL Server mssql+pyodbc://user:password@host/db?driver=ODBC+Driver+18+for+SQL+Server pyodbc (and the ODBC driver)
Redshift redshift+redshift_connector://user:password@host:5439/db sqlalchemy-redshift, redshift_connector
SQLite sqlite:///path/to/file.db none, in the standard library

Install the driver next to interloper (uv pip install pymysql, or a custom image built from the slim variant). Special characters in the password must be URL-encoded.

The URL carries the password, so it is a secret field: encrypted at rest in a deployed instance and masked in the UI. It also loads from the environment (SQL_URL), so SQLConnection() works with no arguments.

The connection's check runs SELECT 1.

Usage

import interloper as il
from interloper_sql import SQLConnection, SQLDestination

warehouse = SQLDestination(
    connection=SQLConnection(url="postgresql+psycopg://user:password@host:5432/analytics"),
    default_dataset="marketing",
)

In a deployed instance you configure this through the UI instead: add a SQL database connection, then a SQL database destination using it.

Datasets are schemas

An asset's dataset is the SQL schema its table lives in. Without one, the destination's default_dataset applies; without either, the table goes in the connection's default schema (public on PostgreSQL, the database itself on MySQL). A named schema is created on first write when the database supports schemas and it does not exist yet.

Tables

A table is created on first write from the asset's effective schema (declared, or inferred during conform): integers are BIGINT, floats DOUBLE, decimals NUMERIC(38, 9), datetimes timezone-aware timestamps, strings TEXT, and nested or repeated fields JSON. An existing table is never altered: columns it lacks are dropped from the write with a warning, and NaN or infinite floats are written as NULL.

Partitions

Writes replace, inside one transaction:

  • unpartitioned: every row is deleted, then the data inserted;
  • a time partition: the rows inside its half-open bounds (column >= start AND column < end) are deleted, then the data inserted;
  • any other partition: the rows equal to its id are deleted;
  • a window: each partition's rows are deleted, then the whole batch is inserted once.

A failed insert rolls the delete back, so a partition is never left empty. Inserts go in batches of 1000 rows. Reads return rows as list[dict]; reading a table that was never written raises DataNotFoundError.

Metadata

Release files for interloper-sql 0.118.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for interloper-sql 0.118.0
File Size Uploaded Provenance Actions
interloper_sql-0.118.0.tar.gz 11.6 kB Without provenance Details for interloper_sql-0.118.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for interloper-sql 0.118.0
File Interpreter ABI Platform Actions
interloper_sql-0.118.0-py3-none-any.whl Python 3 none any Details for interloper_sql-0.118.0-py3-none-any.whl

Total release size: 25.9 kB

Release files / interloper_sql-0.118.0.tar.gz

Download URL interloper_sql-0.118.0.tar.gz
Size 11.6 kB
Tags Source
SHA-256 checksum
How to use checksums
1f8f59aff7f287b5b6d9bb2f917b5528776363932a37fdb1e8dc3b1fdd221d13
BLAKE2b-256 checksum
How to use checksums
625201f81afbb5452a3142e84a800636ecf4c1a5563cf34e29217e627f7a4fa1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / interloper_sql-0.118.0-py3-none-any.whl

Download URL interloper_sql-0.118.0-py3-none-any.whl
Size 14.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
79af0f33dcadce8f8e2d066090b85d55c36a685d930ce2d04cc6288d7064b900
BLAKE2b-256 checksum
How to use checksums
502d212af87a0ec4fc17f0c7f8e84ff1f80a110e4f78485e95cb7a3af8d92afe
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.6 {"installer":{"name":"uv","version":"0.13.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for interloper-sql 0.118.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.118.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page