StreamWright
Declarative data pipelines in YAML: read from APIs, files, databases and ads platforms, shape with DuckDB SQL, write to files, DuckDB, DuckLake, Singer or dlt.
- No per-connector code: authentication, paging, retries, rate limits and incremental state are configuration.
- Connectors on demand from the StreamWright hub:
sw connectors -i <name>. - Jobs:
kind: DuckDBJobSQL pipelines with replay-safe writes;kind: SparkJobwith streamwright-spark.
Install
pip install streamwright
sw connectors -i restapi
Example
A source is a folder: source.yaml and one file per stream.
# my_api/source.yaml
kind: source
name: my_api
spec:
secrets:
api_token: {type: string}
auth:
provider: restapi
type: bearer
token: "{{ secrets.api_token }}"
http:
base_url: https://api.example.com
# my_api/streams/items.yaml
requests:
- name: raw_items
http: {path: /v1/items}
transform:
mode: page
steps:
- name: items
select: SELECT record->>'id' AS id, record->>'name' AS name FROM raw_items
export:
items: {step: items, primary_key: [id]}
sw validate my_api
STREAMWRIGHT_SECRET_API_TOKEN=... sw run my_api --output jsonl:out
Links
Metadata
Release files for streamwright 0.1.11
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| streamwright-0.1.11.tar.gz | 443.4 kB | Without provenance | Details for streamwright-0.1.11.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| streamwright-0.1.11-py3-none-any.whl | Python 3 | none | any | Details for streamwright-0.1.11-py3-none-any.whl |
Total release size: 892.5 kB
Release files / streamwright-0.1.11.tar.gz
| Download URL | streamwright-0.1.11.tar.gz |
|---|---|
| Size | 443.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0ce8eb7e709a431263e12d3d138675e015a05b6b098d0ef98bfe69d1444475a3
|
|
BLAKE2b-256 checksum How to use checksums |
f906fdcf4de2d1f6938b541b93c987b3371659cea7956f5223cabe1e6634017a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.7
|
Release files / streamwright-0.1.11-py3-none-any.whl
| Download URL | streamwright-0.1.11-py3-none-any.whl |
|---|---|
| Size | 449.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f3b0e3d97884e8ea25b31801fb0422da76b972a2bc2511a78fa43ec69e361e34
|
|
BLAKE2b-256 checksum How to use checksums |
b701debc06ce8f17ea94e3f39325a43a8ed2075ecd769a2befb736bab00b0a58
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.7
|
Security for streamwright 0.1.11
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.