tiders 0.1.1

Library for building blockchain pipelines

Tiders is an open-source framework that simplifies getting data out of blockchains and into your favorite tools. Whether you are building a DeFi dashboard, tracking NFT transfers, or running complex analytics, Tiders handles the heavy lifting of fetching, cleaning, transforming and storing blockchain data.

Tiders is modular. A Tiders pipeline is built from four components:

Component	Description
Provider	Data source (HyperSync, SQD, or RPC)
Query	What data to fetch (block range, transaction, logs, filters, field selection)
Steps	Transformations to apply (decode, cast, encode, custom)
Writer	Output destination

Why Tiders?

Most indexers lock you into a specific platform or database. Tiders is built to be modular, meaning you can swap parts in and out without breaking your setup:

• Swap Providers: Don't like your current data source? Switch between HyperSync, SQD, or a standard RPC node by changing one line of code.
• Plug-and-Play data transformations: Need to decode smart contract events or change data types? Use our built-in Rust-powered steps or write your own custom logic.
• Write Anywhere: Send your data to a local DuckDB file for prototyping, or a production-grade ClickHouse or PostgreSQL instance when you're ready to scale.
• Modular Reusable Pipelines: Protocols often reuse the same data structures. You don't need write modules from scratch every time. Since Tiders pipelines are regular Python objects, you can build functions around them, reuse across pipelines, or set input parameters to customize as needed.

Getting Started

See getting started section of the docs.

Two ways to use tiders

Mode	How	When to use
Python SDK	Write a Python script, import tiders	Full control, custom logic, complex pipelines
CLI (No-Code)	Write a YAML config, run tiders start	Quick setup, no Python required, standard pipelines

Both modes share the same pipeline engine.

You can also use tiders codegen to generate a Python script from a YAML config — a quick way to move from no-code to full Python control.

Installation

For both CLI and SDK

pip install tiders

Features

• Continuous Ingestion: Keep your datasets live and fresh. Tiders can poll the chain head to ensure your data is always up to date.
• Switch Providers: Move between HyperSync, SQD, or standard RPC nodes with a single config change.
• No Vendor Lock-in: Use the best data providers in the industry without being tied to their specific platforms or database formats.
• Custom Logic: Easily extend and customize your pipeline code in Python for complete flexibility.
• Advanced Analytics: Seamlessly works with industry-standard tools like Polars, Pandas, Data fusion and PyArrow as the data is fetched.
• Multiple Outputs: Send the same data to a local file and a production database simultaneously.
• Rust-Powered Speed: Core tasks like decoding and transforming data are handled in Rust, giving you massive performance without needing to learn a low-level language.
• Parallel Execution: Tiders doesn't wait around. While it's writing the last batch of data to your database, it’s already fetching and processing the next one in the background.

Data Providers

Connect to the best data sources in the industry without vendor lock-in. Tiders decouples the provider from the destination, giving you a consistent way to fetch data.

Provider	Ethereum (EVM)	Solana (SVM)
HyperSync	✅	❌
SQD	✅	✅
RPC	✅	❌

Transformations

Leverage the tools you already know. Tiders automatically convert data batch-by-batch into your engine's native format, allowing for seamless, custom transformations on every incoming increment immediately before it is written.

Engine	Data format in your function	Best for
Polars	Dict[str, pl.DataFrame]	Fast columnar operations, expressive API
Pandas	Dict[str, pd.DataFrame]	Familiar API, complex row-level operations
DataFusion	Dict[str, datafusion.DataFrame]	SQL-based transformations, lazy evaluation
PyArrow	Dict[str, pa.Table]	Zero-copy, direct Arrow manipulation

Supported Output Formats

Whether local or a production-grade data lake, Tiders handles the schema mapping and batch-loading to your destination of choice.

Writer	Format
DuckDB	DuckDB database
ClickHouse	ClickHouse
Iceberg	Apache Iceberg
Delta Lake	Delta Lake
PyArrow Dataset	Parquet files
PostgreSQL	PostgreSQL
CSV	CSV files

Usage examples

• Examples

Logging

Python code uses the standard logging module of python, so it can be configured according to python docs.

Set RUST_LOG environment variable according to env_logger docs in order to see logs from rust modules.

To run an example with trace level logging for rust modules:

RUST_LOG=trace uv run examples/path/to/my/example

Development

Tiders is composed of some repositories. 3 owned ones.

This repo uses uv for development. Clone all three projects side by side:

git clone https://github.com/yulesa/tiders.git
git clone https://github.com/yulesa/tiders-core.git
git clone https://github.com/yulesa/tiders-rpc-client.git

Local development with tiders-core

Configure tiders to use your local tiders-core Python package by adding to pyproject.toml:

[tool.uv.sources]
tiders-core = { path = "../tiders-core/python", editable = true }

Then sync the environment:

cd tiders
uv sync

For full instructions including building tiders-core and tiders-rpc-client from source, see the Development Setup docs.

Core libraries we use for ingesting/decoding/validating/transforming blockchain data are implemented in tiders-core repo.

Acknowledgements

Tiders is a fork of Cherry and cherry-core, a blockchain data pipeline framework built by the SteelCake team. Cherry laid the architectural foundation that Tiders builds upon, and we're grateful for their work and the open-source spirit that made this continuation possible.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.