Blockweaver

Blockweaver downloads and verifies immutable, feature-selected EVM block datasets. Chains and JSON-RPC providers are configuration, not code. Each successful request publishes one data file and one canonical manifest.

Python 3.11 or newer is required. Dataset writing is supported on Linux and macOS.

uv tool install blockweaver
blockweaver init

init writes a complete executable example with environment-backed RPC URLs and a local chain. Configuration selection is explicit --config, then BLOCKWEAVER_CONFIG, then the platform user config path. The command never overwrites a file.

Edit that generated file for user-specific profiles. For example:

[chains.ethereum]
chain_id = 1
finality_tag = "finalized"
provider = "primary"
verifier = "verifier"

Configuration is strict: unknown keys, unknown profiles, invalid URLs, ambiguous url/url_env pairs, and non-finite or greater-than-one-hour timeouts fail before network access. Chain profiles may override provider and verifier. CLI values override the selected chain and provider profiles, which override global defaults.

Inspect configuration and the closed feature catalog without exposing endpoints. chains reports available_sources; features reports top-level available_sources and per-feature supported_sources.

blockweaver chains
blockweaver features --chain local

Download

Bounds are inclusive. Supply exactly one complete range form:

blockweaver download --from-block 19000000 --to-block 19000999

blockweaver download \
  --from-time 2026-01-01T10:30+01:00 \
  --to-time 2026-01-01T10:45:30+01:00 \
  --feature timestamp \
  --feature block_hash \
  --feature effective_priority_fee_per_gas_p50 \
  --format csv

Dates mean the full UTC day. Datetimes accept timezone-aware hour, minute, or second precision; reduced-precision end bounds include the final second of that period. Blockweaver resolves time bounds against the finalized chain and rejects pre-genesis, empty, future, and partly unfinalized requests instead of clipping them.

block_number is always the first column. Other columns are selected explicitly or inherited from defaults.features. Header features share eth_getBlockByNumber; selected priority-fee percentiles share one eth_feeHistory request per acquisition chunk with only the requested percentiles.

An omitted --id mints a UUID4 and emits it on stderr before acquisition. Reusing an explicit --id resumes only an exact binding of chain, requested and resolved range, features, format, source, and provider profile names. Batch size, concurrency, timeout, and endpoint credentials may change between attempts.

The output is exactly:

ROOT/<uuid4>/
  manifest.json
  blocks.parquet | blocks.csv

The directory UUID must equal the manifest's dataset UUID. Parquet is the typed default. CSV uses canonical decimal integers and UTF-8 strings; manifest.json is its type authority. The unversioned manifest records the tool version, request, resolved range, ordered schema and units, acquisition plan, chain identity, non-secret provider profile names, finality proof, verification samples, byte size, and SHA-256 digest. JSON is sorted, compact, UTF-8, and newline-terminated.

Work is checkpointed under a hidden directory. Complete chunks are digest-bound and validated before reuse. The fully assembled candidate is validated, synced, and atomically renamed without replacement; existing destinations are never overwritten.

Python consumers use the same strict local validator as verify:

from blockweaver import open_dataset

dataset = open_dataset("downloads/11111111-1111-4111-8111-111111111111")
print(dataset.chain_id, dataset.first_block, dataset.last_block, dataset.schema, dataset.data_path)

Verify

Local verification is strict and needs no provider:

blockweaver verify ./downloads/11111111-1111-4111-8111-111111111111

RPC verification uses a configured profile or a direct URL. It checks deterministic samples and refreshes the stored finality proof. --full-rpc checks every row in bounded chunks, including ancestry across chunk boundaries.

blockweaver verify DATASET --provider verifier
blockweaver verify DATASET --rpc-url http://127.0.0.1:8545 --full-rpc

RPC-only tuning requires --provider or --rpc-url. An explicit --config requires --provider; --rpc-url can override that named profile's endpoint.

Progress and errors are JSON Lines on stderr. Errors include stable code and message fields. Success is one JSON receipt on stdout. URLs and environment values are excluded from manifests, receipts, and intentional logs.

Providers must implement EVM JSON-RPC batch requests, historical eth_getBlockByNumber, the configured finalized or safe tag, and eth_feeHistory when priority-fee features are selected. Independent verification consumes quota. Blockweaver checks provider agreement, numbered ancestry to the tagged anchor, a numbered anchor reread, and deterministic row samples; this is strong operational verification, not a trustless consensus client.

BigQuery source

Google Blockchain Analytics is an optional acquisition route for history that an RPC provider cannot serve. Install it explicitly; ordinary RPC installs do not include Google libraries:

uv tool install 'blockweaver[bigquery]'

Configure a strictly validated project.dataset identifier on the chain and one billing project or environment reference. The byte cap is mandatory. When BigQuery is the default source, configuration may omit defaults.provider and the unused primary profile; the verifier profile remains mandatory. Set chains.<name>.bigquery_dataset, then configure [bigquery] with exactly one billing project or project_env and maximum_bytes_billed.

Select it globally with defaults.source = "bigquery" or per request:

blockweaver download --source bigquery --from-block 1000000 --to-block 1000999

The same ranges, features, formats, resume state, receipts, and two-file artifacts apply to both sources. Time ranges are resolved against the configured verifier RPC before BigQuery planning. Blockweaver reads the required blocks, transactions, and receipts schemas, rejects unavailable selected features, then performs a dry run and checks its result schema and estimated bytes. Only then does it execute the fixed whitelisted query with maximum_bytes_billed enforced again by BigQuery. Results stream through bounded pages into the normal checkpoints.

BigQuery rows are not trusted as chain truth. The verifier RPC checks chain ID, resolved edges, target hash, deterministic row samples, numbered ancestry, and a reread finalized or safe anchor before publication. The manifest records the dataset identifier and verifier profile, never the billing project, environment value, credentials, or SQL text. The configured dataset must expose the recognized common Google Blockchain Analytics schema; arbitrary SQL and field mappings are not supported.

For development:

uv sync --locked --dev
uv run pytest
uv run --extra bigquery python -c 'from google.cloud import bigquery'
uv run ruff check src tests
uv run ruff format --check src tests
uv run pyright
uv run vulture src tests --min-confidence 80

Licensed under the MIT License.