zipline-refresh 0.1.dev6707

High-performance Pythonic backtesting engine with Apache Parquet storage

Zipline Refresh

A high-performance Pythonic backtesting engine for algorithmic trading strategies

---

Zipline is a Pythonic event-driven system for backtesting, originally developed by Quantopian. This Refresh fork modernizes the storage layer, eliminates legacy dependencies, and delivers significant performance improvements.

Documentation  ·  Website  ·  Report Bug

---

What's New in Refresh

Storage: bcolz → Apache Parquet

The legacy bcolz storage layer has been fully replaced with Apache Parquet via PyArrow:

	bcolz (legacy)	Parquet (new)
Format	Custom binary + Cython	Standard columnar, zstd compressed
Daily bars	One ctable per field	Single .parquet file per bundle
Minute bars	Fixed-stride padding for early closes	Actual trading minutes only
Dependencies	bcolz (unmaintained, build issues)	pyarrow (actively maintained)
Data types	uint32 (lossy for prices)	float64 (full precision)

Performance Optimizations

Profiling-driven optimizations on the backtest hot path (50 assets, 780 bars/session):

Optimization	Speedup	Detail
Lazy per-field loading	3.2x single field	Load only requested OHLCV fields instead of all 5
NumPy int64 searchsorted	40x per lookup	Replace DatetimeIndex.get_loc() with np.searchsorted on int64 arrays
Vectorized last-traded	17x	np.flatnonzero instead of Python loop for get_last_traded_dt
Batch resample aggregation	5x lifetimes	Vectorized _lifetimes_map and batch load_raw_arrays in DailyHistoryAggregator

Net result: pandas DatetimeIndex overhead reduced from 46% to 6.5% of hot-path time. Overall backtest data layer is ~2x faster.

Benchmark: Time breakdown (50 assets x 780 bars)

Before optimization:
  pandas DatetimeIndex             46.0%  ██████████████████████████████████████████████
  get_value (reader)               13.0%  █████████████
  memoize/lazyval                  10.0%  ██████████

After optimization:
  pandas DatetimeIndex              6.5%  ██████
  get_value (reader)               26.7%  ██████████████████████████
  memoize/lazyval                  12.9%  ████████████
  numpy operations                 12.2%  ████████████

Total hot-path time: 0.44s → 0.24s (1.8x faster)
Per-bar latency: 0.6ms → 0.3ms

---

Features

• Event-Driven Architecture — Realistic simulation with proper order lifecycle, slippage, and commission models
• Pipeline API — Factor-based screening with 20+ built-in technical factors (RSI, MACD, Bollinger, Ichimoku, etc.) and easy CustomFactor extensibility
• Factor Composition — rank(), zscore(), demean(), winsorize(), top(N) with groupby for sector-neutral strategies
• PyData Integration — pandas DataFrames in/out, compatible with matplotlib, scipy, statsmodels, scikit-learn
• Multi-Country Support — 42 country domains with proper trading calendars via exchange_calendars
• Minute & Daily Resolution — Full minute-level backtesting with proper market open/close handling

Installation

Zipline supports Python >= 3.10 and is compatible with current versions of NumFOCUS libraries.

Using pip

pip install zipline-refresh

From source

git clone https://github.com/teleclaws/zipline-refresh.git
cd zipline-refresh
pip install -e .

See the documentation for detailed instructions.

Quickstart

The following implements a simple dual moving average crossover strategy:

from zipline.api import order_target, record, symbol


def initialize(context):
    context.i = 0
    context.asset = symbol('AAPL')


def handle_data(context, data):
    context.i += 1
    if context.i < 300:
        return

    short_mavg = data.history(context.asset, 'price', bar_count=100, frequency="1d").mean()
    long_mavg = data.history(context.asset, 'price', bar_count=300, frequency="1d").mean()

    if short_mavg > long_mavg:
        order_target(context.asset, 100)
    elif short_mavg < long_mavg:
        order_target(context.asset, 0)

    record(AAPL=data.current(context.asset, 'price'),
           short_mavg=short_mavg,
           long_mavg=long_mavg)

Run it

# Ingest data from NASDAQ (requires free API key from https://data.nasdaq.com)
export QUANDL_API_KEY="your_key_here"
zipline ingest -b quandl

# Run backtest
zipline run -f dual_moving_average.py --start 2014-1-1 --end 2018-1-1 -o dma.pickle --no-benchmark

More examples in the zipline/examples directory.

Compatibility Notes

Release 3.05 — Compatible with NumPy 2.0 (requires pandas >= 2.2.2)

Release 3.0 — Updated to pandas >= 2.0 and SQLAlchemy > 2.0

Release 2.4 — Updated to exchange_calendars >= 4.2

Contributing

This project is sponsored by Kavout.

Found a bug or have a suggestion? Open an issue.

License

Apache 2.0. See LICENSE for details.