ampy-proto 2.0.3

Canonical schemas for AmpyFin (generated Python protobufs)

ampy-proto

Canonical Protocol Buffer schemas for the AmpyFin trading system

---

📋 Table of Contents

• 🎯 What We're Solving
• ⚡ Quick Start
• 📊 Available Schemas
• 🏗️ Key Design Principles
• 🛠️ Development
• 📈 Versioning
• 🤝 Contributing
• 📄 License
• 💬 Support

🎯 What We're Solving

AmpyFin is building a self-learning, modular trading system that needs to handle data from multiple sources (DataBento, Tiingo, yfinance, etc.) across different languages (Go, Python, C++). This project provides:

• 🎯 Unified schemas for all financial data (bars, ticks, fundamentals, news, etc.)
• 🌐 Cross-language compatibility with generated code for Go, Python, and C++
• 🎯 Precision guarantees using scaled decimal types instead of floating-point
• ⏰ Time discipline with UTC timestamps and clear event/ingest time separation
• 🔄 Versioned contracts that evolve safely without breaking consumers

⚡ Quick Start

🐍 Python

# Install from PyPI (when published)
pip install ampy-proto

# Or install locally
pip install -e .

from ampy.bars.v1 import bars_pb2
from ampy.common.v1 import common_pb2

# Create a bar
bar = bars_pb2.Bar()
bar.security.symbol = "AAPL"
bar.security.mic = "XNAS"
bar.open.scaled = 1923450
bar.open.scale = 4
bar.high.scaled = 1925600
bar.high.scale = 4
bar.low.scaled = 1922200
bar.low.scale = 4
bar.close.scaled = 1924100
bar.close.scale = 4
bar.volume = 184230

print(f"AAPL bar: ${bar.close.scaled / (10 ** bar.close.scale):.2f}")

🐹 Go

# Install all ampy-proto packages at once
go get github.com/AmpyFin/ampy-proto@v2.0.3

# Or install specific packages only
go get github.com/AmpyFin/ampy-proto/gen/go/ampy/bars/v1@v2.0.3

package main

import (
    "fmt"
    barspb "github.com/AmpyFin/ampy-proto/gen/go/ampy/bars/v1"
)

func main() {
    bar := &barspb.Bar{}
    bar.Security = &barspb.Security{
        Symbol: "AAPL",
        Mic:    "XNAS",
    }
    bar.Open = &barspb.Decimal{
        Scaled: 1923450,
        Scale:  4,
    }
    bar.High = &barspb.Decimal{
        Scaled: 1925600,
        Scale:  4,
    }
    bar.Low = &barspb.Decimal{
        Scaled: 1922200,
        Scale:  4,
    }
    bar.Close = &barspb.Decimal{
        Scaled: 1924100,
        Scale:  4,
    }
    bar.Volume = 184230

    fmt.Printf("AAPL bar: $%.2f\n", float64(bar.Close.Scaled)/float64(1e4))
}

⚡ C++

# Build the library
cd gen/cpp
mkdir build && cd build
cmake .. && make

#include <iostream>
#include "ampy/bars/v1/bars.pb.h"

int main() {
    ampy::bars::v1::Bar bar;
    bar.mutable_security()->set_symbol("AAPL");
    bar.mutable_security()->set_mic("XNAS");
    bar.mutable_open()->set_scaled(1923450);
    bar.mutable_open()->set_scale(4);
    bar.mutable_high()->set_scaled(1925600);
    bar.mutable_high()->set_scale(4);
    bar.mutable_low()->set_scaled(1922200);
    bar.mutable_low()->set_scale(4);
    bar.mutable_close()->set_scaled(1924100);
    bar.mutable_close()->set_scale(4);
    bar.set_volume(184230);

    std::cout << "AAPL bar: $" 
              << (bar.close().scaled() / std::pow(10, bar.close().scale()))
              << std::endl;
    return 0;
}

📊 Available Schemas

🏷️ Domain	📝 Purpose	🔑 Key Messages
📈 bars	OHLCV price bars	Bar, BarBatch
📊 ticks	Trade and quote data	Tick, TickBatch
📋 fundamentals	Financial statements	Fundamental, FundamentalBatch
📰 news	Market news and sentiment	NewsItem
💱 fx	Foreign exchange rates	FxRate
🏢 corporate_actions	Splits, dividends	CorporateAction
🌍 universe	Tradable securities lists	Universe
🎯 signals	Model outputs and signals	Signal
📋 orders	Order management	Order, OrderRequest
✅ fills	Trade executions	Fill
💼 positions	Portfolio positions	Position
📊 metrics	Operational metrics	Metric
🔧 common	Shared types	Decimal, Money, Security, Meta

🏗️ Key Design Principles

1. 🎯 Precision with Scaled Decimals

Instead of floating-point numbers, we use scaled decimals:

message Decimal {
  int64 scaled = 1;  // The actual value
  int32 scale = 2;   // Decimal places (e.g., 4 = 4 decimal places)
}

Example: scaled: 1923450, scale: 4 represents 192.3450

2. ⏰ Time Discipline

All timestamps are UTC with clear semantics:

• 🕐 event_time: When the market event actually happened
• 📥 ingest_time: When our system received it
• 🎯 as_of: Logical timestamp for downstream processing

3. 🏷️ Security Identification

Use proper security identifiers, not just tickers:

message Security {
  string symbol = 1;    // e.g., "AAPL"
  string mic = 2;       // Market identifier code, e.g., "XNAS"
  string figi = 3;      // Financial Instrument Global Identifier (optional)
  string isin = 4;      // International Securities Identification Number (optional)
}

4. 🔍 Metadata for Traceability

Every message includes metadata for lineage:

message Meta {
  string run_id = 1;           // Unique run identifier
  string source = 2;           // Data source (e.g., "yfinance-go")
  string producer = 3;         // Producer instance ID
  string schema_version = 4;   // Schema version used
  string checksum = 5;         // Optional message checksum
}

🛠️ Development

📋 Prerequisites

• 🔧 Buf for protobuf management
• 🐹 Go 1.23+ for Go code generation
• 🐍 Python 3.9+ for Python code generation
• ⚡ CMake and C++17 compiler for C++ code generation

🔨 Building

# Generate code for all languages
buf generate proto

# Lint protobuf files
buf lint proto

# Check for breaking changes
buf breaking proto --against '.git#branch=main'

# Build Python package
python -m build

# Build C++ library
cd gen/cpp && mkdir build && cd build && cmake .. && make

🧪 Testing

# Run roundtrip tests
python tests/roundtrip/python_roundtrip.py

# Test Go imports
go run examples/go/smoke/main.go

# Test C++ compilation
cd examples/cpp && g++ -I../../gen/cpp smoke.cpp -L../../gen/cpp/build -lampy_proto -lprotobuf -o smoke && ./smoke

📈 Versioning

This project follows semantic versioning:

• 🔴 Major versions (v2, v3): Breaking changes requiring migration
• 🟡 Minor versions (v1.1, v1.2): Additive changes, backward compatible
• 🟢 Patch versions (v1.0.1, v1.0.2, v1.0.3): Bug fixes, backward compatible

Current version: v2.0.3

🤝 Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature/your-feature
3. Make your changes and add tests
4. Run buf lint and buf breaking to ensure compatibility
5. Submit a pull request

📋 Schema Evolution Rules

• ✅ Add new optional fields with default values
• ✅ Add new enum values (append only)
• ✅ Add new messages or services
• ❌ Never change field numbers of existing fields
• ❌ Never change field types of existing fields
• ❌ Never remove fields (mark as deprecated instead)
• ❌ Never renumber enum values

📄 License

MIT License - see LICENSE file for details.

💬 Support

• 🐛 Issues: GitHub Issues
• 💭 Discussions: GitHub Discussions
• 📚 Documentation: Project Wiki

---

---

This project is part of the AmpyFin ecosystem - a self-learning, modular trading system.