irys-core 0.5.0

Core backend & SxT client utilities for Irys Cloud

Irys Core Backend

Folder Structure Overview

irys-core-backend/
│
├── .env                        # Environment variables for SxT, Google, etc.
├── googleSecretManager.json    # Google Cloud service account credentials
├── main.py                     # FastAPI entrypoint, includes SxT router
├── pyproject.toml              # Project metadata and dependencies
├── README.md                   # Project documentation
│
├── src/
│   └── irys_core/
│       ├── __init__.py
│       ├── config.py           # Loads settings from .env using pydantic
│       ├── errors.py           # Custom exception classes for error taxonomy
│       ├── requirement.txt     # Python dependencies
│       ├── utils.py            # Utility functions
│       ├── caching.py          # Caching logic (e.g., Redis)
│       ├── idempotency.py      # Idempotency helpers
│       ├── logging.py          # Logging setup
│       │
│       ├── auth/
│       │   ├── auth0.py        # Auth0 integration
│       │   ├── base.py         # Base authentication logic
│       │   └── sxt_token_refresher.py # SxT token refresh logic
│       │
│       ├── redis/
│       │   └── client.py       # Redis client setup
│       │
│       └── sxt/
│           ├── __init__.py
│           ├── biscuit_gsm.py  # Fetches biscuits from Google Secret Manager
│           ├── sxt_auth.py     # Handles SxT authentication, token retrieval (with retry/timeout)
│           ├── sxt_client.py   # Main SxT SQL query wrapper (retry, timeout, biscuit support)
│           ├── sxt_models.py   # Pydantic models for SxT requests/responses
│           ├── sxt_routes.py   # FastAPI routes for SxT endpoints
│           └── __pycache__/    # Python bytecode cache
│
├── tests/
│   ├── conftest.py             # Pytest fixtures
│   ├── test_auth.py            # Auth module tests
│   ├── test_redis.py           # Redis client tests
│   ├── test_sxt_client.py      # SxT client tests (add more coverage!)
│   └── data/
│       └── sample_payload.json # Example payloads for tests
│
└── .github/
    └── workflows/
        ├── ci.yml              # CI pipeline
        └── release.yml         # Release automation

Key Logic & Loops

• main.py: Starts FastAPI app, includes SxT router for SQL endpoints.
• config.py: Loads all environment variables (API keys, URLs, biscuits) for SxT and Google integration.
• errors.py: Defines custom exceptions for authentication, query, and biscuit errors.
• sxt_auth.py: Handles SxT authentication. Implements retry and timeout logic using tenacity for robust token fetching.
• sxt_client.py: Core wrapper for SxT SQL queries. Supports:
  • Biscuit tokens for tenant/schema access.
  • Retry and timeout for stability.
  • Uses environment variables for connection.
• sxt_models.py: Pydantic models for request/response validation (e.g., SQLRequest, SQLResponse).
• biscuit_gsm.py: Fetches biscuit tokens from Google Secret Manager for secure access.
• sxt_routes.py: FastAPI endpoints for SxT operations, connects HTTP requests to backend logic.
• tests/: Contains unit tests for authentication, Redis, and SxT client. (Expand coverage for error scenarios!)

Important Loops

• Retry Logic: Both authentication (sxt_auth.py) and query execution (sxt_client.py) use retry loops via tenacity to handle transient errors and timeouts.
• Biscuit Fetching: biscuit_gsm.py& biscuit_query() loops through BISCUIT_REGISTRY (spacetime table) and then Google Secret Manager to fetch the biscuits for particular category.

---