pydozer 0.2.1

Connect any data source, combine them in real-time and instantly get low-latency gRPC and REST APIs.
⚡ All with just a simple configuration! ⚡️

Overview

This repository is a python wrapper over gRPC APIs that are automatically when you run Dozer.

Installation

poetry add git+ssh://git@github.com:getdozer/dozer-python.git#main

# or 
pip install pydozer

Dependencies

• Dozer
• Poetry

Querying

Intialize Users Endpoint

api_client = ApiClient("users")

api_client.query()

# Query using $limit, $order_by, $filter
api_client.query({"$limit": 1})

#Count
api_client.count()

gRPC methods

api_client.describe()

#Health
api_client.health()

Ingestion

Initialize Ingestion Client

from dozer.ingest import IngestClient
ingestion_client = IngestClient()

Ingest a data frame

df = pl.read_parquet('tests/trips_small.parquet')
ingestion_client.ingest_df('trips', df, seq_no=1)

Use Arrow Format for ingestion

ingestion_client.ingest_df_arrow('trips', df, seq_no=1)

Ingest raw records

from pydozer.ingest_pb2 import IngestRequest
from pydozer.types_pb2 import Record, Value

user = IngestRequest(
    schema_name="users",
    typ=0,
    old=None,
    new=[Value(int_value=1), Value(string_value="vivek")],
    seq_no=1
)
ingestor.ingest(user)

Ingest in Arrow format

Check out our Docs for more information.

Testing

pytest

Contributing

Please refer to Contributing for more details.