cognitor 0.1.0

Python SDK to extract relevant metrics from Small Language Model inference calls.

cognitor-py

cognitor-py is the Python SDK of our Cognitor platform. It is used to get detailed tracing and observability for Small Language Models applications. All metrics can be saved to a self-hosted instance of the Cognitor platform or to a local file.

At this time, cognitor-py supports HuggingFace transformers models.

Installation

pip install cognitor

Usage

Send logs to a self-hosted instance of Cognitor platform

from cognitor import Cognitor

# Initialize your model and tokenizer
model_name = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
pipe = pipeline("text-generation", model=model_name, tokenizer=tokenizer)

cognitor = Cognitor(
    model_name=model_name,
    tokenizer=tokenizer,
    log_type="database",
    host="localhost",
    port=5432,
    user="postgres",
    password="postgres",
    dbname="cognitor"
)

# Run inference within the monitor context
with cognitor.monitor() as m:
    input_text = "Once upon a time,"
    # Use track() to capture only the inference duration
    with m.track():
        output = pipe(input_text, max_length=50)
    m.capture(input_data=input_text, output=output)

Save logs to a local file

from cognitor import Cognitor

# Initialize your model and tokenizer
model_name = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
pipe = pipeline("text-generation", model=model_name, tokenizer=tokenizer)

cognitor = Cognitor(
    model_name=model_name,
    tokenizer=tokenizer,
    log_type="file",
)

# Run inference within the monitor context
with cognitor.monitor() as m:
    input_text = "Once upon a time,"
    # Use track() to capture only the inference duration
    with m.track():
        output = pipe(input_text, max_length=50)
    m.capture(input_data=input_text, output=output)