Falcon, a versatile and advanced platform, stands at the forefront of technological innovation, offering comprehensive support for Large Language Models (LLMs), audio processing, and beyond. It is expertly designed to cater to a wide array of needs, including training, inference, and meticulous monitoring, ensuring a seamless and efficient experience for users.
Project description
AIaaS Falcon Observability
Falcon Observability
The Falcon Observability project is a Streamlit-based application designed for comprehensive log analytics and performance evaluation of AI models. This tool offers insightful visualizations and metrics derived from log data, enabling users to assess model behavior, track performance indicators, and conduct cost analysis for Azure services based on token usage.
Features
Log Analytics and Visualization
· Date Range Selection: Allows users to select date ranges for log analysis.
· Time Filtering: Provides the option to filter logs based on specified start and end times.
· Filter ID Input: Enables users to input a filter ID for more granular log retrieval.
· Log Display: Renders log data in a table format using Streamlit's st.data_editor
.
· Request Flow Visualization: Generates a line chart using Plotly Express to visualize the flow of requests over time.
· Average Token Count Visualization: Displays the average token count over time in a line chart.
Performance Analysis
· Metrics Overview: Presents metrics such as total requests, average perplexity, toxicity, ARI (Automated Readability Index), Flesch-Kincaid Grade (FKG), total/request token count, and average token count.
· Performance Metrics Visualization: Offers a detailed analysis of model performance metrics (e.g., Perplexity, Toxicity, ARI, FKG) over time through line chart visualizations.
Cost Analysis (Azure)
· Token-Based Cost Calculation: Computes and displays costs for Azure services based on token usage rates.
· Price Visualization: Illustrates the calculated prices for request and response tokens in a line chart and table format.
User Interface and Interaction
· Sidebar Selection: Allows users to choose between 'Online' or 'Local' mode for log retrieval.
· Form Submission: Utilizes Streamlit forms for submitting date range, time, and filter criteria.
· Expander for Data Selection: Provides an expandable interface for users to select log data based on specified criteria.
· Performance Analysis Trigger: Allows users to trigger the analysis of model performance metrics with a button click.
Session State Management
· State Persistence: Uses Streamlit's session state (st.session_state
) to manage and update various analytics results, loading states, and dataframes.
Installation
Clone the repository and install the required dependencies:
pip install falcon_observability
Usage
Starting the Service
from falcon_observability.main import FalconObservability
server=FalconObservability(port=8555)
server.start()
Manage Service
server.status()
Stop Service
server.stop()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for falcon_observability-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b8565322ed834fa90bf0cc73560e3953cb27a63402660c14dafe57a43db611c0 |
|
MD5 | 380368644a79482a422d2566ffa7aaef |
|
BLAKE2b-256 | 65322e48eea0dce7447261b2bf71cb04d36cd659432266746625675653ff1a75 |
Hashes for falcon_observability-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3a7e9350163dc02336b4caf24acd3f749a0772daed8424b9f62e65077fb07128 |
|
MD5 | ae78549999e3aa4ba628a0f71a9f5218 |
|
BLAKE2b-256 | 47218daae990c1810d169d02d4fc3cd18e48cffcd483bf223d397c83a0b6df81 |