Agent Spy Monitor
An operational monitoring library for AI agent applications that tracks token usage, costs, performance metrics, and environmental impact.
Features
-
Token Counting
- Total tokens
- Input tokens
- Output tokens
-
Cost Calculation
- Based on the token usage and model pricing
-
Performance Metrics
- Time taken for each call
- CPU and memory consumption
-
Environmental Impact
- Estimated CO₂ emissions
-
Logging
- Comprehensive logs of all operations
-
Visualization
- CLI summaries
- Streamlit dashboard for detailed insights
Installation
You can install Agent Spy Monitor via PyPI using pip:
pip install agent-spy-monitor
Quick Start
from agent_spy import AgentSpy
# Initialize the monitor
spy = AgentSpy(model="gpt-4o")
# Start monitoring
spy.start()
# Your AI agent code here
input_text = "Hello, how are you?"
output_text = "I'm doing well, thank you!"
# Set token counts
spy.set_token_counts(input_text, output_text)
# End monitoring
spy.end()
# View results
print(f"Total cost: ${spy.cost:.6f}")
print(f"Total tokens: {spy.total_tokens}")
print(f"Carbon emissions: {spy.carbon_emissions:.6f} kg CO2")
Advanced Usage
Using with CrewAI
from agent_spy import AgentSpy
spy = AgentSpy(model="gpt-4o")
spy.start()
# Run your CrewAI workflow
crew_output = your_crew.kickoff()
# Extract token usage from CrewAI output
spy.set_token_usage_from_crew_output(crew_output)
spy.end()
Extended Monitoring with Visualization
from agent_spy import AgentSpyExtended
spy = AgentSpyExtended(model="gpt-4o")
spy.start()
# Your AI operations here
spy.end()
# Show CLI summary
spy.visualize(method='cli')
# Or launch Streamlit dashboard
spy.visualize(method='streamlit')
Supported Models
The library supports cost calculation for various AI models:
- OpenAI: GPT-4o, GPT-4o-mini, GPT-4, GPT-3.5-turbo, and more
- Anthropic: Claude-3 variants, Claude-2, Claude Instant
- Google: Gemini Pro, Gemini Flash
Configuration Options
# Disable resource monitoring for better performance
spy = AgentSpy(model="gpt-4o", enable_monitoring=False)
# Custom encoding for token counting
spy = AgentSpy(model="custom-model", encoding_name="cl100k_base")
API Reference
AgentSpy Class
Methods:
start(): Begin monitoringend(): Stop monitoring and calculate metricsset_token_counts(input_text, output_text): Manually set token countsset_token_usage_from_crew_output(crew_output): Extract from CrewAI outputcount_tokens(text): Count tokens in text
Properties:
cost: Total cost in USDtotal_tokens: Total token countinput_tokens: Input token countoutput_tokens: Output token counttotal_time: Execution time in secondscarbon_emissions: Estimated CO₂ emissions in kg
AgentSpyExtended Class
Extends AgentSpy with visualization capabilities:
Additional Methods:
visualize(method='cli'): Show CLI summaryvisualize(method='streamlit'): Launch Streamlit dashboard
License
MIT License
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
Author
Nidhish Wakodikar
- Email: nidhishwakodikar@gmail.com
- GitHub: @nidhishwakodikar
Changelog
v0.1.0
- Initial release
- Token counting and cost calculation
- Performance monitoring
- Carbon emissions estimation
- CLI and Streamlit visualization
Metadata
Release files for agent-spy-nidhish 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| agent_spy_nidhish-0.1.0.tar.gz | 9.4 kB | Without provenance | Details for agent_spy_nidhish-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| agent_spy_nidhish-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for agent_spy_nidhish-0.1.0-py3-none-any.whl |
Total release size: 18.7 kB
Release files / agent_spy_nidhish-0.1.0.tar.gz
| Download URL | agent_spy_nidhish-0.1.0.tar.gz |
|---|---|
| Size | 9.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8746e3dce1eaa08226a451d9caca493d239be35dc24e39461e34c08fdfa16dc4
|
|
BLAKE2b-256 checksum How to use checksums |
3e03d8d88093804d2c1ec77cde6757cd76275f771b246dbc9c43ef76335667f2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.5
|
Release files / agent_spy_nidhish-0.1.0-py3-none-any.whl
| Download URL | agent_spy_nidhish-0.1.0-py3-none-any.whl |
|---|---|
| Size | 9.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
afd533cbdb07298de8440cb7ed3813c79b09707b8b81db377e1febd7f86fabdb
|
|
BLAKE2b-256 checksum How to use checksums |
97f276e3cffd39ef7b68e26f265301255ef2306a825d6e8fd08a1efd3b97c0b9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.5
|
Security for agent-spy-nidhish 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.