Metadata-Version: 2.4
Name: kalytera
Version: 0.1.1
Summary: Real-time quality monitoring and failure detection for production AI agents
Project-URL: Homepage, https://kalytera.ai
Project-URL: Documentation, https://docs.kalytera.ai
Project-URL: Repository, https://github.com/your-org/kalytera
Project-URL: Bug Tracker, https://github.com/your-org/kalytera/issues
Author-email: Kalytera <priya@kalytera.ai>
License: MIT
Keywords: agents,ai,evaluation,llm,monitoring,observability
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Requires-Dist: aiohttp>=3.9.0
Provides-Extra: dev
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.1.0; extra == 'dev'
Requires-Dist: requests>=2.31.0; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: server
Requires-Dist: alembic==1.13.1; extra == 'server'
Requires-Dist: anthropic>=0.40.0; extra == 'server'
Requires-Dist: fastapi==0.110.0; extra == 'server'
Requires-Dist: numpy>=1.24.0; extra == 'server'
Requires-Dist: pandas==2.0.3; extra == 'server'
Requires-Dist: plotly==5.19.0; extra == 'server'
Requires-Dist: psycopg2-binary==2.9.9; extra == 'server'
Requires-Dist: pydantic==2.5.3; extra == 'server'
Requires-Dist: python-dotenv==1.0.1; extra == 'server'
Requires-Dist: python-multipart==0.0.27; extra == 'server'
Requires-Dist: sqlalchemy==2.0.27; extra == 'server'
Requires-Dist: streamlit>=1.37.0; extra == 'server'
Requires-Dist: uvicorn[standard]==0.27.1; extra == 'server'
Description-Content-Type: text/markdown

# 🧠 Kalytera - Production-Ready AI Agent Performance Intelligence

**The complete platform for monitoring, testing, and optimizing any AI agent in production.**

---

## 🚀 **What You Get**

**Kalytera is a finished product that agents can immediately use to build and test.** No more low-level work - just integrate and get instant insights.

### ✅ **Production-Ready SDK**
- **One-line integration**: `iq.track(user_input, agent_response)`
- **Non-blocking monitoring**: Never slows down your agent
- **Auto-batching**: Efficient data transmission
- **Error-safe**: Agent keeps running even if Kalytera is down

### ✅ **Autonomous Testing Framework**
- **Comprehensive test suites**: Coding, Customer Service, Data Science, Sales
- **Automated evaluation**: LLM judges score every response
- **Performance grading**: A+ to F grades with specific recommendations
- **Continuous monitoring**: Real-time health checks

### ✅ **Enterprise Dashboard**
- **Clear agent identification**: See exactly which agents are monitored
- **Evaluation coverage**: Prominent display of sample percentages
- **Key metrics at top**: Quality scores, success rates, performance indicators
- **Actionable insights**: Specific developer recommendations with priorities

---

## 📦 **Complete Installation**

```bash
# 1. Clone Kalytera
git clone <repo>
cd Kalytera

# 2. API is already deployed at:
# https://kalytera-api-z9it.onrender.com

# 3. Dashboard is running at:
# http://localhost:8509
```

**That's it. Kalytera is ready for production use.**

---

## 🎯 **5-Minute Quick Start**

### **Step 1: Monitor Any Agent (2 lines of code)**
```python
from kalytera_sdk import Kalytera

# Initialize once
iq = Kalytera(agent_id="my-awesome-agent")

# Monitor any interaction (non-blocking)
iq.track(
    user_input="How do I fix this bug?", 
    agent_response="Here's how to fix it..."
)

# Get real-time insights
insights = iq.get_insights()
performance_score = iq.get_performance_score()  # 0.0 - 1.0
recommendations = iq.get_recommendations()
```

### **Step 2: Test Agent Performance (1 line)**
```python
from agent_testing_framework import AgentTester

# Test any agent function
def my_agent(user_input: str) -> str:
    return "Agent response here"

# Run comprehensive tests
tester = AgentTester("my-agent")
tester.register_agent(my_agent)
results = tester.run_full_test_suite()

# Get performance report
report = tester.generate_performance_report()
print(report)  # Detailed A+ to F grade with recommendations
```

### **Step 3: View Enterprise Dashboard**
**URL: http://localhost:8509**

- ✅ **Agent identification**: See which agents are being evaluated
- ✅ **Evaluation coverage**: 1.8% (3 of 171 interactions evaluated)  
- ✅ **Key metrics**: Quality scores, success rates, performance indicators
- ✅ **Actionable insights**: Specific developer recommendations

---

## 🏭 **Production Examples**

### **Coding Agent Integration**
```python
class CodingAgent:
    def __init__(self):
        self.kalytera = Kalytera(agent_id="production-coding-agent")
    
    def respond(self, user_input: str) -> str:
        response = self.generate_response(user_input)
        
        # Track with Kalytera (non-blocking)
        self.kalytera.track(user_input, response)
        
        return response
```

### **Customer Service Agent**
```python
class CustomerServiceAgent:
    def __init__(self):
        self.kalytera = Kalytera(agent_id="customer-service-agent")
    
    def handle_request(self, customer_input: str) -> str:
        response = self.generate_response(customer_input)
        
        # Automatic performance monitoring
        self.kalytera.track(customer_input, response)
        
        return response
```

### **Autonomous Testing**
```python
# Test any agent automatically
tester = AgentTester("production-agent")
tester.register_agent(my_agent_function)

# Run full test suite
results = tester.run_full_test_suite()
# Output: Pass rate: 85.2% (Grade: A)

# Continuous monitoring
tester.continuous_monitoring(interval_minutes=60)
```

---

## 📊 **What Kalytera Monitors**

### **Usage Analytics**
- Session volumes and patterns
- Intent classification across all agent types
- Workflow completion rates
- Response times and performance

### **Quality Assessment** 
- **LLM-as-a-Judge evaluation**: Autonomous scoring of every response
- Quality scores by agent type and intent
- Failure pattern detection
- Root cause analysis

### **Performance Insights**
- **Real-time recommendations**: Specific actions to improve agent performance
- A+ to F grading system
- Critical issue identification
- Developer action items with priorities

### **Loss Pattern Analysis**
- Dropout detection in agent workflows  
- High-impact failure identification
- Recommended fixes for common problems

---

## 🎯 **Agent Testing Framework**

### **Comprehensive Test Suites**
- **Coding Agents**: Debug errors, write functions, optimize code
- **Customer Service**: Handle complaints, billing issues, account recovery
- **Data Science**: Analyze data, create visualizations, generate insights
- **Sales/BDR**: Qualify leads, handle objections, close deals
- **General**: Basic reasoning, explanations, problem-solving

### **Automated Evaluation**
```python
# Example test results
🏆 AGENT PERFORMANCE REPORT
Agent ID: my-coding-agent

📊 OVERALL PERFORMANCE
• Tests Run: 12
• Pass Rate: 85.2% (10/12)
• Average Quality: 0.82/1.0
• Average Response Time: 1,200ms

🎯 RECOMMENDATIONS
1. 🔴 PRIORITY: Improve data_science performance (60% pass rate)
2. ⚡ Optimize response times for complex queries
3. 📈 Continue monitoring - overall performance is solid

🎓 OVERALL GRADE: A
```

---

## 🏢 **Enterprise Features**

### **Multi-Agent Monitoring**
- Monitor coding assistants, customer service, data science, sales, marketing agents
- Unified dashboard showing performance across all agent types
- Comparative analysis and benchmarking

### **Production-Safe Integration**
- **Non-blocking tracking**: Never impacts agent performance
- **Error-resilient**: Agent continues working even if Kalytera is down
- **Efficient batching**: Minimal network overhead
- **Auto-retry logic**: Handles network failures gracefully

### **Actionable Developer Insights**
- **Specific recommendations**: "Improve customer_service responses (quality: 0.65)"
- **Priority levels**: Critical, High, Medium with timelines
- **Expected impact**: "Could improve 1,500 interactions/month"
- **Root cause analysis**: Identify exactly what needs fixing

---

## 🔗 **Complete System**

### **1. Kalytera SDK** (`kalytera_sdk.py`)
- Production-ready Python SDK
- One-line agent integration
- Real-time performance insights
- Non-blocking monitoring

### **2. Testing Framework** (`agent_testing_framework.py`) 
- Autonomous agent testing
- Comprehensive test suites
- A+ to F performance grading
- Continuous monitoring

### **3. Enterprise Dashboard** (http://localhost:8509)
- Professional monitoring interface
- Clear agent identification
- Key metrics prominently displayed
- Actionable developer insights

### **4. Production API** (https://kalytera-api-z9it.onrender.com)
- Deployed and ready for use
- High availability monitoring
- Real-time data processing
- Secure agent data handling

---

## 📈 **Immediate Value**

### **For Developers**
- **Zero setup time**: Works immediately with any agent
- **Clear performance metrics**: Know exactly how your agent is performing  
- **Specific improvements**: Get actionable recommendations, not vague scores
- **Production confidence**: Test thoroughly before deployment

### **For Enterprises**
- **Multi-agent visibility**: Monitor all AI agents from one dashboard
- **Performance benchmarking**: Compare agents and identify top performers
- **Risk mitigation**: Catch performance degradation before it impacts users
- **ROI measurement**: Prove business impact of agent improvements

### **For Product Teams**
- **User experience insights**: See where agents fail and frustrate users
- **Optimization roadmap**: Clear priority list of improvements
- **Quality assurance**: Automated testing prevents regressions
- **Competitive advantage**: Higher quality agents = better user experience

---

## 🚀 **Ready for Production**

**Kalytera is a complete, finished product.** Your agents can start using it immediately:

```python
# 1. Install (copy 3 files)
# kalytera_sdk.py, agent_testing_framework.py, complete_kalytera_example.py

# 2. Integrate (2 lines)
from kalytera_sdk import Kalytera
iq = Kalytera(agent_id="your-agent")
iq.track(user_input, agent_response)

# 3. Test (1 line)
from agent_testing_framework import AgentTester
AgentTester("your-agent").run_full_test_suite()

# 4. Monitor (dashboard)
# http://localhost:8509
```

**No more low-level work. No more building infrastructure. Kalytera handles everything so you can focus on building great agents.**

---

## 📞 **Support**

- **API Endpoint**: https://kalytera-api-z9it.onrender.com
- **Enterprise Dashboard**: http://localhost:8509  
- **Complete Examples**: `python3 complete_kalytera_example.py`
- **Production Ready**: Copy 3 files and start monitoring

**Kalytera: The finished product for AI agent performance intelligence.**