Metadata-Version: 2.4
Name: aisa-checker
Version: 0.1.0
Summary: AISA - Agentic AI Systems Architecture Compliance Checker. Evaluate agentic AI projects against the AISA 7-layer reference architecture.
Author-email: Omer Nacar <o.najar@tuwaiq.edu.sa>, Deema Alquffari <deema.alquffari@gmail.com>, Mohammed Alkhalifa <m.alkhalifa@tuwaiq.edu.sa>
License-Expression: MIT
Project-URL: Homepage, https://huggingface.co/AISA-Framework
Project-URL: Repository, https://github.com/AISA-Framework/aisa
Project-URL: Documentation, https://github.com/AISA-Framework/aisa#readme
Keywords: agentic-ai,architecture,compliance,governance,evaluation,llm,agents,aisa
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.20; extra == "anthropic"
Provides-Extra: google
Requires-Dist: google-generativeai>=0.5; extra == "google"
Provides-Extra: all
Requires-Dist: openai>=1.0; extra == "all"
Requires-Dist: anthropic>=0.20; extra == "all"
Requires-Dist: google-generativeai>=0.5; extra == "all"
Dynamic: license-file

# AISA - Agentic AI Systems Architecture Compliance Checker

[![PyPI version](https://badge.fury.io/py/aisa.svg)](https://pypi.org/project/aisa/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Evaluate your agentic AI system against the AISA reference architecture.**

AISA checks your project across 7 architectural layers and 4 cross-layer contracts, identifying gaps in governance, evaluation, safety, and operational readiness that most agentic AI frameworks miss.

## Quick Start

```bash
pip install aisa
```

### Pattern-Based Scan (fast, no API key)

```bash
aisa check ./my-agent-project
```

### AI-Powered Analysis (deep, requires API key)

```bash
# Option 1: Pass API key directly
aisa ai-check ./my-agent-project --api-key sk-...

# Option 2: Use environment variable
export OPENAI_API_KEY=sk-...    # or ANTHROPIC_API_KEY or GOOGLE_API_KEY
aisa ai-check ./my-agent-project

# Option 3: Choose provider
aisa ai-check ./my-agent-project --provider anthropic --api-key sk-ant-...
```

### Full Analysis (best accuracy - combines both)

```bash
aisa full-check ./my-agent-project --report
```

### Interactive Assessment (works for any system)

```bash
aisa assess
```

## What It Evaluates

AISA evaluates your project across the complete agentic AI lifecycle:

| Layer | What's Checked |
|-------|---------------|
| **L1: LLM Foundation** | Model adapters, prompt templates, context management, safety filters |
| **L2: Tool & Environment** | Tool schemas, validation, sandboxing, permissions, MCP, error handling |
| **L3: Cognitive Agent** | Planning, memory, goal tracking, reflection, feedback integration |
| **L4: Infrastructure** | Orchestration, state management, multi-agent, tracing, logging |
| **L5: Evaluation** | Component tests, trajectory eval, behavioral monitoring, regression tests |
| **L6: Dev & Deployment** | Versioning, CI/CD, A/B testing, benchmarking, staged rollout |
| **L7: Governance** | Policy-as-code, privacy, fairness, accountability, human oversight |

Plus 4 cross-layer contracts: **Policy**, **Telemetry**, **Versioning**, **Budget**.

## Scoring Modes

| Mode | Speed | Accuracy | API Key |
|------|:-----:|:--------:|:-------:|
| `check` | Fast | Good | No |
| `ai-check` | Slow | Better | Yes |
| `full-check` | Slowest | Best | Yes |
| `assess` | Manual | Depends on answers | No |

### How Scoring Works

- **Pattern scan** (`check`): Searches for 200+ code signatures (imports, function calls, config patterns) mapped to AISA criteria. Fast and deterministic.
- **AI analysis** (`ai-check`): Sends your code to an LLM that semantically analyzes architecture, not just keywords. Catches patterns that regex misses.
- **Combined** (`full-check`): Runs both, takes the higher score per criterion. Most accurate.
- **Assessment** (`assess`): 55 yes/no/partial questions. Works for any system, including non-Python or proprietary.

## Output

### Terminal Report

```
  LAYER COVERAGE:
  --------------------------------------------------------
  L1 LLM Foundation         ██████░░░░░░░░░░░░░░  33.3%  [Minimal]
  L2 Tool & Environment     ██░░░░░░░░░░░░░░░░░░  14.3%  [Absent]
  L3 Cognitive Agent        ███░░░░░░░░░░░░░░░░░  16.7%  [Absent]
  L4 Infrastructure         ░░░░░░░░░░░░░░░░░░░░   0.0%  [Absent]
  L5 Evaluation             ░░░░░░░░░░░░░░░░░░░░   0.0%  [Absent]
  L6 Dev & Deployment       ██░░░░░░░░░░░░░░░░░░  14.3%  [Absent]
  L7 Governance             ░░░░░░░░░░░░░░░░░░░░   0.0%  [Absent]

  OVERALL AISA SCORE: 9/100
  GRADE: F - Significant Gaps
```

### Markdown Report (`--report`)

Generates a detailed `aisa_report.md` with:
- Per-criterion scores with evidence
- Prioritized gap list
- Actionable recommendations per gap
- Framework detection results

## Supported Frameworks

Auto-detects and evaluates projects using:

- LangChain / LangGraph
- CrewAI
- Microsoft AutoGen
- OpenAI Assistants / Agents SDK
- Anthropic Claude SDK
- LlamaIndex
- Microsoft Semantic Kernel
- Google ADK

## LLM Providers (for ai-check / full-check)

| Provider | Environment Variable | Default Model |
|----------|---------------------|---------------|
| OpenAI | `OPENAI_API_KEY` | gpt-4o |
| Anthropic | `ANTHROPIC_API_KEY` | claude-sonnet-4-20250514 |
| Google | `GOOGLE_API_KEY` | gemini-2.0-flash |

```bash
# Override provider and model
aisa ai-check ./project --provider anthropic --model claude-sonnet-4-20250514
```

## Grading Scale

| Grade | Score | Meaning |
|:-----:|:-----:|---------|
| A | 80-100 | Production Ready |
| B | 60-79 | Maturing |
| C | 40-59 | Developing |
| D | 20-39 | Early Stage |
| F | 0-19 | Significant Gaps |

## About AISA

AISA (Agentic AI Systems Architecture) is a unified layered reference architecture for designing, deploying, evaluating, and governing agentic AI systems. Proposed by Tuwaiq Academy (TRDC), it provides a shared vocabulary and design blueprint for building production-grade autonomous AI systems.

**Paper:** Nacar, O., Alquffari, D., & Alkhalifa, M. (2026). "AISA: A Unified Architecture for Agentic AI Systems."

**Resources:** [HuggingFace](https://huggingface.co/AISA-Framework) | [Paper PDF](https://huggingface.co/AISA-Framework)

## Contributing

Contributions welcome! Areas we'd love help with:

- Additional detection patterns for frameworks
- New layer criteria based on real-world deployments
- Integration with CI/CD platforms
- Support for additional LLM providers

## License

MIT License - see [LICENSE](LICENSE) for details.

## Citation

```bibtex
@misc{nacar2026aisa,
  title={AISA: A Unified Architecture for Agentic AI Systems},
  author={Nacar, Omer and Alquffari, Deema and Alkhalifa, Mohammed},
  year={2026},
  institution={TRDC - Tuwaiq Academy},
}
```
