Metadata-Version: 2.4
Name: sofe
Version: 0.1.0
Summary: Stairway Open FinOps Engine — FinOps Policies as Code for AWS
Author-email: Carlos Cortez <carlos@cortez.cloud>
License: Apache-2.0
Project-URL: Homepage, https://github.com/breakingthecloud/sofe
Project-URL: Repository, https://github.com/breakingthecloud/sofe
Project-URL: Issues, https://github.com/breakingthecloud/sofe/issues
Keywords: finops,aws,cloud-governance,policy-as-code,cost-optimization
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: System :: Systems Administration
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: boto3>=1.34
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: click>=8.0
Requires-Dist: rich>=13.0

# 🏗️ SOFE — Stairway Open FinOps Engine

**FinOps Policies as Code for AWS.**

SOFE evaluates declarative YAML policies against live AWS infrastructure and produces actionable findings — idle resources, missing tags, governance violations, and cost savings opportunities.

```bash
sofe evaluate --policies ./policies/ --profile production
```

```
────────────────────────────────────────────────────────────────────────────────
Severity   Policy                    Resource             Message
────────────────────────────────────────────────────────────────────────────────
🟠 high    no-idle-ec2               i-0abc123def         avg_cpu = 2.1% (threshold: <5%)
🟡 medium  require-cost-tags         i-0def456ghi         missing: costCenter, owner
🟡 medium  no-unattached-ebs         vol-789abc           180 days old, 500GB
────────────────────────────────────────────────────────────────────────────────
Summary: 3 findings | Potential savings: $365.00/mo
```

---

## Why SOFE?

### The Problem

Teams today manage cloud costs **reactively** — they see the bill spike, panic, then scramble to find what changed. Existing tools either:

- **Alert on total spend** (AWS Budgets) — no root cause, no policy enforcement
- **Scan for security** (Prowler, ScoutSuite) — not cost-focused
- **Estimate costs** (Infracost) — pre-deploy only, no runtime enforcement
- **Lock you in** (Sentinel/HCP) — vendor-specific, not portable

**No tool does:** declarative cost+governance policies that evaluate against **live** infrastructure and produce findings with dollar-amount savings.

### The Solution

SOFE fills this gap:

```yaml
# policies/no-idle-production.yaml
apiVersion: sofe/v1
kind: Policy
metadata:
  name: no-idle-production
  description: "Flag idle EC2 in production (< 5% CPU for 30 days)"
spec:
  scope:
    environments: [production]
    resource_types: [aws.ec2]
  rule:
    metric: avg_cpu_utilization
    period: 30d
    operator: "<"
    threshold: 5
  severity: high
  actions:
    - type: recommend
      suggestion: "Rightsize or terminate"
      estimated_savings: calc
```

Write a policy once. Run it daily. Get findings with savings.

---

## Who Should Use SOFE?

| Role | Why SOFE matters |
|------|-----------------|
| **Cloud/DevOps Engineers** | Automate governance checks in CI/CD. `sofe evaluate --fail-on high` blocks deploys that violate cost policies. |
| **FinOps Practitioners** | Define cost optimization rules as code. Track compliance across accounts. Quantify waste. |
| **Platform Engineers** | Enforce tagging standards, idle resource cleanup, and architecture best practices at scale. |
| **CTOs / Engineering Managers** | Visibility into cloud waste without manual audits. "We save $X/month because of these policies." |
| **AWS Partners / Consultants** | Deliver FinOps assessments faster with repeatable, auditable policy evaluations. |

---

## Why SOFE is Key for FinOps + Governance

### 1. FinOps: Cost Optimization as Code

Traditional FinOps is manual: someone opens Cost Explorer, finds waste, creates a ticket. SOFE automates this:

```
Write policy → sofe evaluate → findings with $ savings → action
```

Every policy produces **quantified savings**: "$340/mo if you terminate this idle instance."

### 2. Cloud Governance: Policies that Actually Enforce

Tags, encryption, public access, budget limits — every team has rules but no enforcement. SOFE makes them executable:

```yaml
- require-cost-tags       → "All resources must have owner + costCenter"
- s3-encryption-required  → "All S3 buckets must have encryption enabled"
- no-public-without-waf   → "No public-facing resource without WAF"
```

Not just documentation. Actual enforcement in CI/CD.

### 3. DevOps: Shift-Left Cost Awareness

Add `sofe evaluate --fail-on high` to your GitHub Action or CI pipeline. Developers see cost violations **before** merge, not after the bill arrives.

### 4. BYaML Integration: Architecture-Aware FinOps

SOFE uses [BYaML](https://byaml.org) component types (`aws.ec2`, `aws.s3`, etc.) — the same type system used for architecture governance. This means:

- Policies reference the same types as your architecture definitions
- Findings map directly to BYaML components
- Cost data correlates with architecture versions

---

## Quick Start

### Install

```bash
pip install sofe
```

### Write Your First Policy

```yaml
# policies/require-tags.yaml
apiVersion: sofe/v1
kind: Policy
metadata:
  name: require-cost-tags
  description: "All resources must have owner and costCenter tags"
spec:
  scope:
    resource_types: [aws.ec2, aws.rds, aws.s3]
  rule:
    metric: has_tag:owner
    operator: "=="
    threshold: 0
  severity: medium
  actions:
    - type: finding
```

### Validate

```bash
sofe validate --policies ./policies/
```

### Evaluate

```bash
# Against real AWS (uses your AWS profile)
sofe evaluate --policies ./policies/ --profile production

# Output as JSON (for automation)
sofe evaluate --policies ./policies/ --format json > findings.json

# CI/CD mode (exit code 1 if high/critical found)
sofe evaluate --policies ./policies/ --fail-on high
```

---

## How It Works

```
┌─────────────────┐     ┌──────────────┐     ┌──────────────────────┐
│ Policy Loader   │     │  Collectors  │     │  Evaluation Engine   │
│                 │     │              │     │                      │
│ Reads YAML      │────▶│ AWS APIs:    │────▶│ For each policy:     │
│ Validates       │     │ EC2, RDS     │     │ match scope →        │
│ schema          │     │ S3, Lambda   │     │ evaluate condition → │
│                 │     │ CloudWatch   │     │ if violated →        │
└─────────────────┘     └──────────────┘     │   generate finding   │
                                             └──────────┬───────────┘
                                                        │
                                              ┌─────────▼─────────┐
                                              │  Output            │
                                              │  • Table (CLI)     │
                                              │  • JSON (CI/CD)    │
                                              │  • Markdown (PRs)  │
                                              └────────────────────┘
```

---

## Supported Metrics

| Metric | Source | Resources |
|--------|--------|-----------|
| `avg_cpu_utilization` | CloudWatch (30d avg) | EC2, RDS |
| `monthly_cost` | Cost Explorer | All |
| `running_days` | LaunchTime | EC2, RDS |
| `has_tag:{key}` | Tags API | All |
| `storage_used_gb` | CloudWatch | S3, EBS |
| `connections` | CloudWatch | RDS |
| `invocations` | CloudWatch | Lambda |

---

## Pre-Built Policies

| Policy | Type | Severity |
|--------|------|----------|
| `no-idle-ec2` | Cost Optimization | high |
| `no-idle-rds` | Cost Optimization | high |
| `require-cost-tags` | Governance | medium |
| `no-oversized-staging` | Cost Optimization | high |
| `s3-lifecycle-required` | Storage | medium |
| `s3-encryption-required` | Security/Cost | high |
| `no-unattached-ebs` | Storage | medium |
| `no-old-snapshots` | Storage | low |
| `budget-exceeded` | Budget | critical |
| `no-public-without-waf` | Security/Cost | high |

---

## CI/CD Integration

### GitHub Actions

```yaml
- name: FinOps Policy Check
  run: |
    pip install sofe
    sofe evaluate --policies ./policies/ --fail-on high --format json > findings.json
```

### Exit Codes

| Code | Meaning |
|:----:|---------|
| 0 | No violations (or below `--fail-on` threshold) |
| 1 | Violations found at or above `--fail-on` severity |

---

## Comparison

| Tool | Cost Policies | Live Eval | Savings Calc | CI/CD | Open Source |
|------|:---:|:---:|:---:|:---:|:---:|
| **SOFE** | ✅ | ✅ | ✅ | ✅ | ✅ |
| AWS Budgets | ❌ (alerts only) | ❌ | ❌ | ❌ | ❌ |
| Infracost | 🟡 (pre-deploy) | ❌ | ✅ | ✅ | ✅ |
| OPA/Rego | ✅ (security) | ❌ | ❌ | ✅ | ✅ |
| Sentinel | ✅ | ❌ | ❌ | ✅ | ❌ (HCP only) |
| Prowler | ❌ (security only) | ✅ | ❌ | ✅ | ✅ |

---

## Ecosystem: Competitors & Complementary Tools

### Competitors (overlap with SOFE)

| Tool | Type | What it does | How SOFE differs |
|------|------|-------------|-----------------|
| **OPA / Rego** | OSS | General policy engine (security-focused) | SOFE is cost/FinOps-focused with savings calculations. OPA doesn't calculate $. |
| **HashiCorp Sentinel** | Proprietary | Policy-as-code for Terraform | Locked to HCP/Terraform Cloud. SOFE is runtime (evaluates live infra, not just plans). |
| **Infracost** | OSS | Cost estimation pre-deploy | Pre-deploy only. SOFE evaluates running infra + historical drift. Complementary. |
| **AWS Config Rules** | AWS Native | Compliance rules on AWS resources | Limited to AWS, no cost focus, no CI/CD output, no portability. |
| **Prowler** | OSS | Security & compliance scanning | Security-focused (CIS, PCI-DSS). Doesn't calculate cost savings or enforce FinOps. |
| **Checkov** | OSS | IaC static analysis (Terraform, CF) | Pre-deploy only (scans .tf files). SOFE scans live resources. |
| **Cloud Custodian** | OSS | Policy engine for cloud resources | Closest competitor. Actions (stop/terminate) built-in. SOFE is lighter, YAML-first, FinOps-focused. |
| **Kubecost** | OSS/Paid | Kubernetes cost monitoring | K8s only. SOFE covers all AWS services. |
| **Vantage** | SaaS | FinOps dashboard + alerts | Dashboard, not policy engine. No CI/CD. No custom rules. |
| **CloudZero** | SaaS | Cost intelligence platform | Enterprise SaaS ($$$). No self-hosted. No policies-as-code. |
| **Spot.io / NetApp** | SaaS | Cloud optimization + autoscaling | Optimization execution, not policy definition. Complementary. |
| **Apptio Cloudability** | SaaS | Enterprise FinOps platform | Enterprise-only, expensive. No CI/CD integration. No code-first approach. |
| **nOps** | SaaS | AWS cost optimization + scheduling | Automation focus. No declarative policies. |
| **CAST AI** | SaaS | K8s cost optimization | K8s-only autoscaling. Not a policy engine. |

### Complementary Tools (use alongside SOFE)

| Tool | How it complements SOFE |
|------|------------------------|
| **Infracost** | Pre-deploy cost estimation → SOFE catches what slipped through post-deploy |
| **Terraform / OpenTofu** | Defines infra → SOFE evaluates if running infra matches cost policies |
| **AWS Cost Explorer** | Data source → SOFE collectors fetch from it |
| **CloudWatch** | Metrics source → SOFE uses CPU, connections, invocations |
| **Steampipe** | SQL-based cloud inventory → could be alternate data source for SOFE |
| **Prometheus + Grafana** | Monitoring → SOFE could consume Prometheus metrics (future) |
| **BYaML** | Architecture definitions → SOFE policies use same type system |
| **byaml-finops-mcp** | MCP tools for AI assistants → SOFE findings feed into AI reasoning |
| **FinOptix** | AI model for FinOps → explains SOFE findings in natural language |
| **GitHub Actions / GitLab CI** | CI/CD → SOFE runs as pipeline step with `--fail-on` |
| **Slack / PagerDuty** | Notifications → SOFE can webhook findings (future) |
| **Neo4j** | Graph DB → SOFE findings + BYaML relationships = cost propagation graph (future) |

### The SOFE Position

```
┌─────────────────────────────────────────────────────────────────┐
│                    Cloud Cost Lifecycle                           │
├─────────────┬──────────────┬──────────────────┬─────────────────┤
│  PLAN       │  DEPLOY      │  RUN             │  OPTIMIZE       │
│             │              │                  │                 │
│ Infracost   │ Sentinel     │ ★ SOFE ★         │ Spot.io         │
│ Checkov     │ OPA/Rego     │ Cloud Custodian  │ CAST AI         │
│             │ Checkov      │ AWS Config       │ nOps            │
│             │              │ Prowler          │ Kubecost        │
├─────────────┴──────────────┴──────────────────┴─────────────────┤
│  VISIBILITY: Vantage, CloudZero, Apptio, AWS Cost Explorer       │
└─────────────────────────────────────────────────────────────────┘

SOFE lives in the RUN phase: evaluate LIVE infrastructure against
declarative FinOps policies. Produce findings with dollar savings.
```

### Why SOFE vs Cloud Custodian?

Cloud Custodian is the closest open source alternative. Key differences:

| | SOFE | Cloud Custodian |
|--|------|----------------|
| **Focus** | FinOps + cost governance | Security + compliance + ops |
| **Policy format** | Clean YAML (Pydantic-validated) | Complex YAML with filters/actions DSL |
| **Savings calculation** | Built-in ($ per finding) | Not included |
| **BYaML integration** | Native (same type system) | None |
| **AI reasoning** | FinOptix integration (future) | None |
| **CI/CD** | `--fail-on` exit code | Requires wrapper |
| **Scope** | AWS first, multi-cloud future | AWS + Azure + GCP |
| **Maturity** | New (2026) | Mature (2016+, Capital One) |

SOFE is opinionated toward **FinOps** — every finding has a dollar amount. Cloud Custodian is a general-purpose policy engine that happens to work on cloud resources.

---

## License

Apache 2.0 — free to use, modify, and distribute.

---

## Contributing

1. Fork the repo
2. Add a policy to `policies/` or a collector to `sofe/collectors/`
3. Submit a PR

---

## Built by

[Carlos Cortez](https://cortez.cloud) — AWS Community Hero, CTO @ BWIT Solutions.
Part of the [BYaML](https://byaml.org) ecosystem for cloud architecture governance.
