Metadata-Version: 2.4
Name: cloudleak
Version: 1.0.3
Summary: Production-grade AWS FinOps CLI scanner
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: boto3>=1.34
Requires-Dist: botocore>=1.34
Requires-Dist: rich>=13.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: jinja2>=3.1
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-mock>=3.12; extra == "dev"
Requires-Dist: moto[all]>=5.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: pyinstaller>=6.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Dynamic: license-file

# cloudleak

> Production-grade, read-only AWS FinOps CLI scanner. Find the resources in your AWS account that are spending money for no productive reason.

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![PyPI version](https://img.shields.io/pypi/v/cloudleak.svg)](https://pypi.org/project/cloudleak/)
[![CI](https://github.com/ragnild/CloudLeak/actions/workflows/ci.yml/badge.svg)](https://github.com/ragnild/CloudLeak/actions/workflows/ci.yml)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://python.org)

---

## What is cloudleak?

cloudleak connects to your AWS account using your existing credentials and scans resources across one region or all regions, producing a prioritised finding report. Each finding includes the specific evidence that triggered it, an estimated monthly waste figure, and a concrete recommendation.

**143 rules** across EC2, EBS, Networking, RDS, S3, Lambda, CloudWatch, AI/ML, Containers, Security, Governance, Analytics, Messaging, Desktop, and more.

### Why not just use Cost Explorer?

Cost Explorer tells you what you spent. cloudleak tells you _why_ you're over-spending and which specific resources to address. It integrates AWS Compute Optimizer right-sizing signals, checks for resources that are running but idle, and catches governance anti-patterns that inflate your bill invisibly (untagged resources, missing lifecycle policies, provisioned throughput sitting empty).

### Key properties

- **Read-only always.** Zero mutations. Safe to run in production accounts with least-privilege.
- **Graceful permission degradation.** `AccessDenied` marks a rule `SKIPPED` — it never crashes.
- **Offline pricing.** Prices are loaded from a local `prices.json` cache. No live pricing calls during a scan.
- **Parallel execution.** Rule groups run concurrently via a thread pool.
- **Configurable thresholds.** Every idle-day window, CPU threshold, and size limit is driven by `cloudleak.yaml`.
- **No telemetry.** Safe for air-gapped and regulated environments.

---

## Installation

AWS credentials must be configured before running a scan (any standard method: SSO, env vars, `~/.aws/credentials`, instance role).

### pip / pipx (recommended)

Requires Python 3.10+.

```bash
# pipx keeps cloudleak isolated from your other Python environments
pipx install cloudleak

# or plain pip
pip install cloudleak
```

### Standalone binary (no Python required)

Download the pre-built binary for your OS from [GitHub Releases](https://github.com/ragnild/CloudLeak/releases/latest):

| OS | File |
|---|---|
| Linux x86-64 | `cloudleak-linux-x86_64` |
| macOS (Apple Silicon) | `cloudleak-macos-arm64` |
| Windows x86-64 | `cloudleak-windows-x86_64.exe` |

```bash
# Linux / macOS — make executable first
chmod +x cloudleak-linux-x86_64
./cloudleak-linux-x86_64 --version
```

Standalone binaries are the recommended installation for **air-gapped and regulated environments** where `pip install` is not available.

### Docker

```bash
docker run --rm \
  -e AWS_ACCESS_KEY_ID \
  -e AWS_SECRET_ACCESS_KEY \
  -e AWS_SESSION_TOKEN \
  -e AWS_DEFAULT_REGION=us-east-1 \
  ghcr.io/ragnild/cloudleak scan aws region us-east-1

# Or mount your ~/.aws directory for SSO / named profiles
docker run --rm \
  -v ~/.aws:/root/.aws:ro \
  ghcr.io/ragnild/cloudleak scan aws region us-east-1 --profile my-profile
```

### Homebrew (macOS / Linux)

```bash
brew tap ragnild/cloudleak
brew install cloudleak
```

### Install from source

```bash
git clone https://github.com/ragnild/CloudLeak
cd CloudLeak
pip install -e ".[dev]"
```

---

## Quick start

### Step 1 — Check permissions

```bash
cloudleak doctor aws
```

This runs a pre-flight check against your current credentials. It reports which permissions are available and which rules will be `SKIPPED` due to missing permissions.

### Step 2 — Refresh the pricing cache

```bash
cloudleak prices update
```

Downloads on-demand pricing from the AWS Pricing API and writes `~/.cloudleak/prices.json`. Run this once, then again whenever you want fresh prices (the scanner warns you if the cache is older than 30 days).

### Step 3 — Scan a region

```bash
cloudleak scan aws region us-east-1
```

That's it. cloudleak prints a colour-coded report to your terminal and exits.

---

## CLI Reference

```
cloudleak scan aws region <region>                  Scan a single region
cloudleak scan aws --all-regions                    Scan all active regions
cloudleak scan aws region <region> --profile <name> Use a named AWS CLI profile
cloudleak scan aws region <region> --export json    Export findings to JSON
cloudleak scan aws region <region> --export csv     Export findings to CSV
cloudleak scan aws region <region> --export html    Export findings to HTML report
cloudleak scan aws region <region> --export all     All formats simultaneously
cloudleak scan aws region <region> --config ./cloudleak.yaml  Custom config file
cloudleak scan aws --rules ec2,rds,s3 region <region>         Scope to rule groups
cloudleak scan aws region <region> --severity HIGH            Only HIGH findings
cloudleak doctor aws                                Pre-flight permission check
cloudleak prices update                             Refresh local pricing cache
cloudleak rules list                                List all rules
cloudleak rules describe <rule-id>                  Full spec for one rule
```

### `cloudleak scan aws`

| Flag | Default | Description |
|------|---------|-------------|
| `region <region>` | — | Scan a specific AWS region |
| `--all-regions` | false | Enumerate and scan all active regions |
| `--profile <name>` | (env/default) | Named AWS CLI profile |
| `--export <format>` | console | `json`, `csv`, `html`, `all` |
| `--output-file <path>` | stdout | Write export to file |
| `--config <path>` | (searched) | Path to `cloudleak.yaml` |
| `--rules <groups>` | all | Comma-separated rule groups: `ec2`, `ebs`, `networking`, `rds`, `s3`, `lambda`, `cloudwatch`, `aiml`, `containers`, `security`, `governance`, `analytics`, `messaging`, `desktop` |
| `--severity <level>` | all | Filter: `HIGH`, `MEDIUM`, `LOW` |
| `--verbose` | false | Show per-rule PASS/FOUND/SKIP detail |

### `cloudleak doctor aws`

Checks every IAM permission in the manifest against your current credentials. Exits `0` if all available, `1` if any missing.

### `cloudleak prices update`

Calls the AWS Bulk Pricing API (`us-east-1` endpoint) and writes `~/.cloudleak/prices.json`. Requires `pricing:GetProducts` permission. Does not need to run in a specific region.

---

## Configuration

cloudleak searches for `cloudleak.yaml` in: `./`, `~/.cloudleak/`, `/etc/cloudleak/`.

Copy `cloudleak.yaml.example` from the repo as your starting point — every key is documented inline.

### Key sections

| Section | Controls |
|---------|----------|
| `scan.lookback_days` | CloudWatch metric lookback window (default: 14) |
| `scan.parallel_workers` | Max concurrent rule groups (default: 10) |
| `thresholds.ec2.*` | EC2 idle/stopped thresholds |
| `thresholds.ebs.*` | EBS unattached/snapshot/AMI age thresholds |
| `thresholds.networking.*` | LB/NAT/VPN/TGW idle thresholds |
| `thresholds.rds.*` | RDS/Aurora/ElastiCache idle thresholds |
| `thresholds.s3.*` | S3 request/lifecycle/EFS thresholds |
| `thresholds.lambda.*` | Lambda/SFN/SQS idle thresholds |
| `thresholds.cloudwatch.*` | Log group retention/idle/alarm thresholds |
| `thresholds.aiml.*` | SageMaker/Bedrock/Comprehend/Rekognition/Kendra/Lex thresholds |
| `thresholds.containers.*` | ECS/ECR/EKS/AppRunner thresholds |
| `thresholds.security.*` | Secrets Manager/KMS/IAM idle thresholds |
| `thresholds.governance.*` | Required tags list |
| `thresholds.analytics.*` | Redshift/OpenSearch/Glue/Kinesis/MSK/EMR idle thresholds |
| `thresholds.messaging.*` | Amazon MQ/Firehose idle thresholds |
| `thresholds.desktop.*` | WorkSpaces/AppStream idle thresholds |
| `pricing.cache_file` | Path to `prices.json` |
| `pricing.warn_if_stale_days` | Age after which a stale-cache warning is shown |

---

## IAM Setup

### Attach the managed policy

The `iam/cloudleak-read-policy.json` file in this repo is a deployable IAM policy that grants every read permission cloudleak needs.

```bash
# Create the policy in your account
aws iam create-policy \
  --policy-name CloudLeakReadOnly \
  --policy-document file://iam/cloudleak-read-policy.json

# Attach to a role (CI/CD)
aws iam attach-role-policy \
  --role-name your-scanner-role \
  --policy-arn arn:aws:iam::<account-id>:policy/CloudLeakReadOnly

# Or attach to a user
aws iam attach-user-policy \
  --user-name your-scanner-user \
  --policy-arn arn:aws:iam::<account-id>:policy/CloudLeakReadOnly
```

### Verify permissions

After attaching, run `cloudleak doctor aws` to confirm which permissions are active.

### Note on pricing

The `cloudleak prices update` command additionally requires `pricing:GetProducts`. This is a separate permission from scan permissions. If you only want to run scans using a pre-built `prices.json`, you do not need to grant `pricing:GetProducts` to the scanner role.

---

## Output Formats

### Console (default)

Colour-coded by severity, grouped by service. Includes a summary table at the end.

```
╔══════════════════════════════════════════════════════════════════╗
║  cloudleak  ·  Account: 123456789012  ·  Region: us-east-1      ║
╚══════════════════════════════════════════════════════════════════╝

── EC2 & Compute ──────────────────────────────────────────────────

  🔴 HIGH  EC2-001  Stopped EC2 Instances
     i-0abc123def456789  (m5.2xlarge)  stopped 23 days
     Attached EBS: 200 GB gp3  →  est. waste: $16.00/mo
     └─ Terminate or start. Create AMI first if needed.

── SUMMARY ────────────────────────────────────────────────────────
  Total findings:           34    Est. monthly waste:  $1,847.23
  HIGH: 18   MEDIUM: 12   LOW: 4   Skipped rules: 2
```

### JSON (`--export json`)

Structured output with scan metadata, summary, findings array, and skipped rules list. Schema defined in `PRODUCT_SPEC.md §7`.

### CSV (`--export csv`)

Flat table, one row per finding. Columns: `rule_id, rule_name, severity, service, resource_id, resource_name, resource_arn, region, estimated_monthly_cost_usd, recommendation, evidence_json`.

### HTML (`--export html`)

Self-contained single-file report. No external CDN. Includes summary cards, a bar chart of the top 10 most expensive findings, and a sortable/filterable table.

---

## All 114 Rules

### EC2 & Compute

| Rule ID | Name | Severity |
|---------|------|----------|
| EC2-001 | Stopped EC2 Instances | HIGH |
| EC2-002 | Idle EC2 Instances (Low CPU) | HIGH |
| EC2-003 | Oversized EC2 Instances (Compute Optimizer) | MEDIUM |
| EC2-004 | On-Demand Candidates for Reserved Instances / Savings Plans | HIGH |
| EC2-012 | Unattached Elastic IPs | HIGH |
| EC2-013 | Detached Network Interfaces (ENIs) | MEDIUM |
| EC2-014 | Idle EC2 Dedicated Hosts | HIGH |
| EC2-015 | Burstable Instance Candidates (Wrong Family) | MEDIUM |
| EC2-016 | Unused EC2 Key Pairs | LOW |
| EC2-018 | Empty Placement Groups | LOW |
| EC2-019 | Unused EC2 Capacity Reservations | HIGH |

### EBS & Block Storage

| Rule ID | Name | Severity |
|---------|------|----------|
| EC2-005 | Unattached EBS Volumes | HIGH |
| EC2-006 | Zero-Activity EBS Volumes | HIGH |
| EC2-007 | Oversized EBS Volumes (Compute Optimizer) | MEDIUM |
| EC2-008 | Old EBS Snapshots (No Linked AMI) | MEDIUM |
| EC2-009 | Snapshots from Terminated Instances | HIGH |
| EC2-010 | Old Unused AMIs | MEDIUM |
| EC2-011 | Invalid or Broken AMIs | LOW |
| EC2-017 | Oversized Root Volume vs Instance Needs | MEDIUM |
| EC2-020 | EBS gp2 Volumes Eligible for gp3 Migration | LOW |

### Load Balancers & Networking

| Rule ID | Name | Severity |
|---------|------|----------|
| NET-001 | Idle Load Balancers | HIGH |
| NET-002 | Load Balancers with No Healthy Targets | HIGH |
| NET-003 | Empty Target Groups | MEDIUM |
| NET-004 | Idle NAT Gateways | HIGH |
| NET-005 | NAT Gateway with No Route Table References | HIGH |
| NET-006 | VPN Connections with Both Tunnels Down | MEDIUM |
| NET-007 | Idle Transit Gateway Attachments | MEDIUM |
| NET-008 | Idle VPC Interface Endpoints | MEDIUM |
| NET-009 | Unused Customer Gateways | LOW |
| NET-010 | Unused Internet Gateways | LOW |
| NET-011 | Idle Global Accelerators | MEDIUM |
| NET-012 | WAF WebACLs with No Associated Resources | MEDIUM |
| NET-013 | Idle CloudFront Distributions | MEDIUM |
| NET-015 | Direct Connect Virtual Interfaces in Down State | MEDIUM |

### RDS & Databases

| Rule ID | Name | Severity |
|---------|------|----------|
| RDS-001 | Idle RDS Instances | HIGH |
| RDS-002 | Stopped RDS Instances | HIGH |
| RDS-003 | Oversized RDS Instances | MEDIUM |
| RDS-004 | Old Manual RDS Snapshots | MEDIUM |
| RDS-005 | RDS Instances Using gp2 Storage | MEDIUM |
| RDS-006 | Idle Aurora Clusters | HIGH |
| RDS-007 | Aurora Serverless v1 Without Auto-Pause (Idle) | HIGH |
| RDS-008 | Idle ElastiCache Clusters | HIGH |
| RDS-009 | Oversized ElastiCache Nodes | MEDIUM |
| RDS-010 | Idle DynamoDB Tables (Provisioned Mode) | MEDIUM |
| RDS-011 | DynamoDB Provisioned Capacity Under-Utilization | MEDIUM |
| RDS-012 | Old Manual Aurora Cluster Snapshots | MEDIUM |
| RDS-013 | Idle DocumentDB Clusters | HIGH |
| RDS-014 | Idle Neptune Clusters | HIGH |
| RDS-015 | Idle MemoryDB for Redis Clusters | HIGH |
| RDS-016 | Idle RDS Proxy Endpoints | MEDIUM |

### S3 & Storage

| Rule ID | Name | Severity |
|---------|------|----------|
| S3-001 | S3 Buckets with No Requests | MEDIUM |
| S3-002 | S3 Buckets with No Lifecycle Policy | LOW |
| S3-003 | S3 Buckets with No Storage Tiering Transition | LOW |
| S3-004 | S3 Versioning Enabled Without Version Expiry Rule | LOW |
| S3-005 | S3 Intelligent-Tiering on Predictably-Accessed Buckets | LOW |
| S3-006 | Requester Pays Disabled on Cross-Account Shared Buckets | LOW |
| S3-007 | Empty Glacier Vaults | LOW |
| S3-008 | Idle EFS File Systems | HIGH |
| S3-009 | EFS Over-Provisioned Throughput | MEDIUM |
| S3-010 | Idle FSx File Systems | HIGH |

### Lambda & Serverless

| Rule ID | Name | Severity |
|---------|------|----------|
| LAM-001 | Lambda Functions with No Invocations | MEDIUM |
| LAM-002 | Lambda Provisioned Concurrency Underutilized | HIGH |
| LAM-003 | Lambda Functions with Over-Allocated Memory | MEDIUM |
| LAM-004 | API Gateway APIs with No Requests | MEDIUM |
| LAM-005 | API Gateway REST APIs with No Backend Integrations | MEDIUM |
| LAM-006 | Idle Step Functions State Machines | LOW |
| LAM-007 | EventBridge Rules with No Targets | LOW |
| LAM-008 | SQS Queues with No Activity | LOW |
| LAM-009 | Lambda Function URLs with No Traffic | LOW |
| LAM-010 | Idle API Gateway WebSocket APIs | MEDIUM |

### CloudWatch & Logging

| Rule ID | Name | Severity |
|---------|------|----------|
| CW-001 | CloudWatch Log Groups Without Retention Policy | LOW |
| CW-002 | CloudWatch Log Groups with Zero Ingestion | LOW |
| CW-003 | CloudWatch Alarms Stuck in INSUFFICIENT_DATA | LOW |
| CW-004 | DynamoDB Contributor Insights Enabled on Idle Tables | MEDIUM |
| CW-005 | X-Ray Sampling Rules for Services with No Traces | LOW |
| CW-006 | CloudWatch Metric Streams with No Active Consumers | MEDIUM |

### AI/ML Services

| Rule ID | Name | Severity |
|---------|------|----------|
| ML-001 | SageMaker Endpoints with No Invocations | HIGH |
| ML-002 | SageMaker Notebook Instances Stopped Too Long | MEDIUM |
| ML-003 | SageMaker Studio Apps Running Idle | MEDIUM |
| ML-004 | Stuck SageMaker Training Jobs | HIGH |
| ML-005 | Bedrock Provisioned Throughput with No Invocations | HIGH |
| ML-006 | Amazon Comprehend Endpoints with No Requests | HIGH |
| ML-007 | Rekognition Custom Labels Models Running Idle | HIGH |
| ML-008 | SageMaker Model Packages Unused After 90 Days | LOW |
| ML-009 | Kendra Indexes with No Query Activity | HIGH |
| ML-010 | Lex V2 Bot Aliases with No Conversations | MEDIUM |

### Containers

| Rule ID | Name | Severity |
|---------|------|----------|
| CON-001 | ECS Services with Zero Running Tasks | HIGH |
| CON-002 | ECR Repositories with No Recent Image Pulls | LOW |
| CON-003 | Old ECR Images Never Pulled | MEDIUM |
| CON-004 | EKS Clusters with No Active Nodes | HIGH |
| CON-005 | ECS Fargate Services with Low CPU Utilization | MEDIUM |
| CON-006 | App Runner Services with No Requests | HIGH |

### Security & Identity

| Rule ID | Name | Severity |
|---------|------|----------|
| SEC-001 | Shield Advanced with No Protected Resources | HIGH |
| SEC-002 | Unused ACM Certificates | LOW |
| SEC-003 | Secrets Manager Secrets Not Accessed | MEDIUM |
| SEC-004 | Unused KMS Customer Managed Keys | MEDIUM |
| SEC-005 | IAM Roles with No Recent Activity | MEDIUM |
| SEC-007 | Macie Enabled with No Classification Jobs | MEDIUM |

### Tagging & Governance

| Rule ID | Name | Severity |
|---------|------|----------|
| GOV-001 | Untagged Critical Resources | MEDIUM |
| GOV-002 | CloudFormation Stacks in ROLLBACK_COMPLETE | HIGH |
| GOV-003 | Service Catalog Tainted Provisioned Products | MEDIUM |
| GOV-004 | RDS Instances Without Cost Allocation Tags | LOW |

### Analytics & Data

| Rule ID | Name | Severity |
|---------|------|----------|
| ANA-001 | Idle Redshift Clusters | HIGH |
| ANA-002 | Idle OpenSearch / Elasticsearch Domains | HIGH |
| ANA-003 | Glue Development Endpoints Running Idle | HIGH |
| ANA-004 | Idle Kinesis Data Streams | MEDIUM |
| ANA-005 | Idle MSK (Managed Kafka) Clusters | HIGH |
| ANA-006 | EMR Clusters with No Active Steps | HIGH |
| ANA-007 | Glue ETL Jobs Scheduled but Never Successfully Executed | MEDIUM |
| ANA-008 | Redshift Serverless Namespaces with No RPU Activity | MEDIUM |

### Messaging & Streaming

| Rule ID | Name | Severity |
|---------|------|----------|
| MSG-001 | Idle Amazon MQ Brokers | HIGH |
| MSG-002 | Firehose Delivery Streams with No Records | MEDIUM |

### Desktop & Streaming

Severity varies by resource configuration: `HIGH` for always-on resources, `LOW` for auto-stop or on-demand resources.

| Rule ID | Name | Severity |
|---------|------|----------|
| DSK-001 | Amazon WorkSpaces Not Used | HIGH / LOW |
| DSK-002 | AppStream 2.0 Fleets with No Active Sessions | HIGH / LOW |

---

## Contributing

### Adding a rule

1. Create a file in `src/rules/<group>.py` (or add to an existing group file).
2. Implement the `Rule` protocol: `rule_id`, `name`, `service`, `group`, `severity`, `description`, `rationale`, `permissions`, `thresholds`, and `check(ctx) -> list[Finding]`.
3. Call `register(_YourRuleClass())` at the bottom of the rule module — the rule self-registers when the module is imported.
4. Add a decision contract doc to `docs/rules/`.
5. Write a unit test in `tests/unit/` with a mocked AWS response fixture.
6. Run `cloudleak rules list` to confirm it appears.

### Development setup

```bash
git clone https://github.com/ragnild/cloudleak
cd cloudleak
pip install -e ".[dev]"
pytest tests/unit/
```

### Principles — always enforce before merging

- No write API calls (`Create`, `Delete`, `Update`, `Put`, `Stop`, `Modify`)
- No hardcoded thresholds — always read from config with a default
- `AccessDenied` → `SKIPPED`, never a crash
- No live pricing calls during a scan
- Every `Finding` has `resource_id`, `resource_arn`, `evidence`, `estimated_monthly_cost_usd`, `recommendation`

---

## License

MIT — see [LICENSE](LICENSE).
