Metadata-Version: 2.4
Name: cloud-infra-auditor
Version: 1.1.0
Summary: Production-inspired AWS infrastructure auditing and cost optimization CLI
Author: Harshil Kalsariya
License-Expression: MIT
Keywords: aws,boto3,devops,finops,cloud,cli,automation,python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: boto3>=1.40
Requires-Dist: typer>=0.16
Requires-Dist: rich>=14.0
Provides-Extra: dev
Requires-Dist: pytest>=9.1; extra == "dev"
Requires-Dist: moto>=5.1; extra == "dev"
Requires-Dist: black>=25.1; extra == "dev"
Requires-Dist: ruff>=0.12; extra == "dev"
Dynamic: license-file

```
   ________    ____  __  ______
  / ____/ /   / __ \/ / / / __ \
 / /   / /   / / / / / / / / / /
/ /___/ /___/ /_/ / /_/ / /_/ /
\____/_____/\____/\____/_____/
    ___   __  ______  ______________  ____
   /   | / / / / __ \/  _/_  __/ __ \/ __ \
  / /| |/ / / / / / // /  / / / / / / /_/ /
 / ___ / /_/ / /_/ // /  / / / /_/ / _, _/
/_/  |_\____/_____/___/ /_/  \____/_/ |_|

//  cloud-infra-auditor  |||  scan → audit → report → cleanup  //
```

# cloud-infra-auditor

> AWS infrastructure auditing and cost optimization — from the terminal.

<p align="left">
  <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3.11+-blue.svg" alt="Python 3.11+"></a>
  <a href="./LICENSE"><img src="https://img.shields.io/badge/license-MIT-yellow.svg" alt="MIT License"></a>
  <img src="https://img.shields.io/badge/tests-pytest%20%2B%20moto-green.svg" alt="Tests">
  <img src="https://img.shields.io/badge/linter-ruff-purple.svg" alt="Ruff">
  <img src="https://img.shields.io/badge/formatter-black-black.svg" alt="Black">
  <img src="https://img.shields.io/badge/CI-github%20actions-blue.svg" alt="GitHub Actions">
  <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html"><img src="https://img.shields.io/badge/AWS-boto3-orange.svg" alt="Boto3"></a>
  <img src="https://img.shields.io/badge/release-v1.1.0-brightgreen.svg" alt="Release v1.1.0">
</p>

---

## Overview

Cloud accounts accumulate waste quietly — unattached volumes, orphaned Elastic IPs, instances running at 2% CPU for months. Most teams discover this on the billing page.

`cloud-infra-auditor` connects to your AWS account, scans for unused and underutilized resources, and surfaces findings as a Rich terminal report, a JSON export, or a CSV. It follows a deliberate **scan → analyze → report → review → cleanup** workflow. Nothing is modified unless you explicitly confirm it.

As of **v1.1.0**, multi-region scans run in parallel via a shared `ThreadPoolExecutor`-backed utility, cutting typical scan time from **60–65 seconds down to 8–10 seconds**.

---

## What's New in v1.1.0

- **Parallel multi-region scanning** — regions are scanned concurrently instead of sequentially.
- **Shared parallel scanning utility** (`auditor/utils/parallel.py`) — replaces duplicated concurrency logic across scanners.
- **Rich CLI progress bar** — live progress feedback while scans run.
- **Rich scan summary panel** (`auditor/ui/summary.py`) — a consolidated summary view after each scan.
- **Faster execution** — scan time improved from ~60–65 seconds to ~8–10 seconds on multi-region scans.
- **Cleaner CLI UX** — clearer output and more consistent formatting across commands.
- **Refactored concurrency logic** — duplicated threading code consolidated into one shared module.
- **General maintainability improvements** — clearer separation of concerns across the `auditor/` package.

---

## Features

| Category | Capability |
|---|---|
| **Scanning** | Unattached EBS volumes, unassociated Elastic IPs, underutilized EC2 instances |
| **Concurrency** | Parallel multi-region scanning via `ThreadPoolExecutor` |
| **Metrics** | CloudWatch-backed CPU utilization detection |
| **Multi-region** | Scan resources across multiple AWS regions in parallel |
| **CLI UX** | Rich terminal progress bar and scan summary panel |
| **Reporting** | Rich terminal tables, JSON export, CSV export |
| **Cleanup** | Dry-run preview; explicit `--execute` required — nothing deleted silently |
| **Retry** | Configurable retry logic for AWS API calls |
| **Testing** | Pytest + Moto + `unittest.mock` — no real AWS account needed |
| **CI** | GitHub Actions pipeline |
| **Packaging** | `pyproject.toml`, wheel + source dist, `cloud-auditor` entry point |

Summary:

✔ Parallel AWS region scanning
✔ Rich terminal progress
✔ Rich summary panel
✔ ThreadPoolExecutor concurrency
✔ JSON export
✔ CSV export
✔ Cleanup commands
✔ Multi-region support
✔ Installable CLI

---

## CLI Preview

### Help Command

![alt text](docs/images/image.png)

### Scan EBS

![alt text](docs/images/image-1.png)

### Scan EC2

![alt text](docs/images/image-2.png)

### Scan EIP

![alt text](docs/images/image-3.png)

### Cleanup Example

![alt text](docs/images/image-4.png)

### JSON or CSV Report

![alt text](docs/images/image-5.png)
![alt text](docs/images/image-6.png)

###

---

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                      CLI  (Typer)                           │
│        auditor/cli/main.py  ←  scan / report / cleanup      │
│        auditor/cli/scan.py  |  auditor/cli/report.py        │
│        auditor/cli/cleanup.py  |  auditor/cli/auth.py       │
│        auditor/cli/aws.py                                    │
└──────────────────────────┬──────────────────────────────────┘
                           │
          ┌────────────────▼────────────────┐
          │         AWS Layer               │
          │  auditor/aws/session.py         │
          │  auditor/aws/auth.py            │
          │  auditor/aws/regions.py         │
          │  auditor/aws/cloudwatch.py      │
          └────────────────┬────────────────┘
                           │
          ┌────────────────▼────────────────┐
          │   Parallel Scan Utility         │
          │  auditor/utils/parallel.py      │
          │  (ThreadPoolExecutor, shared     │
          │   across all scanners)          │
          └────────────────┬────────────────┘
                           │
       ┌───────────────────┼───────────────────┐
       │                   │                   │
┌──────▼──────┐   ┌────────▼──────┐   ┌───────▼──────┐
│ ebs_scanner │   │ eip_scanner   │   │ ec2_scanner  │
│     .py     │   │     .py       │   │     .py      │
└──────┬──────┘   └────────┬──────┘   └───────┬──────┘
       │                   │                   │
       └───────────────────▼───────────────────┘
              auditor/models/findings.py
              auditor/models/resource.py
                           │
          ┌────────────────▼────────────────┐
          │       Reports Layer             │
          │  report_generator.py            │
          │  report_transformer.py          │
          │  rich_formatter.py              │
          │  json_exporter.py               │
          │  csv_exporter.py                │
          └────────────────┬────────────────┘
                           │
          ┌────────────────▼────────────────┐
          │          UI Layer               │
          │  auditor/ui/summary.py          │
          │  (Rich progress bar + summary   │
          │   panel)                        │
          └────────────────┬────────────────┘
                           │
          ┌────────────────▼────────────────┐
          │       Cleanup Layer             │
          │  auditor/cleanup/dry_run.py     │
          │  auditor/cleanup/ebs_cleanup.py │
          │  auditor/cleanup/eip_cleanup.py │
          └─────────────────────────────────┘
```

---

## Project Structure

```
cloud-infra-auditor/
├── .github/                          # GitHub Actions CI workflows
├── auditor/
│   ├── aws/
│   │   ├── auth.py                   # AWS credential handling
│   │   ├── cloudwatch.py             # CloudWatch metric queries
│   │   ├── regions.py                # Region resolution
│   │   └── session.py                # Boto3 session management
│   ├── cli/
│   │   ├── main.py                   # Typer app entrypoint
│   │   ├── scan.py                   # scan command
│   │   ├── report.py                 # report command
│   │   ├── cleanup.py                # cleanup command
│   │   ├── auth.py                   # auth command
│   │   └── aws.py                    # aws command
│   ├── cleanup/
│   │   ├── ebs_cleanup.py            # EBS volume deletion
│   │   └── eip_cleanup.py            # Elastic IP release
│   ├── reports/
│   │   ├── report_generator.py       # Aggregates scan findings
│   │   ├── report_transformer.py     # Transforms findings for export
│   │   ├── rich_formatter.py         # Rich terminal output
│   │   ├── json_exporter.py          # JSON export
│   │   └── csv_exporter.py           # CSV export
│   ├── scanners/
│   │   ├── ebs_scanner.py            # Unattached EBS detection
│   │   ├── eip_scanner.py            # Unassociated EIP detection
│   │   └── ec2_scanner.py            # Underutilized EC2 detection
│   ├── ui/
│   │   └── summary.py                # Rich progress bar + scan summary panel
│   ├── utils/
│   │   ├── exceptions.py             # Custom exceptions
│   │   ├── helpers.py                # Shared utilities
│   │   ├── logger.py                 # Logging configuration
│   │   ├── retry.py                  # AWS API retry logic
│   │   └── parallel.py               # Shared ThreadPoolExecutor scanning utility
│   └── constants.py                  # Project-wide constants
├── tests/
│   ├── conftest.py
│   ├── test_cleanup.py
│   ├── test_csv_exporter.py
│   ├── test_ebs_scanner.py
│   ├── test_ec2_scanner.py
│   ├── test_eip_scanner.py
│   ├── test_json_exporter_.py
│   ├── test_regions.py
│   ├── test_report_generator.py
│   ├── test_report_transformer.py
│   └── test_session.py
├── config/                           # YAML thresholds and region defaults
├── docs/
├── audit_reports/                    # Runtime audit output (gitignored)
├── reports/                          # Generated exports (gitignored)
├── manual_testing/
├── dist/
│   ├── cloud_infra_auditor-1.1.0.tar.gz
│   └── cloud_infra_auditor-1.1.0-py3-none-any.whl
├── pyproject.toml
├── requirements.txt
├── requirements-dev.txt
└── LICENSE
```

---

## Getting Started

### Prerequisites

| Requirement | Version |
|---|:---:|
| Python | 3.11+ |
| Git | Latest |
| AWS CLI | v2 |
| pip | Latest |

```bash
python --version
pip --version
git --version
aws --version
```

### Clone

```bash
git clone https://github.com/harshil6-lab/cloud-infra-auditor.git
cd cloud-infra-auditor
```

### Virtual Environment

**Windows**
```bash
python -m venv venv
venv\Scripts\activate
```

**Linux / macOS**
```bash
python3 -m venv venv
source venv/bin/activate
```

### Install

```bash
# Runtime only
pip install -r requirements.txt

# Development (recommended)
pip install -e ".[dev]"
```

The development install includes Pytest, Moto, Ruff, and Black.

### Verify

```bash
cloud-auditor --help
```

Expected output:

```
Usage: cloud-auditor [OPTIONS] COMMAND [ARGS]...

  Cloud Infrastructure Auditor & Cost Optimizer
```

---

## AWS Configuration

```bash
# Configure credentials interactively
aws configure               

For detailed AWS credential setup and IAM configuration, see:
- **[AWS Configuration Guide](docs/AWS_CONFIG.md)**

# Verify authentication
aws sts get-caller-identity
```

**Minimum IAM permissions for scanning (read-only):**

```json
{
  "Effect": "Allow",
  "Action": [
    "ec2:Describe*",
    "cloudwatch:GetMetricStatistics"
  ],
  "Resource": "*"
}
```

Cleanup operations additionally require `ec2:ReleaseAddress` and `ec2:DeleteVolume`. These are only invoked after explicit `--execute` confirmation.

---

## Usage

### Scan

```bash
cloud-auditor scan ebs
cloud-auditor scan eip
cloud-auditor scan ec2
```

Multi-region scans now run in parallel by default, with a live Rich progress bar and a summary panel printed at the end of the scan.

### Report

```bash
cloud-auditor report export --format csv
cloud-auditor report export --format json
```

### Cleanup

```bash
# Preview — no changes made
cloud-auditor cleanup ebs --dry-run
cloud-auditor cleanup eip --dry-run

# Execute after confirmation
cloud-auditor cleanup ebs --execute
cloud-auditor cleanup eip --execute
```

### Auth

```bash
cloud-auditor auth identity
cloud-auditor auth profiles
```

### AWS

```bash
cloud-auditor aws regions
```

### Help

```bash
cloud-auditor --help
cloud-auditor scan --help
cloud-auditor cleanup --help
cloud-auditor report --help
```

---

## Testing

All tests run against Moto-mocked AWS services. No live AWS account or credentials required.

```bash
# Full test suite
pytest

# With terminal coverage summary
pytest --cov=auditor --cov-report=term-missing

# HTML coverage report
pytest --cov=auditor --cov-report=html
# Open htmlcov/index.html in your browser to inspect line-by-line code coverage.
```

**Test files:**

| File | Covers |
|---|---|
| `test_ebs_scanner.py` | EBS unattached volume detection |
| `test_eip_scanner.py` | Elastic IP unassociated detection |
| `test_ec2_scanner.py` | EC2 underutilization detection |
| `test_cleanup.py` | Dry-run and cleanup execution |
| `test_report_generator.py` | Report aggregation |
| `test_report_transformer.py` | Report transformation |
| `test_json_exporter_.py` | JSON export |
| `test_csv_exporter.py` | CSV export |
| `test_session.py` | Boto3 session management |
| `test_regions.py` | Region resolution |
| `conftest.py` | Shared fixtures |

---

## Code Quality

```bash
black .
ruff check .
```

---

## Build

```bash
pip install build
python -m build
```

Artifacts generated in `dist/`:

```
dist/
├── cloud_infra_auditor-1.1.0.tar.gz
└── cloud_infra_auditor-1.1.0-py3-none-any.whl
```

---

## Technology Stack

| Category | Technology | Version |
|---|---|:---:|
| Language | Python | 3.11+ |
| CLI Framework | Typer | 0.16+ |
| Cloud SDK | Boto3 | 1.40+ |
| Terminal UI | Rich | 14.0+ |
| Concurrency | ThreadPoolExecutor | stdlib |
| Testing | Pytest | 9.1+ |
| AWS Mocking | Moto | 5.1+ |
| Mock Library | unittest.mock | stdlib |
| Formatter | Black | 25.1+ |
| Linter | Ruff | 0.12+ |
| Packaging | setuptools + build | 68+ |
| Distribution | Wheel + source dist | PEP 427 |
| CI | GitHub Actions | — |
| Config Format | TOML (`pyproject.toml`) | PEP 621 |
| AWS Auth | AWS CLI | v2 |
| Cloud Services | EC2, EBS, Elastic IP, CloudWatch | AWS |

---

## Design Decisions

**Why Typer?**
Type-annotated commands, zero boilerplate, auto-generated `--help` that stays accurate without maintenance.

**Why Rich?**
Tables, progress bars, and summary panels that degrade cleanly in CI environments. No custom rendering code.

**Why a shared `auditor/utils/parallel.py`?**
Each scanner originally implemented its own threading logic for multi-region scans. Consolidating this into a single `ThreadPoolExecutor`-backed utility removed duplicated code and made concurrency behavior consistent and independently testable across all scanners.

**Why `report_transformer.py` separate from `report_generator.py`?**
Generation and transformation are distinct responsibilities. `report_generator.py` aggregates raw findings; `report_transformer.py` shapes them for a specific output format. Keeping these separate makes each independently testable.

**Why dry-run by default?**
Audit tools carry asymmetric risk. Scanning is always safe; deletion is not. The default posture is read-only. Cleanup requires `--execute` explicitly.

**Why Moto for tests?**
Real AWS calls in tests are slow, costly, and non-deterministic. Moto provides accurate service simulation — tests run offline with no live account.

---

## Security

- **Read-only by default.** `scan` requires only `Describe*` and `GetMetricStatistics`. No write permissions needed.
- **Explicit execution gate.** `--dry-run` shows what would be affected; `--execute` prompts per resource before acting. Cleanup never runs from a scan.
- **No credential storage.** The tool delegates entirely to boto3's credential chain — no secrets are read, copied, or logged by the application.
- **Least privilege.** Minimum IAM policy for read-only audit is documented above. Write permissions should be granted only when cleanup is required.

---

## Roadmap

```
[x] AWS session management and credential resolution
[x] EBS unattached volume scanner
[x] Elastic IP unassociated scanner
[x] EC2 underutilization scanner (CloudWatch-backed)
[x] Rich terminal formatter
[x] JSON and CSV export
[x] Report generator and transformer
[x] Dry-run and confirmation-gated cleanup
[x] Pytest + Moto test coverage (11 tests)
[x] GitHub Actions CI
[x] pyproject.toml packaging, wheel + source dist
[x] Parallel scanning via ThreadPoolExecutor
[x] Rich CLI progress bar and scan summary panel
[ ] S3 lifecycle analysis
[ ] Multi-account support via assume-role
[ ] HTML report output
[ ] Slack / PagerDuty alerting integration
```

---

## Author

**Harshil Kalsariya**
[github.com/harshil6-lab](https://github.com/harshil6-lab)

**Sravya Maddipati**
[github.com/sravya-77](https://github.com/sravya-77)

---

## License

[MIT](./LICENSE)
