Metadata-Version: 2.4
Name: s3-pypi-server
Version: 0.3.1
Summary: Deploy and manage a private PyPI server on AWS using S3, API Gateway, CloudFront, and Lambda. Supports PEP 503 package hosting, twine uploads, LDAP/AD and API key authentication, KMS encryption, and VPC placement — all from a single CLI.
Author-email: Topaz Bott <topaz@topazhome.net>
License-Expression: MIT
Project-URL: Homepage, https://github.com/tmb28054/S3-Pypi-Server
Project-URL: Repository, https://github.com/tmb28054/S3-Pypi-Server
Project-URL: Documentation, https://github.com/tmb28054/S3-Pypi-Server/tree/main/docs
Project-URL: Changelog, https://github.com/tmb28054/S3-Pypi-Server/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/tmb28054/S3-Pypi-Server/issues
Keywords: pypi,aws,s3,private,package,repository,cloudformation
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: System :: Software Distribution
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: boto3
Requires-Dist: keyring
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: hypothesis; extra == "test"
Requires-Dist: moto[all]; extra == "test"
Requires-Dist: pylint; extra == "test"
Requires-Dist: bandit; extra == "test"
Requires-Dist: cfn-lint; extra == "test"
Requires-Dist: yamllint; extra == "test"
Dynamic: license-file

# s3-pypi-server

A private PyPI server backed by AWS S3, API Gateway, and CloudFront. Upload Python packages to S3 and install them with `pip` using a PEP 503 compliant simple repository interface. Supports optional KMS encryption, API Gateway authorization (LDAP/AD + API keys), and VPC placement for Lambda functions.

## Architecture

```
pip install ──▶ CloudFront ──▶ API Gateway ──▶ S3 Bucket
                 (cache)        (REST API)     (storage)
                                    │
                              ┌─────┴─────┐
                              │ Authorizer │ (optional)
                              │  Lambda    │
                              └─────┬─────┘
                                    │
                         ┌──────────┼──────────┐
                         ▼                     ▼
                    DynamoDB              Secrets Manager
                   (API keys)           (LDAP config)
```

- **S3** stores distribution files under `packages/{name}/` and HTML index pages under `simple/`.
- **API Gateway** maps `/simple/` URL paths to S3 objects, serving HTML indexes and binary downloads.
- **CloudFront** caches responses and provides HTTPS with TLS 1.2+.
- **Lambda Authorizer** (optional) validates Bearer tokens against DynamoDB or Basic Auth against LDAP/AD.
- **CLI** uploads packages to S3, manages API keys, and configures LDAP secrets.

See [docs/design.md](docs/design.md) for detailed architecture documentation.

## Quickstart

### 1. Deploy the infrastructure

```bash
s3pypi deploy --stack-name my-pypi
```

This deploys the CloudFormation stack (S3 bucket, API Gateway, CloudFront distribution) and automatically saves the stack outputs to your CLI config. The bucket and table names are randomly generated by CloudFormation.

### 2. Upload a package

```bash
# Build your package
python -m build

# Save your defaults (one-time setup)
s3pypi configure --bucket <bucket-name> --cloudfront-distribution-id <distribution-id>

# Upload to your private PyPI
s3pypi upload dist/my_package-1.0.0-py3-none-any.whl
```

### 3. Install from your private PyPI

```bash
pip install my-package --index-url https://<cloudfront-domain>/simple/
```

### Deploy with security features

```bash
s3pypi deploy --stack-name my-pypi \
  --enable-kms-encryption true \
  --enable-authorizer true \
  --vpc-id vpc-abc123 \
  --subnet-ids subnet-111,subnet-222
```

After deploying with the authorizer enabled, configure LDAP and create API keys:

```bash
# Configure LDAP
s3pypi configure-ldap \
  --secret-arn <from stack output> \
  --host ldap.example.com \
  --bind-user "cn=admin,dc=example,dc=com" \
  --bind-password "secret" \
  --entitlement-group "cn=pypi-users,ou=groups,dc=example,dc=com"

# Create an API key for CI/CD
s3pypi apikey --table-name <from stack output> create --description "CI pipeline"
```

## CLI Usage

```
s3pypi deploy --stack-name NAME [--profile P] [--region R] [Key=Value ...]
s3pypi configure [--bucket B] [--cloudfront-distribution-id ID] [--api-key-table-name T] [--ldap-secret-arn ARN]
s3pypi upload <dist_file> [--bucket B] [--cloudfront-distribution-id ID]
s3pypi apikey [--table-name T] <create|list|get|delete|update> [options]
s3pypi configure-ldap --host H --bind-user U --bind-password P --entitlement-group G [--write-entitlement-group WG] [--secret-arn ARN]
```

### deploy

Deploy or update the CloudFormation stack. Outputs are automatically saved to the CLI config.

When run without `--stack-name`, enters interactive mode and prompts for all values (press Enter to accept template defaults):

```bash
$ s3pypi deploy
Stack name (required): my-pypi
AWS region [us-east-1]:
AWS profile []:
Stack name prefix [s3-pypi]:
Cache TTL (seconds) [300]:
Custom domain name:
ACM certificate ARN:
Enable KMS encryption (true/false) [false]: true
Enable authorizer (true/false) [false]: true
Subnet IDs (comma-separated):
VPC ID:
Deploying stack 'my-pypi' in us-east-1...
```

Or pass flags directly:

```bash
s3pypi deploy --stack-name my-pypi --enable-authorizer true --enable-kms-encryption true
```

| Argument | Required | Description |
|---|---|---|
| `--stack-name` | No* | CloudFormation stack name (prompted if not provided) |
| `--profile` | No | AWS CLI / boto3 named profile |
| `--region` | No | AWS region (default: `us-east-1`) |
| `--stack-name-prefix` | No | Resource naming prefix (default: `s3-pypi`) |
| `--cache-ttl` | No | CloudFront cache TTL in seconds (default: `300`) |
| `--domain-name` | No | Custom domain for CloudFront |
| `--acm-certificate-arn` | No | ACM certificate ARN for custom domain |
| `--enable-kms-encryption` | No | `true` or `false` (default: `false`) |
| `--enable-authorizer` | No | `true` or `false` (default: `false`) |
| `--subnet-ids` | No | Comma-separated subnet IDs for VPC |
| `--vpc-id` | No | VPC ID for Lambda placement |

\* If not provided, interactive mode prompts for all values.

### configure

Save default settings so you don't need to pass flags on every command. Settings are stored in `~/.s3pypi/config.json`.

When run without any flags, `configure` enters interactive mode and prompts for each value:

```bash
$ s3pypi configure
S3 bucket name: my-pypi-bucket
CloudFront distribution ID: E1234567890
DynamoDB API key table name: s3-pypi-api-keys
Secrets Manager LDAP secret ARN: arn:aws:secretsmanager:us-east-1:123:secret:s3-pypi-ldap-config
```

Current values are shown in brackets — press Enter to keep them unchanged:

```bash
$ s3pypi configure
S3 bucket name [my-pypi-bucket]:
CloudFront distribution ID [E1234567890]: E9999999999
DynamoDB API key table name [s3-pypi-api-keys]:
Secrets Manager LDAP secret ARN [arn:...]:
```

You can also pass flags directly to skip prompts:

```bash
s3pypi configure \
  --bucket my-pypi-bucket \
  --cloudfront-distribution-id E1234567890 \
  --api-key-table-name s3-pypi-api-keys \
  --ldap-secret-arn arn:aws:secretsmanager:us-east-1:123:secret:s3-pypi-ldap-config
```

### upload

| Argument | Required | Description |
|---|---|---|
| `dist_file` | Yes | Path to `.whl` or `.tar.gz` distribution file |
| `--bucket` | No* | S3 bucket name. Falls back to configured value. |
| `--cloudfront-distribution-id` | No | CloudFront distribution ID to invalidate. |

\* Required if not previously saved via `s3pypi configure`.

### apikey

Manage API keys in the DynamoDB table. Keys use the `__token__` username convention (same as PyPI).

```bash
# Create a read-only key
s3pypi apikey create --description "CI pipeline"

# Create a key with write access (for twine uploads)
s3pypi apikey create --description "Publisher" --access read/write

# List all keys
s3pypi apikey list

# Get details of a key
s3pypi apikey get <key-value>

# Update access level
s3pypi apikey update <key-value> --access read/write

# Delete a key
s3pypi apikey delete <key-value>
```

| Flag | Required | Description |
|---|---|---|
| `--table-name` | No* | DynamoDB table name. Falls back to configured value. |
| `--description` | No | Description for `create` action. |
| `--access` | No | Access level: `read` (default) or `read/write`. Used with `create` and `update`. |

### configure-ldap

Set or update the LDAP/AD configuration in Secrets Manager.

```bash
s3pypi configure-ldap \
  --host ldap.example.com \
  --bind-user "cn=admin,dc=example,dc=com" \
  --bind-password "secret" \
  --entitlement-group "cn=pypi-readers,ou=groups,dc=example,dc=com" \
  --write-entitlement-group "cn=pypi-writers,ou=groups,dc=example,dc=com"
```

| Flag | Required | Description |
|---|---|---|
| `--host` | Yes | LDAP/AD server hostname |
| `--bind-user` | Yes | DN or username for LDAP bind |
| `--bind-password` | Yes | Password for LDAP bind |
| `--entitlement-group` | Yes | DN of the read entitlement group |
| `--write-entitlement-group` | No | DN of the write entitlement group (for uploads) |
| `--secret-arn` | No* | Secrets Manager ARN. Falls back to configured value. |

Exit codes: `0` success, `1` runtime error, `2` argument error.

## CloudFormation Parameters

| Parameter | Default | Description |
|---|---|---|
| `StackNamePrefix` | `s3-pypi` | Prefix for resource naming |
| `CacheTTL` | `300` | CloudFront default cache TTL in seconds |
| `DomainName` | *(empty)* | Optional custom domain (e.g. `pypi.example.org`) |
| `AcmCertificateArn` | *(empty)* | ACM certificate ARN for the custom domain |
| `EnableKMSEncryption` | `false` | Create a CMK and encrypt all resources at rest |
| `EnableAuthorizer` | `false` | Add Lambda authorizer to API Gateway |
| `SubnetIds` | *(empty)* | Comma-separated subnet IDs for Lambda VPC placement |
| `VpcId` | *(empty)* | VPC ID for Lambda VPC placement |

### Custom domain

To use a custom domain like `pypi.example.org`:

1. Request (or import) an ACM certificate in **us-east-1** for your domain.
2. Deploy with both parameters:

```bash
./deploy.sh my-pypi DomainName=pypi.example.org AcmCertificateArn=arn:aws:acm:us-east-1:123456789012:certificate/abc-123
```

3. Create a CNAME or alias DNS record pointing `pypi.example.org` to the CloudFront distribution domain.

### VPC deployment

When your LDAP/AD server is on a private network, deploy Lambdas inside the VPC:

```bash
s3pypi deploy --stack-name my-pypi \
  --enable-authorizer true \
  --vpc-id vpc-abc123 \
  --subnet-ids subnet-111,subnet-222
```

This creates a Security Group allowing all outbound traffic and places the authorizer Lambda in the specified subnets.

## Authentication

When `EnableAuthorizer=true`, the API Gateway requires authentication on all routes. Authentication uses Basic Auth with two modes:

### API keys (`__token__` username)

For CI/CD pipelines and automated systems, use `__token__` as the username and the API key as the password (same convention as PyPI):

```bash
pip install my-package \
  --index-url https://__token__:<api-key>@<endpoint>/simple/
```

### LDAP/AD (corporate credentials)

For developers using corporate credentials:

```bash
pip install my-package \
  --index-url https://user:password@<endpoint>/simple/
```

### Publishing with twine

To upload packages using twine, create an API key with `read/write` access and configure `~/.pypirc`:

```ini
[distutils]
index-servers = private

[private]
repository = https://<endpoint>/simple/
username = __token__
password = <api-key-with-readwrite-access>
```

Then upload:

```bash
twine upload --repository private dist/*
```

### Access levels

- **read** — Can install packages (GET requests only)
- **read/write** — Can install and upload packages (GET and POST requests)

LDAP users in `entitlement_group` get read access. Users in `write_entitlement_group` get read/write access.

## Development

### Setup

```bash
python -m venv .venv
source .venv/bin/activate
pip install -e ".[test]"
```

### Running tests

```bash
pytest                                    # Full test suite
pytest --cov=s3pypi --cov-fail-under=80   # With coverage
pytest -m smoke                           # Smoke tests only
```

### Linting

```bash
pylint s3pypi/
bandit -r s3pypi/
```

## License

MIT — see [LICENSE](LICENSE) for details.
