Metadata-Version: 2.4
Name: heaserver-accounts
Version: 1.20.0
Summary: Manages account information
Home-page: https://code.hci.utah.edu
Author: Comprehensive Oncology Data and Engineering Shared Resource (CODE), Huntsman Cancer Institute, Salt Lake City, UT
Author-email: Andrew.Post@hci.utah.edu
License: Apache License 2.0
Keywords: heaserver-accounts,microservice,healthcare,cancer,research,informatics
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Framework :: AsyncIO
Classifier: Environment :: Web Environment
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: heaserver~=1.71.1
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# HEA Server AWS Accounts Microservice
[Comprehensive Oncology Data and Engineering Shared Resource (CODE)](https://code.hci.utah.edu), [Huntsman Cancer Institute](https://healthcare.utah.edu/huntsmancancerinstitute/),
Salt Lake City, UT

The HEA Server AWS Accounts Microservice is It manages the AWS account information details.


## Version 1.20.0
* Bumped heaserver version to 1.71.1.
* /accounts and /accounts/{id} now reach out to other microservices that are registered as resources to fetch account
  subclasses.
* Restored returning projects in search results as projects. Requires heaserver-folders-aws-s3 version 1.24.0 or
  greater for its /volumes/{volume_id}/awss3projects/filter endpoint.

## Version 1.19.1
* Use time.monotonic for determining how long something ran.
* Bumped heaserver version to 1.64.0.

## Version 1.19.0
* Removed the /awsaccounts/{id}/search/{id} endpoint.
* Cleaned up the search code.

## Version 1.18.17
* More improved exception handling in heaserver.account.service.search_awsaccounts.

## Version 1.18.16
* Improved exception handling in heaserver.account.service.search_awsaccounts.

## Version 1.18.15
* Ensure context_dependent_object_path is populated in the search results.

## Version 1.18.14
* Ensure id_in_index is set when receiving s3 events to insert into the OpenSearch index.

## Version 1.18.13
* Bumped heaserver version to 1.62.9.
* Update existing index records again.

## Version 1.18.12
* Bumped heaserver version to 1.62.7.
* Refresh the OpenSearch index after an insert.

## Version 1.18.11
* Bumped heaserver version to 1.62.6.

## Version 1.18.10
* Bumped heaserver version to 1.62.5.

## Version 1.18.9
* Added logging.

## Version 1.18.8
* Bumped heaserver version to 1.62.4.
* Prevent insertion of duplicate search entries.

## Version 1.18.7
* Replaced incorrect self link for the search results endpoint with hea-actual.

## Version 1.18.6
* Bumped heaserver version to 1.62.3.

## Version 1.18.5
* Bumped heaserver version to 1.62.2.

## Version 1.18.4
* Exit s3event processing when there are no more messages to process.

## Version 1.18.3
* Bumped heaserver version to 1.61.3.
* Fixed volume_id not defined error.

## Version 1.18.2
* Fixed sporadic tuple unpacking error.

## Version 1.18.1
* Bumped heaserver version to 1.61.1.

## Version 1.18.0
* Bumped heaserver version to 1.61.0.
* Use the /peopleviews and /groupviews endpoints for dropdowns.

## Version 1.17.4
* Improved logging when there is an error in the internal/s3event endpoint.

## Version 1.17.3
* Bumped heaserver version to 1.59.0.

## Version 1.17.2
* Bumped heaserver version to 1.58.0.
* Don't treat limit query parameter as a filter parameter.

## Version 1.17.1
* Don't treat return_attr query parameter as a filter parameter.

## Version 1.17.0
* Bumped heaserver version to 1.55.0.
* Sorting account views now works.
* Dropdowns are now sorted.
* Request only required attributes for form template dropdowns.

## Version 1.16.0
* Changed AWS account display name to AWS {account number}.
* Bumped heaserver version to 1.51.0.

## Version 1.15.1
* Bumped heaserver version to 1.50.0.

## Version 1.15.0
* Bumped heaserver version to 1.49.0.
* Allow filtering account view queries by their attributes.

## Version 1.14.0
* Bumped heaserver version to 1.48.0.
* Added support for encrypted RabbitMQ password.

## Version 1.13.1
* Bumped heaserver version to 1.47.1 for a bug fix.

## Version 1.13.0
* Bumped heaserver version to 1.45.1.
* New startup.py module to initialize logging before importing any third-party libraries.
* New optional -e/--env command-line argument for setting the runtime environment (development, staging, or
  production). Logging is now configured based on this setting, unless the older -l/--logging argument is provided. The
  default value is development.
* Logs are now scrubbed.
* Added log files to .gitignore.

## Version 1.12.7
* Catching all s3 events in else that aren't permanent delete or delete marker created so they recorded in opensearch.

## Version 1.12.6
* Bug fix in logic for crud operations in opensearch based off of s3 events.

## Version 1.12.5
* Bumped heaserver version to 1.43.1 to address fixes for search.

## Version 1.12.4
* Document which REST APIs require an Authorization header.
* Bumped heaserver version to 1.43.0.

## Version 1.12.3
* Bumped heaserver version to 1.41.2 to address issue getting AWS account-level collaborator info.

## Version 1.12.2
* Makes search more reliable and now enforcing the current version to be the most recent event from sqs.

## Version 1.12.1
* Write a type property for AWS account collaborators in the AWS account properties template.

## Version 1.12.0
* Bumped heaserver version to 1.41.1.
* Added the new collaborator info to the CJ templates.
* Silenced repeating log message when OpenSearch is not configured.

## Version 1.11.1
* Changed the activity code OpenSearch index updates to hea-update-part to reflect that it's part of an object update,
  and it's a lot of detail that is probably not very interesting to users.
* Bumped heaserver version to 1.37.2 and heaobject version to 1.28.1.

## Version 1.11.0
* Accounts now sends events to AWS Folders to allow for handling of those delete events to update opensearch.

## Version 1.10.3
* Ask AWS for permission to create buckets when deciding whether or not to offer bucket creation in an account.

## Version 1.10.2
* Give all users VIEWER permissions to accounts because there's no way to create nor edit nor delete them yet.

## Version 1.10.1
* Bumped heaserver version to 1.32.0.
* Added instructions and require cb-* name when creating a new bucket (cb-* name is not enforced server-side, it's just
  passed to the client to enforce).

## Version 1.10.0
* We now mark account objects with the new hea-container, hea-self-container, and hea-actual-container rel values.

## Version 1.9.4
* Bumped heaserver version to 1.30.1 to address a potential issue with erroneously assigning CREATOR permissions.

## Version 1.9.3
* Bumped heaserver version to 1.30.0.
* Gracefully handle when S3 event listening is not configured.

## Version 1.9.2
* Fixed context_dependent_object_path for paths with projects and optimized speed for generating paths.
* Search now filters out aws search items with delete markers

## Version 1.9.0
* Added support for group permissions.

## Version 1.8.5
* Fixed search error when bucket_items called not having authorization header.
* Errors are now logged for sqs message ingestion.

## Version 1.8.4
* Fix for handling standard s3 event notification messages and ones that pass through sns.

## Version 1.8.3
* Removed search item from context menu.

## Version 1.8.2
* Upgraded heaserver dependency for bug fix getting temporary AWS credentials.

## Version 1.8.1
* Upgrading HEA Server dependency for bug fixes in Search.

## Version 1.8.0
* Added context path for all search items for navigating functionality.

## Version 1.7.3
* Minor bug fix for handling None with 'in' Operator.

## Version 1.7.1
* Background task now sends fewer sqs request due to doing bulk delete.
* Better exception handling, aborting background task if issues with message ingestion.

## Version 1.7.0
* Adds the scheduled background task for pulling from our sqs queue and ingesting into opensearch
* Improves search endpoint to include permissions and includes values actual_object_*

## Version 1.6.0
* Improved error message when call to create a new bucket fails.
* Repointed action to calculate usage to volumes/{volume_id}/awss3storage. Requires heaserver-folders-aws-s3 version
  1.9.0 or greater.

## Version 1.5.1
* Performance improvements.

## Version 1.5.0
* Added support for Python 3.12.

## Version 1.4.0
* Removed integration tests because they all overlapped with the unit tests.
* Accept the data query parameter for get requests for a speed boost.

## Version 1.3.9
* Dependency upgrades for compatibility with heaserver-keychain 1.5.0.

## Version 1.3.8
* Fixed another /accounts regression when no account ids are passed as query parameters.

## Version 1.3.7
* Stop returning all accounts with an accessible volume; only return the requested accounts.

## Version 1.3.6
* When retrieving /accounts, skip volumes with a null account_id. Fixes crash retrieving accounts.

## Version 1.3.5
* Pull /accounts from volume data rather than trying to get account info from AWS. Helps with performance for users who
have access to lots of accounts.

## Version 1.3.4
* Caching optimizations.

## Version 1.3.3
* Fixed new bucket link.

## Version 1.3.2
* Fixed a caching issue when listing accounts.
* Additional input validation.

## Version 1.3.1
* Avoid timeouts loading accounts, which sporadically caused accounts not to be returned.

## Version 1.3.0
* Present accurate account permissions.
* Added caching for getting accounts list.

## Version 1.2.0
* New /accounts endpoint that returns heaobject.account.AccountView objects.

## Version 1.1.3
* Another attempt at fixing the crash regression.

## Version 1.1.2
* Fixed crash regression.

## Version 1.1.1
* Fixed new bucket form submission.

## Version 1.1.0
* Removed the PUT and DELETE account calls because neither works.

## Version 1.0.8
* Addressed occasional slowdown getting one account.

## Version 1.0.7
* Addressed occasional slowdown getting accounts.

## Version 1.0.6
* Display type display name in properties card, and return the type display name from GET calls.

## Version 1.0.5
* Improved performance getting accounts.

## Version 1.0.4
* Corrected caching issue.

## Version 1.0.3
* Improved error handling when the user lacks authorization for some AWS account information.

## Version 1.0.2
* Avoid a 500 error when retrieving accounts when attempting to access a suspended account.
* AWS account info requests return more complete information.
* Omit shares from the properties template.

## Version 1.0.1
* Improved performance.

## Version 1
Initial release.

## Runtime requirements

* Python 3.10, 3.11, or 3.12.

## Account builder Lambda

[`lambdas/AccountBuilderLambda.py`](lambdas/AccountBuilderLambda.py) is an AWS Lambda function for bootstrapping an AWS
Organizations member account and its HEA S3 storage. It can create an organization member account, or configure an
existing member account, and then:

* configures the member account's OIDC provider and the `aws-Manager.Role` and `aws-Member.Role` roles;
* creates or updates an S3 bucket with encryption, public-access blocking, versioning, inventory, lifecycle rules, and
  object-change notifications;
* sends object notifications directly to an SQS queue in `us-east-1`, or through an SNS topic in other regions; and
* creates EventBridge rules in the four supported regions (`us-east-1`, `us-east-2`, `us-west-1`, and `us-west-2`) that
  forward S3 bucket creation and deletion events to an event bus in the organization management account.

This is an administrative bootstrap function, not part of the accounts microservice process. Deploy it in the AWS
Organizations **management account**. The function assumes `OrganizationAccountAccessRole` in each member account and
uses the Lambda execution role's temporary credentials. Do not configure `AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY`, or `AWS_SESSION_TOKEN` yourself; Lambda supplies these reserved environment variables.

### Prerequisites

Before installing the function:

1. Create the destination S3 inventory bucket in the management account.
2. Create the two customer-managed policies named `aws-Manager.RolePolicy` and `aws-Member.RolePolicy` in the
   management account. The Lambda copies their current default policy documents into each member account.
3. Create an SQS queue in `us-east-1`. For buckets outside `us-east-1`, also create an SNS topic in the bucket's region
   and arrange for that topic to deliver to the SQS queue used by HEA's S3-event ingestion.
4. Create the destination EventBridge custom event bus in `us-east-1` in the management account. Add an event-bus
   resource policy that accepts events from the member accounts; the Lambda creates an `EventBridgeToTargetBusRole`
   identity policy in each member account. Enable CloudTrail management events for S3 if the EventBridge rules are
   expected to receive `CreateBucket` and `DeleteBucket` API calls.
5. For an existing member account, ensure that `OrganizationAccountAccessRole` exists, trusts the management account,
   and has enough permissions to administer IAM, S3, and EventBridge. AWS Organizations creates this role automatically
   when this Lambda creates a new account. Service control policies can still restrict what the role can do.

The Lambda execution role needs the standard CloudWatch Logs permissions plus permission to perform the following
management-account operations. Restrict the resource ARNs where the AWS API supports it:

```text
organizations:CreateAccount
organizations:DescribeAccount
organizations:DescribeCreateAccountStatus
sts:GetCallerIdentity
sts:AssumeRole                           on arn:aws:iam::*:role/OrganizationAccountAccessRole
iam:GetPolicy
iam:GetPolicyVersion                    on the two source managed policies
s3:GetBucketPolicy
s3:PutBucketPolicy                      on the inventory bucket
sqs:GetQueueAttributes
sqs:SetQueueAttributes                  on the us-east-1 event queue
sns:GetTopicAttributes
sns:SetTopicAttributes                  on the regional event topics
```

AWS documents the special considerations and minimum permission for
[creating an Organizations member account](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_create.html).

### Build and install

The source imports `jwt`, which is supplied by the `PyJWT` distribution and is not included in the Lambda Python
runtime. Build a zip archive with the dependency and put both the dependency and `AccountBuilderLambda.py` at the root
of the archive, as required for a
[Python Lambda zip deployment](https://docs.aws.amazon.com/lambda/latest/dg/python-package.html):

Linux:

```bash
mkdir -p build/account-builder-lambda
python -m pip install --target build/account-builder-lambda PyJWT
cp lambdas/AccountBuilderLambda.py build/account-builder-lambda/
(cd build/account-builder-lambda && zip -r ../account-builder-lambda.zip .)
```

Windows PowerShell:

```powershell
New-Item -ItemType Directory -Force build\account-builder-lambda
py -m pip install --target build\account-builder-lambda PyJWT
Copy-Item lambdas\AccountBuilderLambda.py build\account-builder-lambda\
Compress-Archive -Path build\account-builder-lambda\* `
    -DestinationPath build\account-builder-lambda.zip -Force
```

In IAM, create an execution role whose trust policy allows `lambda.amazonaws.com` to assume it. Attach
`AWSLambdaBasicExecutionRole` for logging and a customer-managed policy containing the permissions listed above. Then
create the function in `us-east-1`, replacing the example account ID and role name. The function constructs its SQS
queue URL and destination EventBridge ARN in `us-east-1`:

Linux:

```bash
aws lambda create-function \
    --region us-east-1 \
    --function-name AccountBuilderLambda \
    --runtime python3.12 \
    --architectures x86_64 \
    --handler AccountBuilderLambda.lambda_handler \
    --role arn:aws:iam::111122223333:role/hea-account-builder-lambda \
    --zip-file fileb://build/account-builder-lambda.zip \
    --timeout 900 \
    --memory-size 256
```

Windows PowerShell:

```powershell
aws lambda create-function `
    --region us-east-1 `
    --function-name AccountBuilderLambda `
    --runtime python3.12 `
    --architectures x86_64 `
    --handler AccountBuilderLambda.lambda_handler `
    --role arn:aws:iam::111122223333:role/hea-account-builder-lambda `
    --zip-file fileb://build/account-builder-lambda.zip `
    --timeout 900 `
    --memory-size 256
```

The 15-minute timeout accommodates the polling performed while AWS Organizations creates an account and while the new
account's S3 service and `OrganizationAccountAccessRole` become available. If the function already exists, deploy a
new archive with:

Linux:

```bash
aws lambda update-function-code \
    --region us-east-1 \
    --function-name AccountBuilderLambda \
    --zip-file fileb://build/account-builder-lambda.zip
```

Windows PowerShell:

```powershell
aws lambda update-function-code `
    --region us-east-1 `
    --function-name AccountBuilderLambda `
    --zip-file fileb://build/account-builder-lambda.zip
```

The equivalent console installation is: create a Python 3.12 function from scratch, select the execution role described
above, upload `build/account-builder-lambda.zip`, set the handler to
`AccountBuilderLambda.lambda_handler`, and set the timeout to 15 minutes. See AWS's
[Python handler naming rules](https://docs.aws.amazon.com/lambda/latest/dg/python-handler.html#python-handler-naming)
for why the module and function names form that handler value.

### Invoke the function

Invoke the function synchronously with a JSON object. These keys must always be present because the handler indexes them
directly: `token`, `accountid`, `accountname`, `bucketname`, `accountemail`, `oidcproviderurl`, and `oidcthumbprint`.
Pass an empty `accountid` to create a new member account; in that case `accountemail` must be a unique AWS account email.
If `accountid` is populated, the existing account is configured and `accountemail` is ignored.

The normally required configuration keys are `inventorybucket`, `sqsname`, `snsname`, `region`, and
`eventbridgename`. `snsname` is only used outside `us-east-1`. The `orgid` key is currently accepted but unused.
Bucket names must comply with the AWS S3 bucket naming rules.

For example, the following event configures an existing account and creates or updates its bucket:

```json
{
    "token": "<trusted JWT containing a sub claim>",
    "accountid": "444455556666",
    "accountname": "research-a",
    "accountemail": "",
    "bucketname": "cb-research-a-data",
    "oidcproviderurl": "id.example.org/realms/hea",
    "oidcthumbprint": "0123456789abcdef0123456789abcdef01234567",
    "mode": "create",
    "exclude": "",
    "lifecycletype": "moderate",
    "inventorybucket": "hea-inventory-reports",
    "sqsname": "hea-s3-events",
    "snsname": "",
    "region": "us-east-1",
    "eventbridgename": "hea-s3-bucket-events"
}
```

Save the event as `account-builder-event.json` and invoke it:

Linux:

```bash
aws lambda invoke \
    --region us-east-1 \
    --function-name AccountBuilderLambda \
    --cli-binary-format raw-in-base64-out \
    --payload file://account-builder-event.json \
    account-builder-response.json
```

Windows PowerShell:

```powershell
aws lambda invoke `
    --region us-east-1 `
    --function-name AccountBuilderLambda `
    --cli-binary-format raw-in-base64-out `
    --payload file://account-builder-event.json `
    account-builder-response.json
```

Supported modes are `create`, `update`, `delete`, and `delete_all`. Both `create` and `update` run the same
configuration path. `delete` removes an empty bucket; `delete_all` first deletes its objects and versions. Supported
`lifecycletype` values are `aggressive`, `moderate`, and `minimal`; the default for a newly created bucket is
`aggressive`. Set `exclude` to `lifecycles` to avoid applying the requested archival lifecycle to an existing bucket.

The function decodes `token` with signature verification disabled and uses only its `sub` claim as the STS role-session
name. Consequently, do not expose this Lambda through a public Function URL or unauthenticated API Gateway. Grant
`lambda:InvokeFunction` only to the trusted HEA caller that has already authenticated the token. A successful invocation
returns `statusCode` 200 and a JSON string in `body`; handled errors return `statusCode` 500 in the same envelope.

## Account creation controller

[`scripts/AccountCreationController.py`](scripts/AccountCreationController.py) is a local command-line batch client for
the account builder Lambda. It obtains a Keycloak access token with the resource-owner password grant, uses the token to
assume an organization-administrator role through `sts:AssumeRoleWithWebIdentity`, and invokes the
`AccountBuilderLambda` function synchronously in `us-east-1`. It supports creating, updating, and deleting the accounts
and buckets described by a tab-separated input file. Its `info` mode instead inventories buckets in the accounts under
one AWS Organizations organizational unit.

The Lambda function name is currently hard-coded as `AccountBuilderLambda`. Install the Lambda under that name, as in
the preceding instructions, or change `FunctionName` in the controller before running it.

### Install the controller

Run the controller from a trusted administrative workstation with network access to Keycloak and the AWS APIs. It
requires Python 3.10 or newer and the following packages in addition to the standard library:

Linux:

```bash
python -m venv .venv-account-controller
source .venv-account-controller/bin/activate
python -m pip install aiohttp boto3 python-dotenv PyJWT
```

Windows PowerShell:

```powershell
py -m venv .venv-account-controller
.\.venv-account-controller\Scripts\Activate.ps1
py -m pip install aiohttp boto3 python-dotenv PyJWT
```

The source also imports `collect_manifest_checksums` from `find_manifest_checksums`, although it does not currently call
that function. This repository does not include `find_manifest_checksums.py`; provide that internal module in
[`scripts`](scripts), install it from your organization's trusted package source, or remove the unused import before
running the controller. Without the module, Python stops at import time.

Create `scripts/.env`. The controller resolves this file relative to the script, including when it is bundled as an
executable:

```dotenv
ADMIN_ROLE=OrganizationAccountAccessRole
KEYLOAK_URL=https://id.example.org/realms/hea/protocol/openid-connect/token
ORG_ADMIN_ROLE_ARN=arn:aws:iam::111122223333:role/hea-organization-admin
CLOUD_TEMPLATE_BUCKET_NAME=hea-cloud-templates
THUMBPRINT=0123456789abcdef0123456789abcdef01234567
OIDC_PROVIDER_URL=id.example.org/realms/hea
INVENTORY_BUCKET=hea-inventory
SQS_QUEUE_NAME=hea-s3-events
SQS_TOPIC_NAME=hea-s3-events
EVENT_BRIDGE_NAME=hea-s3-bucket-events
ORG_ID=o-exampleorgid
ENVIRONMENT=dev
```

The variables have the following meanings:

| Variable | Purpose |
| --- | --- |
| `ADMIN_ROLE` | Role name assumed with the Keycloak token in each member account for `info` mode. |
| `KEYLOAK_URL` | Keycloak/OpenID Connect token endpoint. The spelling `KEYLOAK_URL` matches the source. |
| `ORG_ADMIN_ROLE_ARN` | Management-account role assumed with the Keycloak token before Organizations and Lambda calls. |
| `CLOUD_TEMPLATE_BUCKET_NAME` | Sent to the Lambda as `sourcebucket`; the current Lambda accepts but does not use it. |
| `THUMBPRINT` | SHA-1 certificate thumbprint used when configuring the member-account IAM OIDC provider. |
| `OIDC_PROVIDER_URL` | IAM OIDC provider URL without the `https://` prefix. |
| `INVENTORY_BUCKET` | Base name for regional inventory buckets. |
| `SQS_QUEUE_NAME` | Name of the management-account SQS event queue in `us-east-1`. |
| `SQS_TOPIC_NAME` | Name of the management-account SNS event topic used outside `us-east-1`. |
| `EVENT_BRIDGE_NAME` | Name of the management-account EventBridge custom event bus in `us-east-1`. |
| `ORG_ID` | Sent to the Lambda as `orgid`; the current Lambda accepts but does not use it. |
| `ENVIRONMENT` | Must be `prod`, `stg`, or `dev`. Non-production values become an inventory-bucket suffix. |

For example, an `INVENTORY_BUCKET` of `hea-inventory`, a region of `us-west-2`, and an environment of `dev` produce
`hea-inventory-uswest2-dev`. Production omits the final environment suffix. Create all resulting inventory buckets
before using the controller.

The IAM OIDC provider and trust policy for `ORG_ADMIN_ROLE_ARN` must accept the Keycloak issuer, the `core-browser`
audience, and `sts:AssumeRoleWithWebIdentity`. The role needs `lambda:InvokeFunction` on the
`AccountBuilderLambda` function. For `info` mode it also needs these Organizations permissions:

```text
organizations:ListRoots
organizations:ListOrganizationalUnitsForParent
organizations:ListAccountsForParent
organizations:DescribeAccount
```

Each member account's `ADMIN_ROLE` must trust the same web identity and allow `s3:ListAllMyBuckets` and
`s3:GetBucketLocation` for `info` mode. Applicable IAM policies and AWS Organizations service control policies must
also permit the operations.

The controller decodes the access token without locally verifying its signature, and its application-level admin-role
check is currently commented out. AWS STS still evaluates the token while assuming the configured roles, so the role
trust policies are the effective authorization boundary. Do not use a broadly trusted organization-administrator role.

### Input files

Input files are TSV files without a header row. Every bucket name must follow the AWS S3 bucket naming rules. The
columns depend on the mode:

| Mode | Columns |
| --- | --- |
| `create` | `bucket_name`, `account_email`, `account_name`, optional `region` |
| `create` with `--account_email` | `bucket_name`, `account_name` |
| `update` | `bucket_name`, `account_name`, `account_id`, optional `region` |
| `delete` | `bucket_name`, `account_name`, `account_id`, optional `region` |
| `delete_all` | `bucket_name`, `account_name`, `account_id`, optional `region` |

When a row omits `region`, the value of `--region` is used; its default is `us-east-1`. For example, a create file could
contain:

```text
cb-research-a-data	research-a@example.org	research-a	us-east-1
cb-research-b-data	research-b@example.org	research-b	us-west-2
```

Alternatively, use `--account_email` as an email template and provide two-column rows:

```text
cb-research-a-data	research-a
cb-research-b-data	research-b
```

The controller inserts `--account_email_count` immediately before the `@` sign and increments it for each row. For
example, `--account_email research@example.org --account_email_count 10` generates `research10@example.org`, then
`research11@example.org`.

An update or delete file could contain:

```text
cb-research-a-data	research-a	444455556666	us-east-1
```

### Run the controller

Username, password, output file, mode, and an input file are needed for account-changing operations. The following
example creates every account in `create-accounts.tsv`:

Linux:

```bash
python scripts/AccountCreationController.py \
    --username '<keycloak-username>' \
    --password '<keycloak-password>' \
    --output_file created-accounts.tsv \
    --mode create \
    --account_params_file create-accounts.tsv \
    --lifecycletype moderate
```

Windows PowerShell:

```powershell
py scripts\AccountCreationController.py `
    --username '<keycloak-username>' `
    --password '<keycloak-password>' `
    --output_file created-accounts.tsv `
    --mode create `
    --account_params_file create-accounts.tsv `
    --lifecycletype moderate
```

Passing a password on the command line can expose it in shell history and the local process list. Run the controller
only on a controlled workstation, prevent the history entry where the shell supports that, and use a short-lived
administrative credential.

Use `update`, `delete`, or `delete_all` with the corresponding TSV layout. `delete` can remove only an empty bucket.
`delete_all` deletes all objects and versions first and therefore causes unrecoverable data loss; the controller asks
for confirmation twice and displays the affected bucket names before invoking the Lambda.

To list buckets for the accounts directly beneath an organizational unit, use `info`. This mode does not use an account
parameters file:

Linux:

```bash
python scripts/AccountCreationController.py \
    --username '<keycloak-username>' \
    --password '<keycloak-password>' \
    --output_file account-buckets.tsv \
    --mode info \
    --org_unit 'Core Browser'
```

Windows PowerShell:

```powershell
py scripts\AccountCreationController.py `
    --username '<keycloak-username>' `
    --password '<keycloak-password>' `
    --output_file account-buckets.tsv `
    --mode info `
    --org_unit 'Core Browser'
```

The output is a headerless TSV file with `bucket_name`, `account_name`, `account_id`, and `region`, in that order. This
is also the input layout accepted by `update`, `delete`, and `delete_all`. Use `--version` to print the controller
version without performing an AWS action.

## Development environment

### Build requirements
* Any development environment is fine.
* On Windows, you also will need:
    * Build Tools for Visual Studio 2019, found at https://visualstudio.microsoft.com/downloads/. Select the C++ tools.
    * git, found at https://git-scm.com/download/win.
* On Mac, Xcode or the command line developer tools is required, found in the Apple Store app.
* Python 3.10, 3.11, or 3.12: Download and install it from https://www.python.org, and select the options to install for all
users and add Python to your environment variables. The install for all users option will help keep you from
accidentally installing packages into your Python installation's site-packages directory instead of to your virtualenv
environment, described below.
* Create a virtualenv environment using the `python -m venv <venv_directory>` command, substituting `<venv_directory>`
with the directory name of your virtual environment. Run `source <venv_directory>/bin/activate` (or `<venv_directory>/Scripts/activate` on Windows) to activate the virtual
environment. You will need to activate the virtualenv every time before starting work, or your IDE may be able to do
this for you automatically. **Note that PyCharm will do this for you, but you have to create a new Terminal panel
after you newly configure a project with your virtualenv.**
* From the project's root directory, and using the activated virtualenv, run `pip install wheel` followed by
  `pip install -r requirements_dev.txt`. **Do NOT run `python setup.py develop`. It will break your environment.**

### Running tests
Run tests with the `pytest` command from the project root directory. To improve performance, run tests in multiple
processes with `pytest -n auto`.

### Testing using Swagger
Run `python ./run-swaggerui.py` and open up http://locahost:8080/docs in your web browser to get a UI for making REST
API calls.

### Versioning
Use semantic versioning as described in
https://packaging.python.org/guides/distributing-packages-using-setuptools/#choosing-a-versioning-scheme. In addition,
while development is underway, the version should be the next version number suffixed by `.dev`.

### Version tags in git
Version tags should follow the format `heaserver-awsaccounts-<version>`, for example, `heaserver-awsaccounts-1.0.0`.

### Uploading to an index server
The following instructions assume separate stable and staging indexes. Numbered releases, including alphas and betas, go
into the stable index. Snapshots of works in progress go into the staging index. Thus, use staging to upload numbered
releases, verify the uploaded packages, and then upload to stable.

From the project's root directory:
1. For numbered releases, remove `.dev` from the version number in setup.py, tag it in git to indicate a release,
and commit to version control. Skip this step for developer snapshot releases.
2. Run `python setup.py clean --all sdist bdist_wheel` to create the artifacts.
3. Run `twine upload -r <repository> dist/<wheel-filename> dist/<tarball-filename>` to upload to the
 repository. The repository name has to be defined in a twine configuration file such as `$HOME/.pypirc`.
4. For numbered releases, increment the version number in setup.py, append `.dev` to it, and commit to version
control with a commit message like, "Prepare for next development iteration."
