Metadata-Version: 2.4
Name: gluekit
Version: 1.2.1
Summary: A developer‑centric Glue CLI built to speed local iteration with git/uv‑style commands, while bridging to highly controlled (eg Fedramped) AWS consoles for PII‑sensitive runs and smoothing onboarding around Glue’s quirks—like the lack of script‑to‑notebook conversion and the friction of refactoring Glue scripts.
Author-email: Mike Kranz <kranz-michael@norc.org>
Keywords: python
Requires-Python: <4.0,>=3.10
Description-Content-Type: text/markdown
Requires-Dist: boto3>=1.28.0
Requires-Dist: python-dotenv>=1.2.1
Requires-Dist: python-slugify>=8.0.4
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: rapidfuzz>=3.14.3
Requires-Dist: typer>=0.9.0
Provides-Extra: job-dev
Requires-Dist: moto[s3,ssm]>=5.1.16; extra == "job-dev"
Requires-Dist: sqlframe[duckdb]>=4.3.0; extra == "job-dev"
Provides-Extra: spark-dev
Requires-Dist: moto[s3,ssm]>=5.1.16; extra == "spark-dev"
Requires-Dist: pyspark<4,>=3.3; extra == "spark-dev"

# Gluekit

Gluekit works with one active AWS Glue job in a local workspace. Activate a job, edit its local definition or arguments, convert its script and notebook, then push or pull files through AWS Glue and S3.

```bash
gluekit activate my-job --profile my-aws-profile
gluekit status
gluekit settings show
gluekit settings set --worker-type G.1X --number-of-workers 5
gluekit params set --temp-dir s3://my-bucket/tmp/
gluekit script convert
gluekit notebook convert
gluekit push --to aws --dry-run
gluekit push --to aws
```

Install with `uv tool install .` or run from this repository with `uv run gluekit`.

## Commands

| Command | Purpose |
| --- | --- |
| `activate <job>` | Select one job and optionally an AWS profile for later commands. `activate --clear` clears the selection. |
| `settings show/set` | Inspect or change local Glue job fields such as `Role`, `WorkerType`, and `Timeout`. |
| `params show/set` | Inspect or change `DefaultArguments` and `NonOverridableArguments`. |
| `params save <key> <value>` | Save a reusable value for the active job; add `--profile` or `--global` for that scope. `params saved [job]` inspects these values. |
| `script convert` / `notebook convert` | Convert to the other representation. |
| `script sync` / `notebook sync` | Refresh generated metadata from the local job definition. |
| `script import-settings` / `notebook import-settings` | Import represented metadata into the local job definition. |
| `push --to aws` / `pull --from aws` | Transfer the full selected job. Pass a job name explicitly to override the selection; pass `'*'` explicitly for all jobs. |
| `script push/pull` / `notebook push/pull` | Transfer only that artifact. |
| `init`, `build`, `status` | Set up profile values, build local dependencies, and inspect the active job. |
| `local`, `run` | Set up local fixtures and run Glue scripts locally. |

The default transfer endpoint is AWS. The `--to` and `--from` flags accept `aws`; a GitHub transfer backend is planned separately. Transfer commands use local files as the other endpoint, and do not need `--include` or `--exclude`. To inspect the available options, use `gluekit <command> --help`.

### Script and notebook conversion

```bash
uvx gluekit notebook convert my-job  # notebook → clean Python script
uvx gluekit script convert my-job    # script → notebook with Glue magics
```

Omit `my-job` to use the active job. `--artifact-only` skips the local job
configuration and generated magics; `--dry-run` writes neither artifacts nor state.

Notebook exports contain Python and commented markdown, with no percent cell
markers, generated Glue configuration header, or commented pyproject/PEP 723
metadata. Existing percent markers are removed as well. Python indentation and
user comments are retained; Gluekit does not reformat executable code. Non-Python
magics such as SQL remain as commented documentation, not executable Python.

Cell boundaries live separately under `notebook_cells["my-job"]` in
`.gluekit/glue_set.json`. The `markers` object maps **1-based script line numbers**
to the marker inserted *before* that line when reconstructing notebook cells:

```json
{"markers": {"1": "#%%", "9": "#%%", "15": "#%%[markdown]"}}
```

Each entry also records script/notebook paths and line hashes. At conversion time,
Python's standard-library `difflib` relocates boundaries after insertions,
replacements, and deletions; `ast` moves unsafe boundaries to the start of the
containing top-level statement. Functions, decorators, classes, loops, `with`,
`try/except/finally`, multiline expressions, and `match` blocks stay intact.
Markdown is restored from its script comments without accumulating comment prefixes.
The map updates on conversion, not continuously while editing; no watcher or extra
dependency is needed. Line-diff matching is best-effort for moved or repeated code,
but AST validation still prevents a boundary from splitting a statement.

For a script without saved boundaries or explicit percent markers, Gluekit groups
global setup and starts cells around top-level definitions and compound statements.
Nested blocks remain in their containing cell. Existing explicit markers take
precedence over saved boundaries and are also checked for scope safety. Invalid
Python causes conversion to fail before writing the notebook or its saved map.

Script-to-notebook conversion generates the existing Glue session magics from the
local job configuration, but omits `%iam_role`. It replaces script-only
`job.commit()` statements with no-ops so otherwise empty suites remain valid.
`notebook sync` also omits the role, and importing a notebook without a role leaves
the configured job role intact. Explicitly authored role magics can still be imported.
See the [AWS interactive-session magic reference](https://docs.aws.amazon.com/glue/latest/dg/interactive-sessions-magics.html).

`script sync` remains an explicit opt-in to the older commented Glue configuration
representation; a clean notebook export does not contain settings for
`script import-settings`. Use `notebook import-settings` or edit the job configuration
instead. Saved cell metadata is local workspace state, never an AWS job parameter.

### State migration

Existing `.gluekit/glue_set.json` files with a `checkout` entry are read as the active selection. The next state write stores it under `active`. The old `checkout` command and other deprecated aliases are no longer registered. The existing `local` and `run` behavior remains available.

## CLI reference

<!-- BEGIN: CLI_HELP -->
> Auto-generated from live CLI `--help` output.

### `gluekit --help`

```text
Usage: gluekit [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion  Install completion for the current shell.
  --show-completion     Show completion for the current shell, to copy it or
                        customize the installation.
  --help                Show this message and exit.

Commands:
  activate  Replace the active job.
  build     Build local artifacts; only touches AWS when --push is used.
  init      Initialize local repo-level Gluekit settings from AWS CLI...
  pull      Transfer the full active job between local files and AWS.
  push      Transfer the full active job between local files and AWS.
  run       Run a Glue script locally with emulated Glue libraries and...
  status    Show the active job and its local-to-S3 artifact mappings.
  settings  Edit the active Glue job definition.
  params    Edit the active job's Glue arguments.
  script    Work with the active job's script.
  notebook  Work with the active job's notebook.
  local     Local Glue fixtures and setup.
```


### `gluekit activate --help`

```text
Usage: gluekit activate [OPTIONS] [job_name]

  Replace the active job.

Arguments:
  job_name  Glue job name to activate for later commands.

Options:
  --config-dir <path>  Directory containing Glue job config files.  [default:
                       glue/configs]
  -p, --profile <str>  AWS CLI credential profile name to store as the active
                       gluekit profile scope for saved params.
  --dry-run            Show gluekit profile-scoped config updates without
                       writing files.
  --clear              Remove the active job selection without deleting saved
                       params or local files.
  --local              Check out the single local development settings file in
                       .gluekit/local.json.
  --help               Show this message and exit.

  **Examples**

  - `gluekit activate my-job`

  - `gluekit activate --local`

  - `gluekit activate --clear`
```


### `gluekit status --help`

```text
Usage: gluekit status [OPTIONS]

  Show the active job and its local-to-S3 artifact mappings.

Options:
  --config-dir <path>  Directory containing Glue job config files.  [default:
                       glue/configs]
  --help               Show this message and exit.

  **Examples**

  - `gluekit status`

  - `gluekit status --config-dir glue/configs`
```


### `gluekit settings --help`

```text
Usage: gluekit settings [OPTIONS] COMMAND [ARGS]...

  Edit the active Glue job definition.

Options:
  --help  Show this message and exit.

Commands:
  show  Show the active job's local Glue definition.
  set   Manage local AWS Glue job-definition properties.
```


### `gluekit settings show --help`

```text
Usage: gluekit settings show [OPTIONS] [job_name]

  Show the active job's local Glue definition.

Arguments:
  job_name

Options:
  --config-dir <path>  [default: glue/configs]
  --help               Show this message and exit.
```


### `gluekit settings set --help`

```text
Usage: gluekit settings set [OPTIONS] [job_name]

  Manage local AWS Glue job-definition properties.

Arguments:
  job_name  Glue job name. Defaults to the active job selection.

Options:
  --description <str>             Set the job description.
  --role <str>                    Set the IAM role for the job.
  --glue-version <str>            Set the Glue version.
  --worker-type <str>             Set the worker type.
  --number-of-workers <int>       Set the number of workers.
  --timeout <int>                 Set the job timeout in minutes.
  --max-retries <int>             Set the max retry count.
  --execution-class <str>         Set the execution class.
  --max-concurrent-runs <int>     Set the max concurrent runs.
  --job-run-queuing-enabled / --no-job-run-queuing-enabled
                                  Enable or disable job run queuing.
  --security-configuration <str>  Set the security configuration name.
  --maintenance-window <str>      Set the maintenance window.
  --log-uri <str>                 Set the log URI.
  --command-name <str>            Set Command.Name.
  --python-version <str>          Set Command.PythonVersion.
  --runtime <str>                 Set Command.Runtime.
  --remote-script-location <str>  Set Command.ScriptLocation.
  --local-script-location <str>   Set SourceControlDetails.ScriptLocation.
  --notebook-location <str>       Set SourceControlDetails.NotebookLocation and
                                  NotebookPath.
  --source-control-provider <str>
                                  Set SourceControlDetails.Provider.
  --source-control-owner <str>    Set SourceControlDetails.Owner.
  --source-control-repository <str>
                                  Set SourceControlDetails.Repository.
  --source-control-branch <str>   Set SourceControlDetails.Branch.
  --source-control-folder <str>   Set SourceControlDetails.Folder.
  --add-connection <str>          Add one or more Glue connections.
  --remove-connection <str>       Remove one or more Glue connections.
  --dry-run                       Show what would be updated without writing
                                  files.
  --config-dir <path>             Directory containing Glue job config files.
                                  [default: glue/configs]
  --help                          Show this message and exit.

  **Examples**

  - `gluekit settings set --description 'Updated job description' --glue-version
  5.0`

  - `gluekit settings set my-job --worker-type G.1X --number-of-workers 5`
```


### `gluekit params --help`

```text
Usage: gluekit params [OPTIONS] COMMAND [ARGS]...

  Edit the active job's Glue arguments.

Options:
  --help  Show this message and exit.

Commands:
  show   Show the active job's local Glue arguments.
  set    Manage local AWS Glue runtime and job arguments.
  save   Save a reusable value for the active job, profile, or all jobs.
  saved  Show stored local Glue config params.
```


### `gluekit params show --help`

```text
Usage: gluekit params show [OPTIONS] [job_name]

  Show the active job's local Glue arguments.

Arguments:
  job_name

Options:
  --config-dir <path>  [default: glue/configs]
  --help               Show this message and exit.
```


### `gluekit params set --help`

```text
Usage: gluekit params set [OPTIONS] [job_name]

  Manage local AWS Glue runtime and job arguments.

Arguments:
  job_name  Glue job name. Defaults to the active job selection.

Options:
  --add-additional-python-modules <str>
                                  Add one or more values to --additional-python-
                                  modules.
  --remove-additional-python-modules <str>
                                  Remove one or more values from --additional-
                                  python-modules.
  --add-extra-py-files <str>      Track local relative Python files in
                                  SourceControlDetails.ExtraPyFiles and --extra-
                                  py-files.
  --remove-extra-py-files <str>   Remove local Python files from
                                  SourceControlDetails.ExtraPyFiles and --extra-
                                  py-files.
  --add-extra-jars <str>          Add one or more values to --extra-jars.
  --remove-extra-jars <str>       Remove one or more values from --extra-jars.
  --add-additional-python-files <str>
                                  Track local paths in
                                  SourceControlDetails.AdditionalPythonFiles and
                                  --extra-py-files.
  --remove-additional-python-files <str>
                                  Remove local paths from
                                  SourceControlDetails.AdditionalPythonFiles and
                                  --extra-py-files.
  --add-extra-files <str>         Track local paths in
                                  SourceControlDetails.ExtraFiles and --extra-
                                  files.
  --remove-extra-files <str>      Remove local paths from
                                  SourceControlDetails.ExtraFiles and --extra-
                                  files.
  --add-default-argument <str>    Set a DefaultArguments entry using KEY=VALUE.
                                  Repeat as needed.
  --remove-default-argument <str>
                                  Remove a DefaultArguments entry by key. Repeat
                                  as needed.
  --add-non-overridable-argument <str>
                                  Set a NonOverridableArguments entry using
                                  KEY=VALUE. Repeat as needed.
  --remove-non-overridable-argument <str>
                                  Remove a NonOverridableArguments entry by key.
                                  Repeat as needed.
  --enable-metrics / --disable-metrics
                                  Set DefaultArguments.--enable-metrics to true
                                  or false.
  --enable-observability-metrics / --disable-observability-metrics
                                  Set DefaultArguments.--enable-observability-
                                  metrics to true or false.
  --enable-glue-datacatalog / --disable-glue-datacatalog
                                  Set DefaultArguments.--enable-glue-datacatalog
                                  to true or false.
  --enable-continuous-cloudwatch-log / --disable-continuous-cloudwatch-log
                                  Set DefaultArguments.--enable-continuous-
                                  cloudwatch-log to true or false.
  --enable-spark-ui / --disable-spark-ui
                                  Set DefaultArguments.--enable-spark-ui to true
                                  or false.
  --enable-job-insights / --disable-job-insights
                                  Set DefaultArguments.--enable-job-insights to
                                  true or false.
  --enable-auto-scaling / --disable-auto-scaling
                                  Set DefaultArguments.--enable-auto-scaling to
                                  true or false.
  --job-bookmark-option <str>     Set DefaultArguments.--job-bookmark-option.
  --temp-dir <str>                Set DefaultArguments.--TempDir.
  --spark-event-logs-path <str>   Set DefaultArguments.--spark-event-logs-path.
  --datalake-formats <str>        Set DefaultArguments.--datalake-formats.
  --spark-conf <str>              Set DefaultArguments.--conf.
  --continuous-log-log-group <str>
                                  Set DefaultArguments.--continuous-log-
                                  logGroup.
  --continuous-log-log-stream-prefix <str>
                                  Set DefaultArguments.--continuous-log-
                                  logStreamPrefix.
  --continuous-log-conversion-pattern <str>
                                  Set DefaultArguments.--continuous-log-
                                  conversionPattern.
  --executor-cores <int>          Set DefaultArguments.--executor-cores.
  --customer-driver-env-vars <str>
                                  Set DefaultArguments.--customer-driver-env-
                                  vars.
  --customer-executor-env-vars <str>
                                  Set DefaultArguments.--customer-executor-env-
                                  vars.
  --python-modules-installer-option <str>
                                  Set DefaultArguments.--python-modules-
                                  installer-option.
  --dry-run                       Show what would be updated without writing
                                  files.
  --config-dir <path>             Directory containing Glue job config files.
                                  [default: glue/configs]
  --help                          Show this message and exit.

  **Examples**

  - `gluekit params set --temp-dir s3://bucket/tmp --enable-metrics`

  - `gluekit params set my-job --add-default-argument survey_id=test`
```


### `gluekit params save --help`

```text
Usage: gluekit params save [OPTIONS] {key} {value}

  Save a reusable value for the active job, profile, or all jobs.

Arguments:
  key    Saved parameter name.  [required]
  value  Saved parameter value.  [required]

Options:
  --job <str>          Override the active job.
  --global             Save params as global defaults for all jobs.
  -p, --profile <str>  Save params under this gluekit profile scope, usually
                       matching an AWS CLI profile name; does not contact AWS.
  --help               Show this message and exit.

  **Examples**

  - `gluekit params save default_arguments.--TempDir s3://my-bucket/tmp/`

  - `gluekit params save additional_python_modules "pydantic==2.11.7" --global`
```


### `gluekit params saved --help`

```text
Usage: gluekit params saved [OPTIONS] [job_name]

  Show stored local Glue config params.

Arguments:
  job_name  Optional Glue job name to view merged params (global + job).

Options:
  -p, --profile <str>  Include params from this gluekit profile scope, usually
                       matching an AWS CLI profile name; does not contact AWS.
  --help               Show this message and exit.

  **Examples**

  - `gluekit params saved my-job`

  - `gluekit params saved`
```


### `gluekit script --help`

```text
Usage: gluekit script [OPTIONS] COMMAND [ARGS]...

  Work with the active job's script.

Options:
  --help  Show this message and exit.

Commands:
  convert          Convert the script to a notebook.
  sync             Refresh generated script metadata from the local job...
  import-settings  Import represented script metadata into the local job...
  push             Upload the local script to AWS.
  pull             Download the remote script from AWS.
```


### `gluekit script convert --help`

```text
Usage: gluekit script convert [OPTIONS] [job_name]

  Convert the script to a notebook.

Arguments:
  job_name

Options:
  --use-config / --artifact-only  [default: use-config]
  --dry-run
  --config-dir <path>             [default: glue/configs]
  --help                          Show this message and exit.
```


### `gluekit script sync --help`

```text
Usage: gluekit script sync [OPTIONS] [job_name]

  Refresh generated script metadata from the local job config.

Arguments:
  job_name

Options:
  --dry-run
  --config-dir <path>  [default: glue/configs]
  --help               Show this message and exit.
```


### `gluekit script import-settings --help`

```text
Usage: gluekit script import-settings [OPTIONS] [job_name]

  Import represented script metadata into the local job config.

Arguments:
  job_name

Options:
  --dry-run
  --config-dir <path>  [default: glue/configs]
  --help               Show this message and exit.
```


### `gluekit script push --help`

```text
Usage: gluekit script push [OPTIONS] [job_name]

  Upload the local script to AWS.

Arguments:
  job_name

Options:
  --to <str>                      [default: aws]
  --dry-run
  --config-dir <path>             [default: glue/configs]
  -p, --profile <str>
  --auto-login / --no-auto-login  [default: auto-login]
  --help                          Show this message and exit.
```


### `gluekit script pull --help`

```text
Usage: gluekit script pull [OPTIONS] [job_name]

  Download the remote script from AWS.

Arguments:
  job_name

Options:
  --from <str>                    [default: aws]
  --dry-run
  --config-dir <path>             [default: glue/configs]
  -p, --profile <str>
  --auto-login / --no-auto-login  [default: auto-login]
  --help                          Show this message and exit.
```


### `gluekit notebook --help`

```text
Usage: gluekit notebook [OPTIONS] COMMAND [ARGS]...

  Work with the active job's notebook.

Options:
  --help  Show this message and exit.

Commands:
  convert          Convert the notebook to a script.
  sync             Refresh generated notebook metadata from the local job...
  import-settings  Import represented notebook metadata into the local job...
  push             Upload the local notebook to AWS.
  pull             Download the remote notebook from AWS.
```


### `gluekit notebook convert --help`

```text
Usage: gluekit notebook convert [OPTIONS] [job_name]

  Convert the notebook to a script.

Arguments:
  job_name

Options:
  --use-config / --artifact-only  [default: use-config]
  --dry-run
  --config-dir <path>             [default: glue/configs]
  --help                          Show this message and exit.
```


### `gluekit notebook sync --help`

```text
Usage: gluekit notebook sync [OPTIONS] [job_name]

  Refresh generated notebook metadata from the local job config.

Arguments:
  job_name

Options:
  --dry-run
  --config-dir <path>  [default: glue/configs]
  --help               Show this message and exit.
```


### `gluekit notebook import-settings --help`

```text
Usage: gluekit notebook import-settings [OPTIONS] [job_name]

  Import represented notebook metadata into the local job config.

Arguments:
  job_name

Options:
  --dry-run
  --config-dir <path>  [default: glue/configs]
  --help               Show this message and exit.
```


### `gluekit notebook push --help`

```text
Usage: gluekit notebook push [OPTIONS] [job_name]

  Upload the local notebook to AWS.

Arguments:
  job_name

Options:
  --to <str>                      [default: aws]
  --dry-run
  --config-dir <path>             [default: glue/configs]
  -p, --profile <str>
  --auto-login / --no-auto-login  [default: auto-login]
  --help                          Show this message and exit.
```


### `gluekit notebook pull --help`

```text
Usage: gluekit notebook pull [OPTIONS] [job_name]

  Download the remote notebook from AWS.

Arguments:
  job_name

Options:
  --from <str>                    [default: aws]
  --dry-run
  --config-dir <path>             [default: glue/configs]
  -p, --profile <str>
  --auto-login / --no-auto-login  [default: auto-login]
  --help                          Show this message and exit.
```


### `gluekit push --help`

```text
Usage: gluekit push [OPTIONS] [job_name] [target_job]

  Transfer the full active job between local files and AWS.

Arguments:
  job_name    Glue job name or pattern to push. Use "*" for all configs.
  target_job  Optional remote Glue job name to push to.

Options:
  --dry-run                       Show what would be uploaded without writing
                                  files.
  --update-config / --no-update-config
                                  Update Glue job configuration after uploading
                                  files.  [default: update-config]
  --build                         Build local package artifacts in the workspace
                                  root before push.
  --build-tool <str>              Build frontend for --build: auto, uv, or
                                  build.  [default: auto]
  -v, --verbose                   Print local build command(s) before execution.
  --config-dir <path>             Directory containing Glue job config files.
                                  [default: glue/configs]
  -p, --profile <str>             AWS CLI credential profile used for real AWS
                                  Glue/S3 API calls; not a gluekit local test
                                  profile.
  --auto-login / --no-auto-login  For real AWS profiles, automatically run 'aws
                                  sso login' when credentials are missing or
                                  expired.  [default: auto-login]
  --to <str>                      Transfer endpoint (aws).  [default: aws]
  --help                          Show this message and exit.

  **Examples**

  - `gluekit push my-job --profile my-sso-profile`

  - `gluekit push source-job target-job --profile my-sso-profile`

  - `gluekit push "my-job-\*"`

  - `gluekit push my-job --build --build-tool auto --dry-run`
```


### `gluekit pull --help`

```text
Usage: gluekit pull [OPTIONS] [job_name] [target_job]

  Transfer the full active job between local files and AWS.

Arguments:
  job_name    Glue job name or pattern to pull. Use "*" for all jobs.
  target_job  Optional local job name to write pulled artifacts to.

Options:
  -f, --force                     Download even if the local config is newer or
                                  equal to LastModifiedOn.
  --dry-run                       Show what would be downloaded without writing
                                  files.
  --config-dir <path>             Directory containing Glue job config files.
                                  [default: glue/configs]
  --write-job-list / --no-write-job-list
                                  Write glue/glue_full_job_list.csv during the
                                  run.  [default: write-job-list]
  -p, --profile <str>             AWS CLI credential profile used for real AWS
                                  Glue/S3 API calls; not a gluekit local test
                                  profile.
  --auto-login / --no-auto-login  For real AWS profiles, automatically run 'aws
                                  sso login' when credentials are missing or
                                  expired.  [default: auto-login]
  --from <str>                    Transfer endpoint (aws).  [default: aws]
  --help                          Show this message and exit.

  **Examples**

  - `gluekit pull my-job --profile my-sso-profile`

  - `gluekit pull source-job target-job --profile my-sso-profile`

  - `gluekit pull "my-job-\*" --force`

  - `gluekit pull "\*" --dry-run --profile my-sso-profile --no-auto-login`
```


### `gluekit init --help`

```text
Usage: gluekit init [OPTIONS]

  Initialize local repo-level Gluekit settings from AWS CLI profiles.

Options:
  --aws-config <path>             Local AWS CLI config file to scan for
                                  credential profile names; defaults to
                                  AWS_CONFIG_FILE or ~/.aws/config.
  --config-dir <path>             Directory containing Glue job config files.
                                  [default: glue/configs]
  --auto-login / --no-auto-login  For selected real AWS profiles, automatically
                                  run 'aws sso login' when credentials are
                                  missing or expired.  [default: auto-login]
  --detect-roles / --no-detect-roles
                                  Fetch the selected real AWS Glue job in each
                                  profile and save its Role as local gluekit
                                  params.  [default: detect-roles]
  --detect-buckets / --no-detect-buckets
                                  Compare S3 URIs in real AWS Glue job configs
                                  across profiles and save local gluekit profile
                                  mappings.  [default: detect-buckets]
  --match-threshold <int>         Minimum rapidfuzz score for non-exact S3 key
                                  matches.  [default: 85]
  --baseline-profile <str>        Selected AWS CLI profile to store as the first
                                  gluekit activate scope and S3 comparison
                                  baseline.
  --profile-job <str>             Real AWS Glue job to inspect for an AWS CLI
                                  profile, as PROFILE=GLUE_JOB_NAME. May be
                                  repeated.
  --help                          Show this message and exit.

  **Examples**

  - `gluekit init`
```


### `gluekit build --help`

```text
Usage: gluekit build [OPTIONS]

  Build local artifacts; only touches AWS when --push is used.

Options:
  --job-name <str>                Active or explicit local Glue job to update
                                  when using --package-whl or --push.
  --dry-run                       Show what would be built and zipped without
                                  running build commands.
  --bump-version, --bump <str>    Bump pyproject.toml [project].version before
                                  building: major, minor, patch, or X.Y.Z.
  --build-tool <str>              Build frontend to use: auto, uv, or build.
                                  [default: auto]
  --out-dir <path>                The output directory to which distributions
                                  should be written.
  --no-version-suffix             Remove the version suffix from generated zip
                                  artifact names.
  -v, --verbose                   Print build commands before execution.
  --package-whl                   After building, replace the tracked dist/*.whl
                                  in the selected Glue config.
  --push                          Push the selected Glue config and artifacts
                                  after building.
  --update-config / --no-update-config
                                  For --push, update Glue job configuration
                                  after uploading files.  [default: update-
                                  config]
  --config-dir <path>             Directory containing Glue job config files.
                                  [default: glue/configs]
  -p, --profile <str>             AWS CLI credential profile for profile-scoped
                                  config params and, with --push, real AWS
                                  Glue/S3 API calls.
  --auto-login / --no-auto-login  For --push with a real AWS profile,
                                  automatically run 'aws sso login' when
                                  credentials are missing or expired.  [default:
                                  auto-login]
  --help                          Show this message and exit.

  **Examples**

  - `gluekit build`

  - `gluekit build --build-tool uv`

  - `gluekit build --out-dir build/artifacts --no-version-suffix`

  - `gluekit build --build-tool build --dry-run`
```


### `gluekit local --help`

```text
Usage: gluekit local [OPTIONS] COMMAND [ARGS]...

  Local Glue fixtures and setup.

Options:
  --help  Show this message and exit.

Commands:
  setup   Create or update and activate the local-only development setup.
  status  Show the active local development setup.
  s3      Manage local mocked S3 fixture mappings; does not call AWS S3.
  ssm     Manage local mocked SSM Parameter Store values; does not call AWS...
```


### `gluekit run --help`

```text
Usage: gluekit run [OPTIONS] {script_path}

  Run a Glue script locally with emulated Glue libraries and mocked AWS
  services.

Arguments:
  script_path  Path to the Glue Python script to execute locally.  [required]

Options:
  --glue-arg <str>       Glue script argument in KEY=VALUE form; passed through
                         as --KEY VALUE.
  --job-name <str>       Default JOB_NAME used when not provided in script args.
                         [default: local-glue-job]
  --create-bucket <str>  Create a mocked S3 bucket only inside the local run.
                         Repeat as needed.
  --ssm-parameter <str>  Seed a mocked SSM parameter only inside the local run
                         as NAME=VALUE. Repeat as needed.
  --aws-region <str>     Mock AWS region used for local boto3 clients. Defaults
                         to us-east-1.
  --config-file <file>   Local fixture config file for mocked S3/SSM. Defaults
                         to .gluekit/local.json when local settings are active.
  --runtime <str>        Local DataFrame runtime: sqlframe (fast default) or
                         pyspark (real local Spark).  [default: sqlframe]
  --spark-master <str>   Spark master URL used when --runtime=pyspark.
                         [default: local[2]]
  --help                 Show this message and exit.

  **Examples**

  - `gluekit run ../ppsc-survey/glue/scripts/job_a.py --create-bucket my-input-
  bucket`

  - `gluekit run glue/scripts/job-a.py --ssm-parameter /app/env=dev --glue-arg
  report_date=2026-06-18`

  - `gluekit run glue/scripts/job-a.py -- --JOB_NAME custom-local-job
  --report_date 2026-06-18`
```
<!-- END: CLI_HELP -->

## Package releases

Install the repository task runner with `uv tool install poethepoet==0.48.0`,
then run `poe release-check` for validation and `poe build` for a local wheel
and source distribution. One-off usage is
`uvx --from poethepoet==0.48.0 poe release-check`.

The `publish-to-pypi.yml` workflow has three jobs: **prepare**, **publish**, and
**release**. Pushes to `dev` start or increment a patch development version
(`1.0.0 → 1.0.1.dev1 → 1.0.1.dev2`). Pushes to `main` promote a prerelease to
stable, or increment the patch when the source already has a stable version.
After a stable tag exists, `dev` starts the next patch series. Normal releases
should not edit the version manually. Merge updated `main` back into `dev`
when needed to keep the branches' version baselines aligned.

`poe release --branch dev --source <full-commit-sha>` runs `scripts/release.py`
in a clean checkout of that source. It uses `uv version --no-sync` to update
`pyproject.toml` and `uv.lock`, invokes `poe build`, and creates a local release
commit and annotated tag. This is the CI preparation command: it changes the
local checkout but does not push or publish. The workflow first saves the built
distributions, then atomically pushes the release commit and tag. Both branches
share one release concurrency group; a stale run fails rather than overwriting
newer work. Rapid pushes may supersede pending runs; the latest source should
be released. Workflow pushes use `GITHUB_TOKEN` and do not recursively trigger
another push workflow.

PyPI publication uses the `pypi` GitHub environment and Trusted Publishing bound
to this repository and **`publish-to-pypi.yml`**. No long-lived PyPI token is
needed. Only stable `main` versions get a GitHub Release, after PyPI succeeds.
The branch rules must permit the workflow's version commit; rejected pushes
leave both remote refs unchanged.

For a failed publish, use **Re-run failed jobs** on the original Actions run.
The publish job downloads the saved wheel and source distribution without
rebuilding. Identical PyPI uploads can be retried, including a partially
completed upload. A full rerun recognizes the tagged source/branch before
calculating a version and reuses its original artifact. Existing GitHub Releases
are left intact. If preparation failed before the atomic push, a fresh attempt
can rebuild and replace that run's unpublished artifact. If the original
artifact has expired or been deleted after the push, stop and recover those
exact files; the workflow deliberately does not rebuild a published version.

`poe release-test` exercises the release helper with temporary local Git remotes
and real uv version changes; distribution builds are mocked. No test publishes
packages or contacts cloud providers.

Gluekit's `poe docs-check` creates `.venv-docs` and installs the package with
pip before checking generated CLI help. This preserves the existing docs
resolution behavior separately from `uv.lock` and the heavier development
extras. `poe release-check` runs that check and the release tests. After the
stacked CLI and release PRs land, create `dev` from the updated, released `main`;
do not start it from the older pre-refactor branch.
