Metadata-Version: 2.4
Name: mainframe-modernization-toolkit
Version: 0.1.7
Summary: Deterministic COBOL/JCL analysis and mainframe modernization tooling
Author: Mainframe Migration Toolkit Contributors
License-Expression: Apache-2.0
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: release
Requires-Dist: build>=1.2; extra == "release"
Requires-Dist: twine>=6; extra == "release"
Dynamic: license-file

# Mainframe Modernization Toolkit

Deterministic COBOL and JCL navigation, impact analysis, migration evidence,
and code-generation tools for Python modernization projects.

The package includes:

- A Python CLI with 17 deterministic analysis and generation commands.
- A bundled VS Code extension and COBOL/JCL language server.
- GitHub Copilot Language Model Tools for dependency-aware agent workflows.
- A packaged migration skill and specialized VS Code agent.

The parsers and graph operations do not use an AI model. Given the same source
and configuration, they produce the same result.

## Quick start

### 1. Install the Python package

Python 3.10 or newer is required.

```bash
python -m pip install mainframe-modernization-toolkit
mainframe-toolkit --version
```

If the command is not on `PATH`, use:

```bash
python -m mainframe_modernization_toolkit --version
```

### 2. Export and install the VS Code extension

The wheel contains the matching VSIX. Installing the Python package does not
modify VS Code automatically.

```bash
mainframe-toolkit vsix export --output mainframe-migration-toolkit.vsix
mainframe-toolkit vsix verify mainframe-migration-toolkit.vsix
code --install-extension mainframe-migration-toolkit.vsix
```

Reload VS Code after installation.

The extension automatically searches the Extension Host `PATH`, active and
workspace virtual environments, common script locations, and Python module
fallbacks. If launch fails, review the attempted candidates in the **Mainframe
Migration** output channel. Set `mainframeMigration.pythonPath` to a Python
interpreter containing the package, or set `mainframeMigration.executablePath`
to require one exact toolkit executable. Invalid explicit executable paths do
not silently fall back.

### 3. Initialize a mainframe workspace

Run this from the workspace containing COBOL, copybooks, and JCL:

```bash
mainframe-toolkit workspace init .
```

This adds, without overwriting existing files:

- `mainframe-migration.json` and its schema.
- The relational IR schema.
- `.github/skills/mainframe-jcl-migration/`.
- `.github/agents/mainframe-jcl-migrator.agent.md`.

Edit `mainframe-migration.json` so its source libraries, extensions, encoding,
known external programs, and transport profiles match the workspace.

### 4. Verify the setup

```bash
mainframe-toolkit doctor --workspace .
mainframe-toolkit run migration_preflight -- . --jcl MYJOB --format json
```

Preflight exits `0` when generation is unblocked and `2` when required source
or configuration is missing or ambiguous. Its JSON report includes detected
`capabilities`, a stable `capabilityDigest`, and `capabilityResolutions` that
show which user policy and adapter, if any, applies to each capability.

### 5. Use it in VS Code

The extension provides:

- F12 and hover for COBOL `CALL`, `COPY`, data items, and JCL `EXEC PGM=`.
- Context-aware copybook resolution.
- COBOL/JCL diagnostics, completion, and document symbols.
- Commands to reindex and inspect the dependency graph.
- Five deterministic Language Model Tools for Copilot agent mode.

Run **Mainframe Migration: Reindex COBOL/JCL Workspace** after changing source
library configuration.

Invoke the packaged workflow with a workspace root and JCL boundary:

```text
/mainframe-jcl-migration /path/to/workspace MYJOB migration/MYJOB
```

Alternatively, select the **Mainframe JCL Migrator** custom agent.

## How it works

The toolkit separates deterministic evidence collection from AI reasoning:

1. The language server indexes COBOL programs, copybooks, JCL jobs, calls,
   includes, data declarations, and execution edges.
2. Language Model Tools expose those indexed facts to Copilot.
3. Python commands persist graphs, warnings, contracts, rules, SQL, readers,
   fixtures, scaffolds, and migration reports.
4. The agent reasons over tool output instead of reconstructing dependencies
   from model memory.

Unresolved or unsafe constructs are never silently guessed. Tools emit
structured `BLOCK`, `TODO`, or informational findings for dynamic calls,
missing copybooks, ambiguous libraries, edited PIC clauses, ODO, `REDEFINES`,
transaction dialects, and unterminated SQL blocks.

## Running packaged tools

Prefer the umbrella command:

```bash
mainframe-toolkit run dependency_graph -- . --format json
mainframe-toolkit run impact_analysis -- . --changed ACCTREC --format text
mainframe-toolkit run business_rule_extractor -- app/cbl/VALIDATE.cbl --format markdown
mainframe-toolkit run copybook_to_contract -- app/cpy/ACCTREC.cpy --config mainframe-migration.json --format json
mainframe-toolkit run generate_file_readers -- . --out-dir migration/data --format text
```

The separator `--` ends arguments for `mainframe-toolkit`; everything after it
is passed to the selected tool.

The module form works when the console entry point is not on `PATH`:

```bash
python -m mainframe_modernization_toolkit run dependency_graph -- . --format json
```

Consumer workspaces do not need a `scripts/` directory. Do not locate or run
Python files inside site-packages directly.

## Tool catalog

| Tool | Purpose |
|---|---|
| `migration_preflight` | Validate configured inventory and migration blockers |
| `dependency_graph` | Build CALL, COPY, and EXEC dependency graphs |
| `impact_analysis` | Compute transitive upstream/downstream impact |
| `dead_code_finder` | Report unreferenced programs and copybooks with caveats |
| `sql_extractor` | Extract embedded SQL and host-variable evidence |
| `business_rule_extractor` | Extract reviewable IF/EVALUATE rules |
| `copybook_to_dataclass` | Generate Python models and optional DDL |
| `copybook_to_contract` | Build canonical physical and transport contracts |
| `generate_copybook_fixtures` | Generate deterministic ingestion-only fixtures |
| `generate_file_readers` | Generate binary-safe fixed-width readers |
| `cobol_to_python_skeleton` | Generate disposable traceability scaffolds |
| `jcl_flow_extractor` | Extract JCL steps, DDs, conditions, and flow |
| `migration_complexity_report` | Rank migration effort and risk |
| `characterization_test_scaffolder` | Scaffold golden-master harnesses |
| `generate_program_capsule` | Generate preflight-gated partial evidence capsules |
| `validate_relational_ir` | Validate reviewed relational IR |
| `ir_to_pyspark` | Compile executable relational IR to PySpark |

Each tool is also installed as an individual console entry point, but the
umbrella command is the stable form used by the VS Code extension and agent.

## Language Model Tools

The VS Code extension registers:

- `mainframe_getDependencyGraph`
- `mainframe_getCallers`
- `mainframe_resolveCopybook`
- `mainframe_impactAnalysis`
- `mainframe_runMigrationScript`

`mainframe_runMigrationScript` invokes the pip-installed package. It does not
expect repository scripts in the user's project.

### Agent responses, artifacts, and preflight cache

The runner defaults to `responseMode: "summary"`, returning a compact envelope
with status, findings, counts, digests, `continuationAllowed`, and `nextActions`.
`responseMode: "preview"` adds bounded stdout/stderr excerpts. Complete stdout,
stderr, and result output is always spooled under
`.mainframe-toolkit/runs/<runId>` while the hard artifact limit is not exceeded.
Add `.mainframe-toolkit/` to the consumer repository's `.gitignore` unless run
evidence is intentionally committed.

Identical `migration_preflight` arguments reuse a cached envelope until COBOL,
copybook, JCL/PROC, configuration, artifact fingerprints, or explicit reindex
invalidate it. Agents should obey `continuationAllowed`, perform the listed
`nextActions`, and consume the full artifact instead of rerunning because a
preview was truncated.

`mainframeMigration.maxAgentResponseBytes` limits the serialized response only.
`mainframeMigration.maxArtifactBytes` defaults to a hard 100 MiB combined
stdout/stderr cap. Deprecated `mainframeMigration.maxOutputBytes` remains a
preview compatibility setting and does not terminate the process.

Preflight returns `environmentFacts` from schema-backed configuration such as:

```json
{
  "environment": {
    "cobolDialect": "Enterprise COBOL",
    "compiler": {
      "name": "IBM Enterprise COBOL",
      "version": "6.4",
      "options": ["RENT", "SSRANGE"]
    },
    "sourceFormat": "fixed",
    "runtime": "z/OS batch",
    "runtimeDependencies": {"DB2": "13"},
    "testCommands": ["./run-characterization-tests.sh"]
  }
}
```

Configured values are `KNOWN`. Missing values remain `UNKNOWN`; the toolkit
does not guess them or emit unrelated blockers. Packaged agent guidance allows
one targeted, capped search for each `UNKNOWN`, then requests user evidence.

## Configuration

`mainframe-migration.json` controls:

- Ordered primary and fallback COBOL, copybook, and JCL libraries.
- Source extensions and encoding.
- Known external programs, utilities, and copybooks.
- Generated TODO syntax.
- Physical-to-transport record representations.
- Target strategies and adapters under `targetCapabilities`.

Configuration is authoritative. The tools do not widen searches to guessed
directories when configured resolution fails.

### Capability discovery and target policy

Preflight uses two explicit stages. First, it deterministically discovers only
the capabilities evidenced by the selected JCL and its COBOL/files, then emits
the immutable `capabilities` inventory and `capabilityDigest`. Second, it applies
user-authored `targetCapabilities` policy; discovery never chooses a target
technology.

A concise policy can combine a class default, an exact discovered instance, and
a selector:

```json
{
  "targetCapabilities": {
    "schemaVersion": 1,
    "defaults": {
      "transform.sort_merge": {
        "strategy": "pyspark",
        "adapter": "builtin.pyspark_sort"
      }
    },
    "instances": {
      "storage.indexed_records:dataset:APP.ACCOUNTS": {
        "strategy": "relational_table",
        "adapter": "builtin.relational_keyed_store"
      }
    },
    "selectors": [
      {
        "id": "daily-bulk-loads",
        "capabilityClass": "load.bulk_records",
        "priority": 20,
        "match": {"job": "DAILY*"},
        "policy": {
          "strategy": "relational_bulk_load",
          "adapter": "builtin.bulk_load"
        }
      }
    ]
  }
}
```

Resolution precedence is exact instance, highest-priority matching selector,
capability-class default, then unresolved. Selector array order is irrelevant;
equal-priority selectors with different policies produce `BLOCK` instead of a
guess.

Discovery currently emits `orchestration.batch`, `transform.sort_merge`,
`load.bulk_records`, `storage.indexed_records`, `storage.sequential_records`,
`storage.versioned_generation`, `database.relational`, `transaction.online`,
`messaging.queue`, and `operations.audit`. For a selected JCL, COBOL SQL,
transaction, and queue evidence is limited to local programs in its transitive
CALL/literal dialect-link closure. Messaging requires an exact supported IBM MQ
CALL or CICS `READQ`/`WRITEQ`/`DELETEQ`; generic CALLs and generic CICS blocks do
not qualify. Audit discovery groups `SYSOUT=*`, `SYSPRINT`, and `SYSOUT` DDs per
job step. `security.authorization` is available in policy/schema menus but no
instance is emitted until explicit RACF/security command evidence is parsed.

Built-in adapter IDs are `builtin.relational_keyed_store`,
`builtin.key_value_store`, `builtin.lakehouse_table`,
`builtin.fixed_width_storage`, `builtin.pyspark_sort`, `builtin.sql_sort`,
`builtin.bulk_load`, and `builtin.batch_orchestration`. Preflight records their
declared guarantees in each resolution. Third-party adapters use the
`mainframe_modernization_toolkit.adapters` entry-point group and must pin
`adapterVersion`; unavailable, ambiguous, incompatible, or mismatched adapters
block.

Framework-only descriptors are also available as
`builtin.relational_database_framework`, `builtin.online_transaction_framework`,
`builtin.messaging_queue_framework`, `builtin.audit_framework`, and
`builtin.authorization_framework`. They validate strategy compatibility and
declare only configuration/source-traceability guarantees; they have no provider
implementation, so `generationEligible` remains `false`.

An unresolved policy, or a selected strategy without an adapter, emits a `TODO`
and leaves that capability ineligible for adapter-backed generation. Unaffected
analysis continues, and reviewed relational IR can preserve the missing target
decision as a deterministic code-local TODO using the configured prefix.

The extension invokes `mainframe-toolkit` by default. If VS Code cannot find the
entry point, set `mainframeMigration.executablePath` to its absolute path. Find
it with the Python environment where the package was installed:

```bash
python -c "import shutil; print(shutil.which('mainframe-toolkit'))"
```

## Safety boundaries

- Generated skeletons and capsules are evidence, not complete translations.
- Synthetic fixtures validate ingestion and field boundaries only. They are not
  authoritative expected program outputs.
- Semantic equivalence requires outputs captured from the mainframe or another
  verified implementation.
- Physical binary layouts and text transport layouts are separate contracts.
- PySpark generation requires reviewed, executable relational IR; the compiler
  does not invent business semantics.

## Troubleshooting

### `mainframe-toolkit: command not found`

Use:

```bash
python -m mainframe_modernization_toolkit doctor --workspace .
```

Then configure the extension with the absolute entry-point path if needed.

### The agent tries to run `scripts/<tool>.py`

Upgrade the package, reinstall its bundled VSIX, and replace packaged workspace
customizations only after reviewing local changes:

```bash
python -m pip install --upgrade mainframe-modernization-toolkit
mainframe-toolkit vsix export --output mainframe-migration-toolkit.vsix
code --install-extension mainframe-migration-toolkit.vsix --force
mainframe-toolkit workspace init . --force
```

### The VSIX and Python package versions differ

```bash
mainframe-toolkit --version
mainframe-toolkit vsix verify
```

Export the VSIX from the same Python environment used by the extension tools.

## License

Apache-2.0.
