Metadata-Version: 2.4
Name: informatica-analyzer
Version: 0.3.0
Summary: Informatica PowerCenter XML workflow analyzer
Author: Rahul Barna
License-Expression: MIT
Project-URL: Homepage, https://github.com/rahulbarna00/informatica-analyzer
Project-URL: Repository, https://github.com/rahulbarna00/informatica-analyzer
Project-URL: Issues, https://github.com/rahulbarna00/informatica-analyzer/issues
Keywords: informatica,powercenter,xml,workflow,etl,data-engineering
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Markup :: XML
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Dynamic: license-file

# informatica-analyzer

`informatica-analyzer` is a lightweight Python library and command-line tool for analyzing Informatica PowerCenter XML workflow exports.

It extracts workflow, session, mapping, source, target, lookup, Source Qualifier, SQL, truncate, connection, and flat-file metadata from Informatica XML files. The extracted metadata can be written as JSON and converted into CSV reports for auditing, migration analysis, documentation, and downstream automation.

---

## Features

* Analyze Informatica PowerCenter XML export files
* Extract workflows, sessions, and mappings
* Identify source, target, and lookup objects
* Extract Source Qualifier SQL queries
* Extract Source Qualifier filters and user-defined joins
* Extract pre-SQL and post-SQL statements
* Detect target truncate settings
* Capture database and connection metadata
* Capture flat-file and reject-file metadata where available
* Generate structured JSON output
* Generate CSV reports
* Use as both a CLI tool and a Python library
* Extend with custom validation checks

---

## Installation

Install from PyPI:

```bash
pip install informatica-analyzer
```

Verify the installation:

```bash
informatica-analyzer --help
```

---

## Requirements

* Python 3.10 or newer
* Informatica PowerCenter XML export files

The package is intentionally lightweight and does not require a database connection.

---

## CLI Usage

The main CLI command is:

```bash
informatica-analyzer
```

### Analyze XML files and generate JSON and CSV output

```bash
informatica-analyzer \
  --xml-glob "inputs/*.xml" \
  --json-out-dir outputs/json \
  --csv-out-dir outputs
```

This command:

1. Reads all XML files matching `inputs/*.xml`
2. Writes one JSON file per XML file into `outputs/json`
3. Generates CSV reports in `outputs`

---

### Generate CSV reports from existing JSON files

```bash
informatica-analyzer \
  --json-glob "outputs/json/*.json" \
  --csv-out-dir outputs
```

Use this when XML extraction has already been done and you only want to regenerate CSV reports.

---

### Analyze XML and combine with existing JSON

```bash
informatica-analyzer \
  --xml-glob "inputs/*.xml" \
  --json-glob "outputs/json/*.json" \
  --json-out-dir outputs/json \
  --csv-out-dir outputs
```

This is useful when you want to process new XML files and include already-generated JSON files in the final CSV reports.

---

### Generate compact JSON

```bash
informatica-analyzer \
  --xml-glob "inputs/*.xml" \
  --json-out-dir outputs/json \
  --compact-json
```

By default, JSON output is pretty-printed. Use `--compact-json` to reduce file size.

---

### Explode detail rows

```bash
informatica-analyzer \
  --xml-glob "inputs/*.xml" \
  --json-out-dir outputs/json \
  --csv-out-dir outputs \
  --explode-details
```

By default, the detailed CSV report groups multiple source, target, lookup, and query values into one row per workflow-mapping pair.

With `--explode-details`, the report creates expanded rows for detailed spreadsheet analysis.

---

### Change log level

```bash
informatica-analyzer \
  --xml-glob "inputs/*.xml" \
  --json-out-dir outputs/json \
  --csv-out-dir outputs \
  --log-level INFO
```

Supported log levels:

```text
DEBUG
INFO
WARNING
ERROR
```

---

## Python Library Usage

You can also use `informatica-analyzer` directly from Python.

### Analyze one XML file

```python
from informatica_analyzer import analyze_xml

metrics = analyze_xml("inputs/workflow_export.xml")

print(metrics["summary"])
print(metrics["workflows"])
```

---

### Write metrics JSON for one XML file

```python
from informatica_analyzer import write_metrics_json

json_path = write_metrics_json(
    "inputs/workflow_export.xml",
    "outputs/json/workflow_export.json",
)

print(json_path)
```

---

### Analyze multiple XML files

```python
from informatica_analyzer import analyze_xml_files

result = analyze_xml_files(
    [
        "inputs/workflow_export_1.xml",
        "inputs/workflow_export_2.xml",
    ],
    json_out_dir="outputs/json",
    csv_out_dir="outputs",
)

print(result["json_paths"])
print(result["csv_paths"])
```

---

### Generate JSON only

```python
from informatica_analyzer import analyze_xml_files

result = analyze_xml_files(
    ["inputs/workflow_export.xml"],
    json_out_dir="outputs/json",
    csv_out_dir=None,
)

print(result["json_paths"])
```

---

### Build CSV rows in memory

```python
from informatica_analyzer import build_csv_rows_from_json

rows = build_csv_rows_from_json(
    ["outputs/json/workflow_export.json"]
)

workflow_mapping_counts = rows["workflow_mapping_counts"]
workflow_mapping_details = rows["workflow_mapping_details"]
workflow_mapping_count = rows["workflow_mapping_count"]

print(workflow_mapping_details[:5])
```

---

### Use the extractor directly

```python
from pathlib import Path
from informatica_analyzer import InformaticaXmlMetricsExtractor

extractor = InformaticaXmlMetricsExtractor(Path("inputs/workflow_export.xml"))
metrics = extractor.parse()

print(metrics["details_grouped_by_mapping_name"].keys())
```

---

## Output Files

A typical run produces:

```text
outputs/
├── json/
│   ├── workflow_export_1.json
│   └── workflow_export_2.json
├── workflow_mapping_counts.csv
├── workflow_mapping_details.csv
└── workflow_mapping_count.csv
```

---

## JSON Output

Each XML file produces one JSON file.

Example top-level structure:

```json
{
  "input_file": "inputs/workflow_export.xml",
  "summary": {},
  "workflows": [],
  "details_grouped_by_mapping_name": {},
  "mappings": [],
  "lists": {}
}
```

### Summary

The `summary` block contains high-level counts:

```json
{
  "number_of_workflows": 1,
  "number_of_mappings_in_workflows": 3,
  "number_of_sources_in_those_mappings": 8,
  "number_of_targets_in_those_mappings": 4,
  "number_of_lookups_in_those_mappings": 2
}
```

### Workflows

The `workflows` block contains workflow-level metadata:

```json
[
  {
    "name": "wf_customer_load",
    "number_of_mappings": 2,
    "mapping_names": [
      "m_customer_stage",
      "m_customer_target"
    ],
    "sessions": [
      {
        "workflow_name": "wf_customer_load",
        "session_name": "s_customer_stage",
        "mapping_name": "m_customer_stage"
      }
    ]
  }
]
```

### Mapping details

The `details_grouped_by_mapping_name` block contains mapping-level metadata:

```json
{
  "m_customer_stage": {
    "mapping_name": "m_customer_stage",
    "counts": {
      "sources": 1,
      "targets": 1,
      "lookups": 0,
      "source_qualifier_sql_queries": 1,
      "source_qualifier_filters": 0,
      "source_qualifier_user_defined_joins": 0,
      "pre_sql": 0,
      "post_sql": 0,
      "target_truncate_flags": 1
    },
    "source_table_names": ["CUSTOMER_SRC"],
    "target_table_names": ["CUSTOMER_STG"],
    "lookup_table_names": [],
    "source_qualifier": {
      "sql_queries": [],
      "source_filters": [],
      "user_defined_joins": [],
      "details": []
    },
    "pre_sql": [],
    "post_sql": [],
    "target_truncate": []
  }
}
```

---

## CSV Reports

### `workflow_mapping_counts.csv`

One row per workflow-mapping pair.

| Column         | Description              |
| -------------- | ------------------------ |
| `wf_name`      | Workflow name            |
| `mapping_name` | Mapping name             |
| `n_source`     | Number of source objects |
| `n_target`     | Number of target objects |
| `n_lookup`     | Number of lookup objects |

---

### `workflow_mapping_details.csv`

Detailed workflow and mapping metadata.

| Column               | Description                   |
| -------------------- | ----------------------------- |
| `wf_name`            | Workflow name                 |
| `mapping_name`       | Mapping name                  |
| `s_database`         | Source database or connection |
| `s_table`            | Source table                  |
| `s_flat_file_name`   | Source flat-file name         |
| `s_flat_file_path`   | Source flat-file path         |
| `t_database`         | Target database or connection |
| `t_table`            | Target table                  |
| `t_flat_file_name`   | Target flat-file name         |
| `t_flat_file_path`   | Target flat-file path         |
| `t_reject_file_name` | Target reject file name       |
| `t_reject_file_path` | Target reject file path       |
| `l_table`            | Lookup table                  |
| `sq_query`           | Source Qualifier query        |

---

### `workflow_mapping_count.csv`

One row per workflow.

| Column          | Description                                    |
| --------------- | ---------------------------------------------- |
| `workflow_name` | Workflow name                                  |
| `mapping_count` | Number of unique mappings used by the workflow |

---

## Flat-File Metadata

For mappings that use flat files, the analyzer captures flat-file metadata where available in the Informatica session metadata.

Captured fields include:

* `flat_file_directory`
* `flat_file_name`
* `flat_file_path`
* `reject_file_directory`
* `reject_file_name`
* `reject_file_path`

Example:

```json
{
  "mapping_name": "m_customer_file_load",
  "instance_name": "CUSTOMER_FILE_SRC",
  "table_name": "CUSTOMER_FILE_SRC",
  "object_type": "source",
  "flat_file_directory": "/data/inbound/customer",
  "flat_file_name": "customer.csv",
  "flat_file_path": "/data/inbound/customer/customer.csv"
}
```

---

## Checks

Checks are optional validation utilities that can be built on top of extracted metadata.

The package includes basic file-data checks.

### Check whether a file exists and has data

```python
from pathlib import Path
from informatica_analyzer.checks.file_data import check_file_has_data

result = check_file_has_data(Path("/data/inbound/customer/customer.csv"))

print(result.exists)
print(result.has_data)
print(result.size_bytes)
print(result.reason)
```

Possible result:

```python
FileDataCheckResult(
    path="/data/inbound/customer/customer.csv",
    exists=True,
    has_data=True,
    size_bytes=2048,
    reason=None,
)
```

---

### Check many files

```python
from pathlib import Path
from informatica_analyzer.checks.file_data import check_files_have_data

paths = [
    Path("/data/inbound/customer/customer.csv"),
    Path("/data/inbound/order/order.csv"),
]

results = check_files_have_data(paths)

for result in results:
    print(result.path, result.has_data, result.reason)
```

---

## Extending the Package

New validation or audit logic should be added under:

```text
src/informatica_analyzer/checks/
```

For example:

```text
checks/file_data.py
checks/source_target_files.py
checks/sql_rules.py
checks/connection_rules.py
checks/truncate_rules.py
```

The extractor should focus on reading Informatica XML metadata.

Checks should answer validation questions such as:

* Does the source file exist?
* Does the target file have data?
* Is truncate enabled?
* Is a connection allowed?
* Does a SQL query contain a prohibited pattern?

This keeps extraction and validation separate.

---

## Development

Clone the repository and install in editable mode:

```bash
git clone <your-repository-url>
cd informatica-analyzer
python -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
```

On Windows PowerShell:

```powershell
git clone <your-repository-url>
cd informatica-analyzer
python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install -e ".[dev]"
```

---

## Run Tests

```bash
python -m pytest -q
```

Run one test file:

```bash
python -m pytest tests/test_xml_utils.py -q
```

---

## Project Structure

```text
informatica-analyzer/
├── pyproject.toml
├── README.md
├── src/
│   └── informatica_analyzer/
│       ├── __init__.py
│       ├── api.py
│       ├── cli.py
│       ├── extractor.py
│       ├── models.py
│       ├── pipeline.py
│       ├── xml_utils.py
│       ├── checks/
│       │   ├── __init__.py
│       │   └── file_data.py
│       └── reporting/
│           ├── __init__.py
│           └── csv_reports.py
└── tests/
    └── test_xml_utils.py
```

---

## Troubleshooting

### `informatica-analyzer: command not found`

Install the package in the current Python environment:

```bash
python -m pip install -e .
```

Then verify:

```bash
informatica-analyzer --help
```

---

### `ModuleNotFoundError: No module named 'informatica_analyzer'`

You may be using a different Python environment.

Check:

```bash
which python
python -m pip --version
```

Then reinstall:

```bash
python -m pip install -e .
```

---

### `pytest: command not found`

Install development dependencies:

```bash
python -m pip install -e ".[dev]"
```

Or install pytest directly:

```bash
python -m pip install pytest
```

Then run:

```bash
python -m pytest -q
```

---

### Empty CSV reports

Possible causes:

* The XML export does not contain workflow metadata
* Sessions are not linked to mappings
* The XML export contains partial repository metadata
* The wrong XML files were passed to `--xml-glob`

Start by inspecting the generated JSON file:

```bash
cat outputs/json/your_file.json
```

Check these keys:

```json
{
  "summary": {},
  "workflows": [],
  "details_grouped_by_mapping_name": {}
}
```

---

## License

This project is licensed under the MIT License.

See the [LICENSE](LICENSE) file for details.
