Metadata-Version: 2.4
Name: linti
Version: 0.6.2
Summary: A linter for TM1 TI scripts
Author-email: Florian Scherzberg <florian.scherzberg@deutschebahn.com>
License-Expression: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer[all]>=0.9.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pathspec>=0.12.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Dynamic: license-file

# Linti

## Description

A linter for TM1 TurboIntegrator (TI) code that enforces consistent formatting and best practices. Originally started as an internal project at Deutsche Bahn AG, it is now opened to the community. Contributions are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for details.

## Name Origin

The name "Linti" is a wordplay on "lint" and "TI" (TurboIntegrator). The "-i" ending is commonly used in German nicknames and was chosen intentionally as a small tribute to the project's German roots.

## Motivation

TurboIntegrator processes are the backbone of any TM1 application, yet there has never been a dedicated linter for TI code. Teams rely on manual code reviews and informal conventions that inevitably drift 
over time. We built Linti to close that gap — giving TM1 developers the same kind of automated quality checks that are standard in every other programming ecosystem.

We decided to open-source Linti because the TM1 community is relatively small. A linter only becomes truly useful when it reflects the collective experience of many teams, not just one. By publishing it, we hope to invite contributions from other TM1 practitioners and give back to a community that has always been generous with knowledge.

## Compatibility Notice

This project is an independent compatibility tool for TurboIntegrator (TI) scripts used in TM1.

It is not affiliated with, endorsed by, sponsored by, or maintained by IBM. TM1, Planning Analytics, and related product names are referenced for compatibility and identification purposes only.

## Feature

* Lexer
* Linter
* Rules
* Parser (AST)
* Formatter
* Provider-based input format support

## Supported Input Formats

### Fully Supported (all rules)

- Git-deploy process format (`.json` + linked `.ti`)
  - Reads metadata from JSON and code from `Code@Code.link`.
- PA-code format (`.ti`)
  - Uses `#SECTION Prolog|Metadata|Data|Epilog` and a trailing
    `#JSON_PROPERTIES` block for metadata.
- YAML TM1 process files (`.yaml`, `.yml`)
  - `!TM1py.ProcessObject` and `config.definition` formats.

### TI File Variants (partial support)

- Region-based `.ti` files (`#region Prolog|Metadata|Data|Epilog`)
  - Section-aware rules work (Prolog/Metadata/Data/Epilog context).
  - Rules that require metadata declarations (for example parameters/variables)
    are limited because plain `.ti` region files do not carry metadata blocks.
- Plain `.ti` files (no region markers)
  - Entire file is treated as Prolog.
  - Only Prolog-valid and section-independent rules are meaningful.

## Install via Pypi

The linter is available on PyPI and can be installed using `pip install linti`.

For a quick setup guide, see [GETTING_STARTED.md](GETTING_STARTED.md).

## Configuration

The linter can be configured using a `linti.yaml` configuration file. The file is automatically discovered and loaded from the same directory as the TI file being analyzed.

### Automatic Configuration Discovery

Place a `linti.yaml` file in your project. The linter searches upward from the file being linted until it finds a `linti.yaml`, reaches a project root (`.git`, `pyproject.toml`, `setup.cfg`, `setup.py`), or hits the filesystem root.

```bash
my-project/
├── linti.yaml              # Applies to all TI files below
├── processes/
│   ├── process1.ti
│   └── process2.ti
└── special/
    ├── linti.yaml          # Overrides for this directory
    └── process3.ti
```

When you run `linti processes/process1.ti`, the linter walks up and finds `my-project/linti.yaml`. Files in `special/` use their own local config instead.

### Custom Configuration File

You can also specify a custom configuration file:

```bash
linti process.ti --config custom-config.yaml
```

### Configuration Options

A typical `linti.yaml` file looks like this:

```yaml
rules:
  # F110 - Keyword Casing
  # Enforces consistent casing for keywords (IF, ENDIF, ELSE, WHILE, END)
  # Supported styles: uppercase, lowercase, camelcase
  keyword_casing:
    enabled: true
    style: uppercase

  # F310 - Block Indentation
  # Enforces indentation for IF/WHILE blocks and for continuation lines
  # size: number of spaces per indentation level
  # continuation_style: how a line that continues an earlier statement is
  #   indented - hanging (one level per open parenthesis), aligned (under the
  #   opening parenthesis), or ignore (leave hand-formatted lines alone)
  indentation:
    enabled: true
    size: 4
    continuation_style: hanging

  # F330 - Maximum Line Length
  # Flags lines over `limit` characters and rewraps them at argument or
  # operator boundaries, in the hanging style F310 enforces.
  max_line_length:
    enabled: true
    limit: 120

  # N110 - Variable Prefix Naming
  # Enforces TM1 naming conventions
  # - Numeric variables must start with 'n'
  # - String variables must start with 's'
  variable_prefix:
    enabled: true
    # Allow constants to start with 'c' (e.g., cRate, cMessage)
    # When enabled, constants may only be assigned once.
    allow_constant_prefix: false

  # C410 - Use Hierarchy-Aware Functions
  # Prefer hierarchy-aware functions (e.g. HierarchyElementExists,
  # ElementParent) over standard ones (e.g. DimensionElementExists, ELPAR).
  use_hierarchy_aware_functions:
    enabled: true
    # Base mode:
    # 'consistent' - either style is allowed, but mixing both in one file is reported
    # 'enforce'    - only hierarchy-aware functions are allowed
    mode: consistent

  # C430 - Do Not Use Undocumented Functions
  # Reports TI functions that IBM does not document or support (e.g.
  # DimensionElementInsertByAlias, LockOn, Hex).
  do_not_use_undocumented_functions:
    enabled: true
    # Functions the project knowingly relies on (case-insensitive).
    # Listed here they are never reported; use `# noqa: C430` for one-off cases.
    # allowed_functions:
    #   - DimensionElementInsertByAlias

  # C510 - Function Version Compatibility
  # Reports TI functions that are not available on the targeted PA/TM1 version.
  # Disabled by default: the target depends on your deployment strategy.
  function_version_compatibility:
    enabled: true
    # Optional override of the top-level `target_version` (see below):
    # 'CompatibleWithV11AndV12' - only functions available in both are allowed
    # 'V11'                     - flags functions introduced in v12
    # 'V12'                     - flags functions unsupported in v12
    mode: CompatibleWithV11AndV12

  # X130 - No Hardcoded Secrets
  # Reports secret-looking variables fed from a hardcoded string literal
  # (sPassword = 'letmein') or read out of a cube
  # (sPassword = CellGetS(...)). Neither the value nor the cube
  # coordinates are ever printed.
  # The value is taken from constant evaluation, so a literal split across
  # a concatenation (sPassword = 'let' | 'mein') or routed through another
  # variable is caught too. Half-known values (sPassword = 'prefix_' | pDyn)
  # are not reported - the secret itself may be the dynamic part.
  hardcoded_secret:
    enabled: true
    # Which names count as secret-looking (matched case-insensitively as
    # substrings of the variable name):
    # 'relaxed'  - password, passwd, pwd, secret
    # 'standard' - and apikey, api_key, token, credential
    # 'strict'   - and key, auth, cert, salt, signature
    # 'custom'   - only the names listed under `secret_names`
    mode: standard
    # Extra name fragments, added on top of the preset above
    # (and the whole list when mode is 'custom'):
    # secret_names:
    #   - kennwort
    # Accept credentials kept in a cube (CellGetS, AttrS, ...). Off by
    # default: the TM1 data directory can be encrypted, but rarely is,
    # so anyone with file access reads the value.
    allow_secrets_in_cubes: false
```

### Generic Processes

Some rules treat *generic* (templated) processes more strictly. A process is
considered generic when its name starts with one of the prefixes in the
top-level `generic_prefixes` setting. This single definition is shared by all
rules that care about it (currently `D110` Docstring Region and `C410`
Use Hierarchy-Aware Functions — generic processes are always held to C410's
`enforce` mode regardless of its base `mode`).

```yaml
# Top-level (shared across rules)
generic_prefixes:
  - '}core.'

rules:
  docstring_region:
    enabled: true
```

> **Deprecation:** `generic_prefixes` used to be configured under the docstring
> rule (`rules.docstring_region.generic_prefixes`). That still works on its own,
> but emits a warning — move it to the top-level `generic_prefixes` setting
> instead. Setting **both** the top-level and the per-rule value is a config
> error: linting fails immediately with a message telling you to remove the
> deprecated `rules.docstring_region.generic_prefixes` key.

`rules.one_space_before_equals` has been removed along with the Equals Spacing
rule. If it is still present in an older config, linti warns and ignores it.
Its old ID `F210` is not claimed by any live rule.

### Target Version

Which Planning Analytics / TM1 version your code has to run on is a project-wide
fact, so it lives in the top-level `target_version` setting. Version-aware rules
(currently `C510` Function Version Compatibility) read it from there:

```yaml
# Top-level (shared across rules)
target_version: both   # v11 | v12 | both

rules:
  function_version_compatibility:
    enabled: true
```

`target_version` only *supplies* the version — it never enables a rule. `C510` is
opt-in and stays silent until you set `enabled: true`.

A rule may override the shared value, either with its own `mode` or with a
per-rule `target_version`. Both are accepted; `mode` wins if you set both:

```yaml
target_version: v12

rules:
  function_version_compatibility:
    enabled: true
    mode: V11            # this rule checks against v11 regardless
```

Each key has its own spelling, and each accepts **only** its own:

| Key | Accepted values |
|-----|-----------------|
| `target_version` (top-level or per-rule) | `v11`, `v12`, `both` |
| `mode` (per-rule) | `V11`, `V12`, `CompatibleWithV11AndV12` |

`both` and `CompatibleWithV11AndV12` select the same behaviour, but each spelling
belongs to one key only — `mode: both` and `target_version: V11` are rejected
with a config error. With neither key set, `C510` defaults to
`CompatibleWithV11AndV12`.

### Excluding Files from Linting

The top-level `exclude_paths` key lists files, directories, or glob patterns to
skip during discovery:

```yaml
exclude_paths:
  - generated/
  - vendor/
  - "**/archive/*.ti"
```

- Entries may be individual files (`processes/test.ti`), directories
  (`generated/`), or glob patterns (`**/archive/*.ti`) — all use one common
  matcher.
- A bare name (no `/`, e.g. `generated`) excludes that file or directory
  *anywhere* in the tree (gitignore-style). A pattern containing a `/` (e.g.
  `processes/test.ti`) is anchored to the discovery root — prefix it with
  `**/` to match at any depth.
- CLI `--exclude-path` values **extend** this list rather than replacing it, so
  the effective exclusions are the configured ones plus the CLI ones (duplicates
  removed).

### Input Limits

To stay robust on large or untrusted process files, linti bounds two things via
top-level settings (safe defaults; normal files are unaffected):

```yaml
# Reject files larger than this many bytes before reading them into memory.
max_file_size: 10485760   # 10 MB (default)

# Cap on control-flow (IF/WHILE) nesting depth. Deeper nesting is reported as
# an S900 diagnostic instead of being parsed unboundedly.
max_nesting_depth: 150    # default

# Cap on how many distinct values the constant evaluation index tracks per
# variable (across IF/ELSE branches) before treating it as unknown. Used by
# value-aware rules such as X210 and C410.
max_values_per_variable: 8           # default
```

- A file above `max_file_size` fails with a clear error rather than being read.
- A procedure nested beyond `max_nesting_depth` produces a single `S900`
  diagnostic (`Maximum nesting depth (N) exceeded`) for that procedure and
  linting continues. `S900` is a built-in parser diagnostic, not a configurable
  rule, so it does not appear in `linti explain` / `ALL_RULES.md`.
- `max_values_per_variable` bounds cross-section constant evaluation: once a variable
  could hold more than this many distinct literal values, it degrades to
  "unknown" so value-aware rules stay conservative rather than tracking an
  unbounded set.

### All rules

For a complete reference of all linting rules with detailed configuration examples and usage instructions, see [ALL_RULES.md](ALL_RULES.md).

### Disabling Rules

To disable a specific rule, set `enabled: false`:

```yaml
rules:
  keyword_casing:
    enabled: false
  variable_prefix:
    enabled: true
```

### Inline Suppression (`noqa`)

You can suppress specific rules directly in TI code using `# noqa` comments.
TI uses `#` for comments, so the syntax feels natural.

#### Trailing comment — suppress current line

```
nVar=1;  # noqa: F220
```

#### Standalone comment — suppress the next code line

```
# noqa: F110
if(nVar = 1);
```

#### Procedure-level — first comment before any code suppresses the entire file/procedure

```
# noqa: X110, C310
ExecuteCommand(sCmd, 1);
RunProcess(pProcess);
```

#### Region — suppress a block of lines

```
# noqa-begin: F110
if(nVar = 1);
endif;
# noqa-end: F110
```

Multiple rule IDs can be combined with commas: `# noqa: F110, N110, C220`

Deprecated rule IDs (see [Rule ID migration](#rule-id-migration)) still work in
`noqa` comments for one deprecation cycle; linti resolves them to the canonical
ID and prints a warning telling you which ID to use instead.

## CLI Usage

### Basic Usage

```bash
# Lint a single file
linti process.ti

# Lint with additional debug output
linti process.ti --tokens
linti process.ti --ast
linti process.ti --tokens --ast

# Lint with custom configuration
linti process.ti --config my-config.yaml

# Auto-fix issues
linti process.ti --auto-fix

# Lint a region-based TI file
linti process-regions.ti --auto-fix

# Lint a YAML ProcessObject file
linti process.yaml --auto-fix

# Lint a Git-deploy process (JSON + linked .ti)
linti process.json --auto-fix

# Lint a PA-code process (#SECTION + #JSON_PROPERTIES)
linti pa-code.ti --auto-fix

# Lint all process files in a directory (searched recursively)
linti processes/ --auto-fix
```

### Input Paths and Globs

The positional `<path>` argument accepts individual files, directories, and glob
patterns — and you can pass several at once. linti performs the glob expansion
itself, so patterns behave the same across shells and platforms (quote them so
your shell does not expand them first).

```bash
# A single file, a directory, or a glob
linti process.ti
linti processes/
linti "*.ti"
linti "processes/**/*.ti"     # ** recurses into sub-directories
linti processes/*.json

# Several paths (files, directories, and globs) expanded together
linti processes/ "*.yaml" other/process.ti
```

A file reached through more than one input (overlapping paths or globs) is
**linted only once**.

### Excluding Paths

Skip files, directories, or glob patterns with the repeatable `--exclude-path`
option:

```bash
linti . \
    --exclude-path generated \
    --exclude-path vendor \
    --exclude-path "**/archive/*.ti"
```

Exclusions can also be configured in `linti.yaml` via the top-level
[`exclude_paths`](#excluding-files-from-linting) key. CLI `--exclude-path`
values **extend** the configured list (they never replace it), and duplicates
are ignored.

### Command Help

```bash
linti --help       # Overview of all commands and global options
linti lint --help  # All linting arguments and options (--auto-fix, --select, ...)
```

`lint` is the default command — `linti process.ti` is a shortcut for `linti lint process.ti`.

### Selecting Specific Rules

Use the `--select` option to run only specific rules or groups of rules:

```bash
# Run a specific rule
linti process.ti --select F110

# Run all rules in a category (e.g., all Format rules)
linti process.ti --select F

# Run all rules in a subcategory (e.g., all F2xx - Whitespace rules)
linti process.ti --select F2

# Run multiple rules or groups (comma-separated)
linti process.ti --select F110,C220
linti process.ti --select F,N1,C3

# Combining with other options
linti process.ti --select F --auto-fix
linti process.ti --select N,C --tokens
```

**Selection patterns:**
- `F`, `N`, `D`, `C`, `X` – Select all rules in a category
- `F1`, `F2`, `F3`, `N1`, `N2`, `D1`, `C1`, `C2`, `C3`, `C4`, `X1`, `X2` – Select all rules in a subcategory
- `F110`, `D110`, `C220` – Select a specific rule

A deprecated rule ID is accepted as a full ID (e.g. `--select S220` runs `C220`)
and warns; group prefixes are matched against canonical IDs only.

### Listing Rules

Use the `explain` subcommand to inspect available rules and view detailed guidance for one rule.

```bash
# List all rules with short description and auto-fix support
linti explain

# Explain a specific rule in detail
linti explain F110
```


## Rule Groups

Rules are organized by topic, each with a hierarchical numbering scheme. The
first letter is the category; the hundreds digit is a logical subcategory.

| Letter | Category |
|--------|----------|
| `F` | Formatting |
| `N` | Naming |
| `D` | Documentation |
| `C` | Code Quality |
| `X` | External Interactions |
| `E` | Error |

### Formatting Rules (F1xx, F2xx, F3xx)
Formatting and code style rules:

- **F1xx - Casing**: Keyword capitalization
  - `F110` - Keyword Casing

- **F2xx - Whitespace**: Whitespace and spacing requirements
  - `F220` - Whitespace Around Operators
  - `F230` - Whitespace After Comma
  - `F240` - No Space Before Semicolon
  - `F250` - One Space Inside Parentheses
  - `F260` - No Multiple Spaces
  - `F270` - No Trailing Whitespace

- **F3xx - Layout**: Indentation, line breaks, and code layout
  - `F310` - Block Indentation
  - `F320` - One Statement Per Line
  - `F330` - Maximum Line Length

### Naming Rules (N1xx, N2xx)
Variable and parameter naming conventions:

- **N1xx - Variables**: Variable prefix and casing conventions
  - `N110` - Variable Prefix Naming
  - `N120` - Variables Consistent Casing

- **N2xx - Inputs**: Naming conventions for parameters and data sources
  - `N210` - Parameter Naming
  - `N220` - Data Source Variable Naming

### Documentation Rules (D1xx)
Documentation and process docstring validation:

- **D1xx - Documentation**: Required docstring regions and headers
  - `D110` - Docstring Region

### Code Quality Rules (C1xx, C2xx, C3xx, C4xx, C5xx)
Control flow, mutability, and TM1 best practices:

- **C1xx - Control Flow**: Process execution and control flow patterns
  - `C110` - Empty Block
  - `C120` - Conditional Control Flow
  - `C130` - ItemSkip Block Usage
  - `C140` - Unreachable Code

- **C2xx - Variables**: Variable mutability and assignment constraints
  - `C210` - Read-only Parameters and Variables
  - `C220` - Single-assignment Constants

- **C3xx - Process Design**: How processes call one another
  - `C310` - Literal Process Calls

- **C4xx - TM1 Best Practices**: Idiomatic TM1 API usage
  - `C410` - Use Hierarchy-Aware Functions
  - `C430` - Do Not Use Undocumented Functions

- **C5xx - Version Compatibility**: Compatibility with a target PA/TM1 version
  - `C510` - Function Version Compatibility

### External Interactions Rules (X1xx, X2xx)
Interactions with systems outside the TI process:

- **X1xx - Security**: Security-sensitive operations
  - `X110` - No ExecuteCommand
  - `X120` - ODBCOpen Password Parameter
  - `X130` - No Hardcoded Secrets

- **X2xx - Performance**: Cost of external data access
  - `X210` - Filter ODBC Rows in SQL

### Error Rules (E1xx)
Parse and syntax errors that reduce linting coverage:

- **E1xx - Parsing**: Statements the parser could not understand
  - `E110` - Unparseable Statement

Use `linti explain` to list all rules or `linti explain <RULE_ID>` for detailed information about a specific rule.

### Rule ID migration

The former generic **Semantic (S)** category was split into **Code Quality (C)**
and **External Interactions (X)**, and a few rules were renumbered so each
subcategory describes its topic. **Old IDs keep working for one deprecation
cycle** wherever a rule is referenced by ID — `--select`, `# noqa` comments,
and `linti explain`. Using one resolves to the canonical rule and prints:

```
⚠  Rule ID S220 is deprecated. Use C220 instead.
```

Diagnostics always report the **canonical (new)** ID.

| Old | New | Rule |
|-----|-----|------|
| `N230` | `N120` | Variables Consistent Casing |
| `D410` | `D110` | Docstring Region |
| `S130` | `C110` | Empty Block |
| `S110` | `C120` | Conditional Control Flow |
| `S120` | `C130` | ItemSkip Block Usage |
| `S210` | `C210` | Read-only Parameters and Variables |
| `S220` | `C220` | Single-assignment Constants |
| `S310` | `C310` | Literal Process Calls |
| `S410` | `C410` | Use Hierarchy-Aware Functions |
| `S320` | `X110` | No ExecuteCommand |
| `S330` | `X120` | ODBCOpen Password Parameter |
| `S340` | `X210` | Filter ODBC Rows in SQL |

Every other rule (all Formatting and Naming IDs, plus `E110`) keeps the ID it
already had.

> Configuration in `linti.yaml` is keyed by rule *name* (e.g. `keyword_casing`,
> `constant_assignment`), not by rule ID, so no config changes are needed.

### Auto-Fix Feature

The linter offers an automatic fix for many rules whenever the fix is safe to apply. In the linting issue report, every issue that can be fixed automatically is marked with a 🔧 indicator, so you can see at a glance which rules will be resolved. Apply the fixes with the `--auto-fix` flag.

### Multi-line Statements

A statement that does not fit on one line is laid out in a **hanging indent**: the opening parenthesis stays at the end of its line, everything inside it is indented one level deeper, and the line that closes it returns to the level of the line that opened it.

```ti
IF( nA = 1 );
    sValue = CellGetS(
        'Cube',
        'Element'
    );
ENDIF;
```

Long conditions and concatenations break before their operators:

```ti
IF(
    nA = 1
    & nB = 2
);
```

`F310` enforces this layout and `F330` produces it when a line exceeds the limit, so the two never disagree.

Two things auto-fix will not do: it never joins short lines back together, and it never rewraps a statement that contains a comment or a multi-line string literal — moving either could change what the code means. Such lines are reported without a 🔧.

If your project already wraps arguments by hand and you would rather keep that layout, set `continuation_style` to `aligned` (line wrapped content up under the opening parenthesis) or `ignore` (leave continuation lines alone entirely):

```yaml
rules:
  indentation:
    continuation_style: ignore
```

## TM1 Block Context Awareness

The linter understands TM1's four execution blocks when procedure sections are available (YAML, Git JSON+TI, PA-code `.ti`, or region-based `.ti`):

- **Prolog**: Initialization and setup code
- **Metadata**: Dimension/hierarchy manipulation code
- **Data**: Record-by-record data processing
- **Epilog**: Finalization and cleanup code

Rules can use block context to enforce block-specific requirements. For example:
- The ItemSkip rule (`C130`) only allows `ItemSkip()` in the Metadata or Data blocks
- Rules could require certain variable naming patterns based on the block

Metadata-dependent rules (for example checks based on declared Parameters/Variables)
require formats that provide metadata (`.yaml`, Git JSON+TI, PA-code).

For plain `.ti` files without `#region` sections, the full file is treated as **Prolog**.

## License

This project is licensed under the **Apache License 2.0**. See the [LICENSE](LICENSE) file for details.

The full license text is also available at the [Apache Software Foundation](https://www.apache.org/licenses/LICENSE-2.0).
