Metadata-Version: 2.4
Name: lambda-graphs
Version: 0.2.8
Summary: Generate combined multi-code view graphs
Home-page: https://github.com/PrinOrange/lambda-graphs
License: Apache-2.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: networkx==2.6.3
Requires-Dist: tree-sitter>=0.22
Requires-Dist: tree-sitter-c
Requires-Dist: tree-sitter-cpp
Requires-Dist: tree-sitter-java
Requires-Dist: tree-sitter-javascript
Requires-Dist: deepdiff>=7.0.0
Requires-Dist: pydot==1.4.1
Requires-Dist: typer>=0.9.0
Requires-Dist: loguru==0.6.0
Requires-Dist: setuptools>=69.0; python_version >= "3.12"
Provides-Extra: dev
Requires-Dist: tqdm; extra == "dev"
Requires-Dist: pqdm; extra == "dev"
Requires-Dist: loguru; extra == "dev"
Requires-Dist: GitPython; extra == "dev"
Requires-Dist: pandas; extra == "dev"
Dynamic: license-file

# lambda-graphs: Multi-View Code Representation Tools

lambda-graphs aims to generate combined multi-code view graphs that can be used with various types of machine learning models (sequence models, graph neural networks, etc).

## Overview

`lambda-graphs` is a CLI tool and Python library for generating code representations (AST, CFG, DFG) from source code. It supports **C**, **C++**, **Java**, and **JavaScript**, at both method-level and file-level granularity.

- **AST** (Abstract Syntax Tree)
- **CFG** (Control Flow Graph)
- **DFG** (Data Flow Graph)
- **Combined graphs** (any combination of the above)

`lambda-graphs` provides both a CLI and a **Python API** so you can generate graphs programmatically without shelling out. It is designed to be easily extendable to various programming languages. This is primarily because we use [tree-sitter](https://tree-sitter.github.io/tree-sitter/), a highly efficient incremental parser that supports over 40 languages.

---
## Setup

**1. Create a conda environment:**
```console
conda create -n lambda-graphs python=3.11
```

**2. Activate the environment:**
```console
conda activate lambda-graphs
```

**3. Install the package in development mode:**
```console
pip install -e .
```

**4. Install GraphViz (Optional - for visualization):**

GraphViz is only required if you want to generate DOT, PNG, or SVG output files.

**Ubuntu/Debian:**
```console
sudo apt install graphviz
```

**MacOS:**
```console
brew install graphviz
```

**Windows:**
Download from [graphviz.org](https://graphviz.org/download/)

---
## Generating Graphs

### Using CLI

After setup, use the `lambda-graphs` command directly.

The attributes and options supported by the CLI are well documented and can be viewed by running:
```console
lambda-graphs --help
```

**Single File Analysis:**

Generate a combined CFG and DFG graph for a C++ file:
```console
lambda-graphs --lang "cpp" --code-file ./test.cpp --graphs "cfg,dfg"
```

Generate an AST for a C file with output in JSON format:
```console
lambda-graphs --lang "c" --code-file ./example.c --graphs "ast" --output "json"
```

**Folder Analysis (Multi-file Projects):**

lambda-graphs can analyze entire projects by combining multiple source files from a folder:

```console
lambda-graphs --lang "c" --code-folder ./project/src --graphs "cfg,dfg" --output "json"
```

This will:
1. Recursively scan the folder for all `.c` and `.h` files
2. Combine them into a single temporary file (preserving includes, declarations, definitions)
3. Generate the requested codeviews from the combined source
4. Output results to the `output/` directory

You can customize the combined output file name:
```console
lambda-graphs --lang "cpp" --code-folder ./mylib --combined-name "myproject" --graphs "ast,cfg"
```

**Inline Code Analysis:**

You can also analyze code snippets directly without a file:
```console
lambda-graphs --lang "c" --code "int main() { int x = 5; return x; }" --graphs "ast,cfg"
```

**Additional CLI Options:**

| Option | Description |
|--------|-------------|
| `--output` | Output format: `json`, `dot`, `svg`, or `all` (dot generates PNG; svg generates SVG). Default: `dot` |
| `--collapsed` | Collapse duplicate variable nodes into a single node in AST |
| `--last-def` | Add last definition information to DFG edges (shows where variables were last defined) |
| `--blacklisted` | Comma-separated list of AST node types to exclude from the graph |

**Flag-Codeview Compatibility:**

| Flag | AST | CFG | DFG |
|------|:---:|:---:|:---:|
| `--collapsed` | ✓ | ✗ | ✗ |
| `--blacklisted` | ✓ | ✗ | ✗ |
| `--last-def` | ✗ | ✗ | ✓ |
| `--last-use` | ✗ | ✗ | ✓ |

**Examples:**

```console
# Generate all output formats (DOT, JSON, PNG)
lambda-graphs --lang "c" --code-file test.c --graphs "cfg" --output "all"

# Collapse duplicate variable nodes in DFG
lambda-graphs --lang "cpp" --code-file test.cpp --graphs "ast" --collapsed

# Add last definition tracking to DFG
lambda-graphs --lang "c" --code-file test.c --graphs "dfg" --last-def

# Exclude specific AST node types
lambda-graphs --lang "c" --code-file test.c --graphs "ast,cfg" --blacklisted "comment,string_literal"
```

### Using the Python API

You can also use `lambda-graphs` as a library to get graph objects directly (no file I/O):

```python
from lambda_graphs import generate

# -- Generate from a code string -------------------------------------------
result = generate(
    "cpp",
    code="int main() { int x = 5; return x; }",
    graphs=["ast", "cfg", "dfg"],
)

# Each graph is a networkx.MultiDiGraph
print(result.ast.nodes(data=True))
print(result.cfg.nodes(data=True))
print(result.dfg.nodes(data=True))
print(result.combined.nodes(data=True))

# Graph-level metadata (the "graph" key in JSON output)
print(result.combined.graph)  # {"language": "cpp", "views": ["ast", "cfg", "dfg"]}

# -- Generate from a file or folder ------------------------------------------
result = generate("cpp", code_file="./test.cpp", graphs=["cfg", "dfg"])
result = generate("c", code_folder="./project/src", graphs=["cfg", "dfg"])

# -- With extra options ------------------------------------------------------
result = generate(
    "cpp",
    code="...",
    graphs=["ast", "dfg"],
    collapsed=True,                              # collapse duplicate variable nodes
    last_def=True,                               # add last-def info to DFG edges
    blacklisted=["comment", "number_literal"],   # exclude AST node types
)

# -- Export to disk ----------------------------------------------------------
result.to_json("output.json")   # JSON
result.to_dot("output.dot")     # DOT
result.to_png("output.png")     # PNG image
result.to_svg("output.svg")     # SVG image
```

`generate()` returns a `GraphsResult` object with the following attributes:

| Attribute | Type | Description |
|-----------|------|-------------|
| `.ast` | `nx.MultiDiGraph` \| `None` | AST graph |
| `.cfg` | `nx.MultiDiGraph` \| `None` | CFG graph |
| `.dfg` | `nx.MultiDiGraph` \| `None` | DFG graph |
| `.combined` | `nx.MultiDiGraph` | Combined multi-view graph (always present) |
| `.language` | `str` | Source language |

`generate()` parameters:

| Parameter | Type | Required | Description |
|-----------|------|:--------:|-------------|
| `language` | `str` | ✓ | Source language: `"c"` / `"cpp"` / `"java"` / `"javascript"` |
| `code` | `str` | pick one | Source code as a string |
| `code_file` | `str\|Path` | pick one | Path to a source file |
| `code_folder` | `str\|Path` | pick one | Path to a source folder (auto-merges multi-file projects) |
| `graphs` | `list[str]` | | Which graphs to generate, default `["ast", "cfg", "dfg"]` |
| `collapsed` | `bool` | | Collapse duplicate variable nodes, default `False` |
| `last_def` | `bool` | | Add last-def annotation to DFG edges, default `False` |
| `last_use` | `bool` | | Add last-use annotation to DFG edges, default `False` |
| `blacklisted` | `list[str]` | | AST node types to exclude |
| `combined_name` | `str` | | Custom name for merged file (`code_folder` mode only) |

> See [docs/json-output-format.md](docs/json-output-format.md) for details on the JSON output format.

---
## Limitations

While `lambda-graphs` provides _method-level_ and _file-level_ support, it's important to note the following limitations and known issues:

### General Limitations
- **Syntax Errors in Code**: To ensure accurate codeviews, the input code must be free of syntax errors. Code with syntax errors may not be correctly parsed and displayed in the generated codeviews. Note that the code does not need to be compilable, only syntactically valid.

### C++ Specific Limitations
In addition to the general limitations, the tool has the following limitations specific to C++:

- **Limited Template Metaprogramming Support**: Complex template metaprogramming patterns may not be fully captured in the generated codeviews.

- **Partial Preprocessor Directive Support**: Preprocessor directives (e.g., `#define`, `#ifdef`) are parsed but not fully processed. Conditional compilation may not be accurately reflected in the codeviews.

- **Limited Support for Advanced C++ Features**: Some advanced C++ features such as:
  - Complex inheritance hierarchies
  - Multiple inheritance with virtual functions
  - Template specializations
  - SFINAE patterns
  - Concepts (C++20)

  may not be fully represented in the generated codeviews.

## Acknowledgments

This tool builds upon the tree-sitter parsing framework and is inspired by research on source code representation learning for AI-driven software engineering tasks.
