Metadata-Version: 2.4
Name: lambda-graphs
Version: 0.2.1
Summary: Generate combined multi-code view graphs
Home-page: https://github.com/PrinOrange/lambda-graphs
License: Apache-2.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: networkx==2.6.3
Requires-Dist: tree-sitter>=0.22
Requires-Dist: tree-sitter-c
Requires-Dist: tree-sitter-cpp
Requires-Dist: tree-sitter-java
Requires-Dist: tree-sitter-javascript
Requires-Dist: deepdiff>=7.0.0
Requires-Dist: pydot==1.4.1
Requires-Dist: typer>=0.9.0
Requires-Dist: loguru==0.6.0
Requires-Dist: setuptools>=69.0; python_version >= "3.12"
Provides-Extra: dev
Requires-Dist: tqdm; extra == "dev"
Requires-Dist: pqdm; extra == "dev"
Requires-Dist: loguru; extra == "dev"
Requires-Dist: GitPython; extra == "dev"
Requires-Dist: pandas; extra == "dev"
Dynamic: license-file

# lambda-graphs: Multi-View Code Representation Tool for C, C++ and Java Source Programs

lambda-graphs aims to generate combined multi-code view graphs that can be used with various types of machine learning models (sequence models, graph neural networks, etc).

Tool Demonstration link: [https://youtu.be/50DvEbenp14](https://youtu.be/50DvEbenp14)

## Overview

`lambda-graphs` is a CLI tool for generating customized source code representations from C and C++ programs. Currently, `lambda-graphs` generates codeviews for C and C++, supporting both method-level and file-level code snippets. `lambda-graphs` can be used to generate over 7 possible combinations of codeviews for both languages, including:

- **AST** (Abstract Syntax Tree)
- **CFG** (Control Flow Graph)
- **DFG** (Data Flow Graph)
- **Combined graphs** (any combination of the above)

`lambda-graphs` provides both a CLI and a **Python API** so you can generate graphs programmatically without shelling out. It is designed to be easily extendable to various programming languages. This is primarily because we use [tree-sitter](https://tree-sitter.github.io/tree-sitter/), a highly efficient incremental parser that supports over 40 languages.

---
## Setup

There are two ways to set up lambda-graphs: using Docker (recommended for quick usage) or using a Python virtual environment (recommended for development).

### Option 1: Docker Setup (Recommended for Quick Usage)

Docker provides an isolated environment with all dependencies pre-installed.

**1. Build the Docker image:**
```console
docker build -t lambda-graphs .
```

That's it! You're ready to generate graphs using Docker.

### Option 2: Virtual Environment Setup (Recommended for Development)

**1. Create a new virtual environment:**
```console
python -m venv .venv
```

**2. Activate the environment:**
```console
source .venv/bin/activate  # On Linux/Mac
# or
.venv\Scripts\activate  # On Windows
```

**3. Install the package in development mode:**
```console
pip install -e .
```

**4. Install GraphViz (Optional - for visualization):**

GraphViz is only required if you want to generate DOT or PNG output files.

**Ubuntu/Debian:**
```console
sudo apt install graphviz
```

**MacOS:**
```console
brew install graphviz
```

**Windows:**
Download from [graphviz.org](https://graphviz.org/download/)

---
## Generating Graphs

There are two ways to generate graphs: using Docker or using the CLI directly (after virtual environment setup).

### Option 1: Using Docker

Docker commands mount your current directory to `/work` inside the container, so output files appear in your working directory.

**Single File Analysis:**
```console
docker run --rm -v "$(pwd):/work" -w /work lambda-graphs \
    --lang cpp \
    --code-file ./examples/single/test_single.cpp \
    --graphs "ast,cfg,dfg" \
    --output all
```

**Folder Analysis (Multi-file Projects):**
```console
docker run --rm -v "$(pwd):/work" -w /work lambda-graphs \
    --lang c \
    --code-folder ./examples/multi \
    --combined-name "multi_file_example" \
    --graphs "cfg,dfg" \
    --output all
```

**With Additional Options:**
```console
# Generate only JSON output
docker run --rm -v "$(pwd):/work" -w /work lambda-graphs \
    --lang c \
    --code-file ./examples/single/test_single.c \
    --graphs cfg \
    --output json

# With collapsed nodes and last-def tracking
docker run --rm -v "$(pwd):/work" -w /work lambda-graphs \
    --lang c \
    --code-file ./examples/single/test_single.c \
    --graphs "ast,dfg" \
    --collapsed \
    --last-def \
    --output all
```

### Option 2: Using Example Scripts

The `examples/scripts/` folder contains ready-to-use bash scripts that automatically build the Docker image and run lambda-graphs on a target file or folder, generating CFG, DFG, and AST outputs. Each script handles the build, graph generation, and output file renaming in one step.

To run a script, make it executable and execute it from the repository root:

```console
chmod +x examples/scripts/single_test_single.sh
./examples/scripts/single_test_single.sh
```

Output files will appear in the `output/` directory, named after the source file and view (e.g. `test_single_cfg.png`, `test_single_dfg.json`).

---

### Option 3: Using CLI Directly

After setting up via virtual environment, use the `lambda-graphs` command directly.

**Output Location:** All generated files (JSON, DOT, PNG) are saved to the `output/` directory in your current working directory. The directory is created automatically if it doesn't exist.

The attributes and options supported by the CLI are well documented and can be viewed by running:
```console
lambda-graphs --help
```

**Single File Analysis:**

Generate a combined CFG and DFG graph for a C++ file:
```console
lambda-graphs --lang "cpp" --code-file ./test.cpp --graphs "cfg,dfg"
```

Generate an AST for a C file with output in JSON format:
```console
lambda-graphs --lang "c" --code-file ./example.c --graphs "ast" --output "json"
```

**Folder Analysis (Multi-file Projects):**

lambda-graphs can analyze entire projects by combining multiple source files from a folder:

```console
lambda-graphs --lang "c" --code-folder ./project/src --graphs "cfg,dfg" --output "json"
```

This will:
1. Recursively scan the folder for all `.c` and `.h` files
2. Combine them into a single temporary file (preserving includes, declarations, definitions)
3. Generate the requested codeviews from the combined source
4. Output results to the `output/` directory

You can customize the combined output file name:
```console
lambda-graphs --lang "cpp" --code-folder ./mylib --combined-name "myproject" --graphs "ast,cfg"
```

**Inline Code Analysis:**

You can also analyze code snippets directly without a file:
```console
lambda-graphs --lang "c" --code "int main() { int x = 5; return x; }" --graphs "ast,cfg"
```

**Additional CLI Options:**

| Option | Description |
|--------|-------------|
| `--output` | Output format: `json`, `dot`, `svg`, or `all` (dot generates PNG; svg generates SVG). Default: `dot` |
| `--collapsed` | Collapse duplicate variable nodes into a single node in AST |
| `--last-def` | Add last definition information to DFG edges (shows where variables were last defined) |
| `--blacklisted` | Comma-separated list of AST node types to exclude from the graph |

**Flag-Codeview Compatibility:**

| Flag | AST | CFG | DFG |
|------|:---:|:---:|:---:|
| `--collapsed` | ✓ | ✗ | ✗ |
| `--blacklisted` | ✓ | ✗ | ✗ |
| `--last-def` | ✗ | ✗ | ✓ |
| `--last-use` | ✗ | ✗ | ✓ |

**Examples:**

```console
# Generate all output formats (DOT, JSON, PNG)
lambda-graphs --lang "c" --code-file test.c --graphs "cfg" --output "all"

# Collapse duplicate variable nodes in DFG
lambda-graphs --lang "cpp" --code-file test.cpp --graphs "ast" --collapsed

# Add last definition tracking to DFG
lambda-graphs --lang "c" --code-file test.c --graphs "dfg" --last-def

# Exclude specific AST node types
lambda-graphs --lang "c" --code-file test.c --graphs "ast,cfg" --blacklisted "comment,string_literal"
```

### Option 4: Using the Python API

You can also use `lambda-graphs` as a library to get graph objects directly (no file I/O):

```python
from lambda_graphs import generate

# Generate graphs from a code string
result = generate("cpp", code="int main() { int x = 5; return x; }",
                  graphs=["ast", "cfg", "dfg"])

# Each graph is a networkx.MultiDiGraph
print(result.ast.nodes(data=True))
print(result.cfg.nodes(data=True))
print(result.dfg.nodes(data=True))
print(result.combined.nodes(data=True))

# With options
result = generate(
    "cpp",
    code="...",
    graphs=["ast", "dfg"],
    collapsed=True,                     # collapse duplicate variable nodes
    last_def=True,                      # add last-def info to DFG edges
    blacklisted=["comment", "number_literal"],  # exclude AST node types
)

# From a file
result = generate("cpp", code_file="./test.cpp", graphs=["cfg", "dfg"])

# From a folder (auto-merges multi-file projects)
result = generate("c", code_folder="./project/src", graphs=["cfg", "dfg"])

# Export to disk
result.to_json("output.json")
result.to_dot("output.dot", png=True, svg=True)
```

The returned `GraphsResult` object has four attributes:

| Attribute | Type | Description |
|-----------|------|-------------|
| `.ast` | `nx.MultiDiGraph` or `None` | AST graph |
| `.cfg` | `nx.MultiDiGraph` or `None` | CFG graph |
| `.dfg` | `nx.MultiDiGraph` or `None` | DFG graph |
| `.combined` | `nx.MultiDiGraph` | Combined multi-view graph |

---
## Limitations

While `lambda-graphs` provides _method-level_ and _file-level_ support for both C and C++, it's important to note the following limitations and known issues:

### General Limitations
- **Syntax Errors in Code**: To ensure accurate codeviews, the input code must be free of syntax errors. Code with syntax errors may not be correctly parsed and displayed in the generated codeviews. Note that the code does not need to be compilable, only syntactically valid.

### C++ Specific Limitations
In addition to the general limitations, the tool has the following limitations specific to C++:

- **Limited Template Metaprogramming Support**: Complex template metaprogramming patterns may not be fully captured in the generated codeviews.

- **Partial Preprocessor Directive Support**: Preprocessor directives (e.g., `#define`, `#ifdef`) are parsed but not fully processed. Conditional compilation may not be accurately reflected in the codeviews.

- **Limited Support for Advanced C++ Features**: Some advanced C++ features such as:
  - Complex inheritance hierarchies
  - Multiple inheritance with virtual functions
  - Template specializations
  - SFINAE patterns
  - Concepts (C++20)

  may not be fully represented in the generated codeviews.

---

## Output Examples

### Example 1: C++ Function Pointers and Control Flow

**CLI Command**:

```bash
lambda-graphs --lang "cpp" --code-file paper_assets/function_pointers.cpp --graphs "cfg,dfg"
```
---

**C++ Code Snippet** ([function_pointers.cpp](paper_assets/function_pointers.cpp)):

```cpp
#include <iostream>
void f1(int times) {
    if(!times)
        return;
    std::cout << "In f1()\n";
    f1(times-1);
}
void f2() {
    std::cout << "In f2()\n";
}
int main() {
    void (*fptr_1)(int);
    void (*fptr_2)(void);
    fptr_1 = &f1;
    fptr_2 = &f2;

    int var = 0;
    std::cin >> var;
    (var > 0) ? fptr_1(3) : fptr_2();
}
```
---

**Generated Codeview**:

![C++ Function Pointers Example](paper_assets/function_pointers_cfg_dfg.png)

---

### Example 2: C++ Class with Pass-by-Reference

**CLI Command**:

```bash
lambda-graphs --lang "cpp" --code-file paper_assets/pass_by_reference.cpp --graphs "cfg,dfg"
```
---

**C++ Code Snippet** ([pass_by_reference.cpp](paper_assets/pass_by_reference.cpp)):

```cpp
#include <iostream>

class TestClass {
public:
    int x;
    TestClass(int _x) {
        x = _x + 20;
    }
    void f1(int& a) {
        a += 100;
        a -= x;
    }
};
int main() {
    TestClass obj(30);
    int k = 0;
    obj.f1(k);
    std::cout << k; // prints 50
    return 0;
}
```
---

**Generated Codeview**:

![C++ Class Example](paper_assets/pass_by_reference_cfg_dfg.png)

---

## Code Organization

The code is structured in the following way:

1. **Preprocessing** (`src/lambda_graphs/utils/`): The `multi_file_merger.py` module combines multiple source files from a folder into a single file for analysis.

2. **Parsing** (`src/lambda_graphs/tree_parser/`): For each code-view, first the source code is parsed using the tree-sitter parser. The Parser and ParserDriver are implemented with various functionalities commonly required by all code-views. Language-specific features are further developed in the language-specific parsers (`c_parser.py`, `cpp_parser.py`).

3. **Codeview Generation** (`src/lambda_graphs/codeviews/`): This directory contains the core logic for the various codeviews:
   - `AST/` - Abstract Syntax Tree (language-agnostic)
   - `CFG/` - Control Flow Graph (language-specific: `CFG_c.py`, `CFG_cpp.py`)
   - `DFG/` - Data Flow Graph (language-agnostic)
   - `combined_graph/` - Combines multiple codeviews into a single graph

4. **CLI & API Entry Points** (`src/lambda_graphs/cli.py`, `__init__.py`): The CLI is implemented with Typer. The `generate()` function in `__init__.py` exposes the same functionality as a Python API.

5. **Node Definitions** (`src/lambda_graphs/utils/`): `c_nodes.py` and `cpp_nodes.py` define AST node type categorizations used throughout the codebase.

---

## Acknowledgments

This tool builds upon the tree-sitter parsing framework and is inspired by research on source code representation learning for AI-driven software engineering tasks.
