Skip to content

Getting Started with Plan-Lint

Plan-Lint is a static analysis toolkit that helps validate LLM agent plans before execution. This guide will help you set up and start using Plan-Lint in your projects.

Installation

Install Plan-Lint using pip:

pip install plan-lint

For the latest development version, you can install directly from GitHub:

pip install git+https://github.com/mason-lang/plan-lint.git

Basic Usage

Here's a simple example of how to validate a plan:

from plan_lint import validate_plan

# Define a plan to validate
plan = {
    "steps": [
        {
            "id": "step1",
            "tool": "web_request",
            "parameters": {
                "url": "http://example.com",  # Insecure HTTP URL
                "method": "GET"
            }
        }
    ]
}

# Validate the plan using the default security policies
result = validate_plan(plan)

# Check the result
if result.valid:
    print("Plan is valid and safe to execute!")
else:
    print("Plan validation failed:")
    for violation in result.violations:
        print(f"- {violation.message}")

Integrating with LLM Agents

You can integrate Plan-Lint directly into your agent system's workflow:

from plan_lint import validate_plan

def agent_workflow(user_input):
    # Step 1: Generate a plan using your LLM agent
    plan = generate_plan(user_input)

    # Step 2: Validate the plan before execution
    validation = validate_plan(plan)

    if validation.valid:
        # Step 3a: If valid, execute the plan
        return execute_plan(plan)
    else:
        # Step 3b: If invalid, handle the violations
        return handle_violations(validation.violations)

Using Custom Policies

You can specify custom policies to use during validation:

from plan_lint import validate_plan

# Path to your custom policy files
custom_policies = [
    "path/to/custom_security.rego",
    "path/to/custom_data_privacy.rego"
]

# Validate with custom policies
result = validate_plan(
    plan,
    policies=custom_policies
)

Adding Context Information

Provide additional context to the validation process:

from plan_lint import validate_plan

# Context information that policies can access
context = {
    "user_role": "regular",
    "environment": "production",
    "allowed_domains": ["example.com", "api.example.org"]
}

# Validate with context
result = validate_plan(
    plan,
    context=context
)

Handling Validation Results

Process validation results programmatically:

from plan_lint import validate_plan

result = validate_plan(plan)

if result.valid:
    print("Plan is valid and safe to execute!")
else:
    # Group violations by category
    security_violations = [v for v in result.violations if v.category == "security"]
    privacy_violations = [v for v in result.violations if v.category == "privacy"]

    if security_violations:
        print("Security issues found:")
        for violation in security_violations:
            print(f"- {violation.message}")

    if privacy_violations:
        print("Privacy issues found:")
        for violation in privacy_violations:
            print(f"- {violation.message}")

CLI Usage

Plan-Lint also provides a command-line interface:

# Validate a plan file against default policies
plan-lint validate path/to/plan.json

# Validate with custom policies
plan-lint validate path/to/plan.json --policies path/to/policy1.rego path/to/policy2.rego

# Provide context information
plan-lint validate path/to/plan.json --context path/to/context.json

# Output format options
plan-lint validate path/to/plan.json --output json

Next Steps

Troubleshooting

If you encounter any issues with Plan-Lint, check the following:

  1. Ensure your plan structure follows the expected format
  2. Verify that any custom policy files are valid Rego code
  3. Check that the Open Policy Agent (OPA) dependency is correctly installed

For more help, visit the GitHub repository or submit an issue.