Plan-Lint Examples¶
This page provides practical examples of using Plan-Lint to validate AI agent plans.
Basic Example¶
Sample Plan¶
Let's start with a simple plan that performs a database query and sends an email:
{
"steps": [
{
"id": "step1",
"tool": "database_query",
"parameters": {
"query": "SELECT * FROM customers WHERE id = 123"
}
},
{
"id": "step2",
"tool": "send_email",
"parameters": {
"to": "user@example.com",
"subject": "Your Account Information",
"body": "Here is your requested information: {{step1.result}}"
}
}
]
}
Validation with Default Policies¶
To validate this plan with default policies:
This will check for potential issues such as SQL injection vulnerabilities and sensitive data exposure.
SQL Injection Detection¶
Vulnerable Plan¶
Consider a plan with a potential SQL injection vulnerability:
{
"steps": [
{
"id": "step1",
"tool": "database_query",
"parameters": {
"query": "SELECT * FROM users WHERE username = '" + username + "'"
}
}
]
}
Validation Output¶
Running Plan-Lint on this vulnerable plan:
Would produce output similar to:
Validation Results:
✘ Plan validation failed with 1 violation
Violations:
- [HIGH] sql_injection_detection: Potential SQL injection detected in query (step: step1)
SQL query contains string concatenation which is a common indicator of SQL injection vulnerability
Fixed Plan¶
A safer version of the plan would use parameterized queries:
{
"steps": [
{
"id": "step1",
"tool": "database_query",
"parameters": {
"query": "SELECT * FROM users WHERE username = ?",
"params": [username]
}
}
]
}
Sensitive Data Exposure¶
Vulnerable Plan¶
This plan exposes sensitive data in the logs:
{
"steps": [
{
"id": "step1",
"tool": "authenticate",
"parameters": {
"username": "admin",
"password": "SecretP@ssw0rd"
}
},
{
"id": "step2",
"tool": "log_event",
"parameters": {
"message": "Authentication attempt with credentials: {{step1.parameters}}"
}
}
]
}
Validation Output¶
Would produce:
Validation Results:
✘ Plan validation failed with 1 violation
Violations:
- [CRITICAL] sensitive_data_exposure: Step references sensitive data that could be exposed (step: step2)
Parameter 'message' contains references to authentication credentials from step 'step1'
Fixed Plan¶
A safer version avoids logging sensitive information:
{
"steps": [
{
"id": "step1",
"tool": "authenticate",
"parameters": {
"username": "admin",
"password": "SecretP@ssw0rd"
}
},
{
"id": "step2",
"tool": "log_event",
"parameters": {
"message": "Authentication attempt for user: {{step1.parameters.username}}"
}
}
]
}
Excessive Transaction Amount¶
Vulnerable Plan¶
A plan with an unusually large transaction:
{
"steps": [
{
"id": "step1",
"tool": "transfer_funds",
"parameters": {
"from_account": "12345",
"to_account": "67890",
"amount": 1000000
}
}
]
}
Validation with Context¶
Where context.json contains:
Would produce:
Validation Results:
✘ Plan validation failed with 1 violation
Violations:
- [HIGH] excessive_transaction_amount: Transaction amount exceeds allowed limit (step: step1)
Transaction amount 1000000 exceeds maximum allowed amount of 5000
Fixed Plan¶
A compliant version with a reasonable amount:
{
"steps": [
{
"id": "step1",
"tool": "transfer_funds",
"parameters": {
"from_account": "12345",
"to_account": "67890",
"amount": 3000
}
}
]
}
Custom Policy Example¶
Custom Policy¶
Create a file named enforce_retry_limit.rego:
package planlint.custom
import future.keywords.in
# Detect excessive retry attempts
deny[result] {
some step in input.plan.steps
step.tool == "retry_operation"
to_number(step.parameters.max_attempts) > 5
result := {
"rule": "excessive_retry_attempts",
"message": "Max retry attempts exceeds the recommended limit",
"severity": "medium",
"category": "reliability",
"step_id": step.id,
"metadata": {
"max_attempts": step.parameters.max_attempts,
"recommended_limit": 5
}
}
}
Plan to Validate¶
{
"steps": [
{
"id": "step1",
"tool": "retry_operation",
"parameters": {
"operation": "transfer_funds",
"max_attempts": 10
}
}
]
}
Validation Command¶
Validation Output¶
Validation Results:
✘ Plan validation failed with 1 violation
Violations:
- [MEDIUM] excessive_retry_attempts: Max retry attempts exceeds the recommended limit (step: step1)
Max attempts: 10, Recommended limit: 5
Programmatic Usage Example¶
Here's a more complex example of using Plan-Lint programmatically to validate plans in an AI agent system:
import json
from planlint import validate_plan, load_policies_from_directory
# Load custom policies
custom_policies = load_policies_from_directory("policies/")
# Load context from file
with open("agent_context.json", "r") as f:
context = json.load(f)
def validate_agent_plan(plan, agent_id, user_id):
# Add dynamic context information
plan_context = context.copy()
plan_context["agent_id"] = agent_id
plan_context["user_id"] = user_id
plan_context["timestamp"] = datetime.now().isoformat()
# Validate the plan
result = validate_plan(
plan=plan,
policies=custom_policies,
context=plan_context
)
# Log the validation results
log_validation_result(agent_id, result)
if not result.is_valid:
# Send alert for critical violations
critical_violations = [v for v in result.violations if v.severity == "critical"]
if critical_violations:
send_alert(agent_id, critical_violations)
return result
# In your agent's execution pipeline
async def execute_agent_task(task, agent_id, user_id):
# Generate plan using LLM
plan = await generate_plan(task)
# Validate plan before execution
validation_result = validate_agent_plan(plan, agent_id, user_id)
if validation_result.is_valid:
# Execute the plan
result = await execute_plan(plan)
return result
else:
# Handle validation failure
return {
"status": "error",
"message": "Plan failed validation",
"violations": [v.to_dict() for v in validation_result.violations]
}
Integration with CI/CD Pipeline¶
You can add Plan-Lint validation to your CI/CD pipeline to ensure all deployed agent plans meet your security requirements:
# GitHub Actions workflow example
name: Validate Agent Plans
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
validate-plans:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.9'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install plan-lint
- name: Validate example plans
run: |
for plan in examples/plans/*.json; do
echo "Validating $plan"
plan-lint validate --plan $plan --policies policies/*.rego
done
- name: Run plan-lint tests
run: |
plan-lint test --policies policies/*.rego