Metadata-Version: 2.4
Name: chatty-agent
Version: 0.2.0
Summary: AI Chatbot CLI with advanced sandboxed text and file system interaction
License:                                  Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
           1. Definitions.
        
              "License" shall mean the terms and conditions for use, reproduction,
              and distribution as defined by Sections 1 through 9 of this document.
        
              "Licensor" shall mean the copyright owner or entity authorized by
              the copyright owner that is granting the License.
        
              "Legal Entity" shall mean the union of the acting entity and all
              other entities that control, are controlled by, or are under common
              control with that entity. For the purposes of this definition,
              "control" means (i) the power, direct or indirect, to cause the
              direction or management of such entity, whether by contract or
              otherwise, or (ii) ownership of fifty percent (50%) or more of the
              outstanding shares, or (iii) beneficial ownership of such entity.
        
              "You" (or "Your") shall mean an individual or Legal Entity
              exercising permissions granted by this License.
        
              "Source" form shall mean the preferred form for making modifications,
              including but not limited to software source code, documentation
              source, and configuration files.
        
              "Object" form shall mean any form resulting from mechanical
              transformation or translation of a Source form, including but
              not limited to compiled object code, generated documentation,
              and conversions to other media types.
        
              "Work" shall mean the work of authorship, whether in Source or
              Object form, made available under the License, as indicated by a
              copyright notice that is included in or attached to the work
              (an example is provided in the Appendix below).
        
              "Derivative Works" shall mean any work, whether in Source or Object
              form, that is based on (or derived from) the Work and for which the
              editorial revisions, annotations, elaborations, or other modifications
              represent, as a whole, an original work of authorship. For the purposes
              of this License, Derivative Works shall not include works that remain
              separable from, or merely link (or bind by name) to the interfaces of,
              the Work and Derivative Works thereof.
        
              "Contribution" shall mean any work of authorship, including
              the original version of the Work and any modifications or additions
              to that Work or Derivative Works thereof, that is intentionally
              submitted to Licensor for inclusion in the Work by the copyright owner
              or by an individual or Legal Entity authorized to submit on behalf of
              the copyright owner. For the purposes of this definition, "submitted"
              means any form of electronic, verbal, or written communication sent
              to the Licensor or its representatives, including but not limited to
              communication on electronic mailing lists, source code control systems,
              and issue tracking systems that are managed by, or on behalf of, the
              Licensor for the purpose of discussing and improving the Work, but
              excluding communication that is conspicuously marked or otherwise
              designated in writing by the copyright owner as "Not a Contribution."
        
              "Contributor" shall mean Licensor and any individual or Legal Entity
              on behalf of whom a Contribution has been received by Licensor and
              subsequently incorporated within the Work.
        
           2. Grant of Copyright License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              copyright license to reproduce, prepare Derivative Works of,
              publicly display, publicly perform, sublicense, and distribute the
              Work and such Derivative Works in Source or Object form.
        
           3. Grant of Patent License. Subject to the terms and conditions of
              this License, each Contributor hereby grants to You a perpetual,
              worldwide, non-exclusive, no-charge, royalty-free, irrevocable
              (except as stated in this section) patent license to make, have made,
              use, offer to sell, sell, import, and otherwise transfer the Work,
              where such license applies only to those patent claims licensable
              by such Contributor that are necessarily infringed by their
              Contribution(s) alone or by combination of their Contribution(s)
              with the Work to which such Contribution(s) was submitted. If You
              institute patent litigation against any entity (including a
              cross-claim or counterclaim in a lawsuit) alleging that the Work
              or a Contribution incorporated within the Work constitutes direct
              or contributory patent infringement, then any patent licenses
              granted to You under this License for that Work shall terminate
              as of the date such litigation is filed.
        
           4. Redistribution. You may reproduce and distribute copies of the
              Work or Derivative Works thereof in any medium, with or without
              modifications, and in Source or Object form, provided that You
              meet the following conditions:
        
              (a) You must give any other recipients of the Work or
                  Derivative Works a copy of this License; and
        
              (b) You must cause any modified files to carry prominent notices
                  stating that you changed the files; and
        
              (c) You must retain, in the Source form of any Derivative Works
                  that You distribute, all copyright, patent, trademark, and
                  attribution notices from the Source form of the Work,
                  excluding those notices that do not pertain to any part of
                  the Derivative Works; and
        
              (d) If the Work includes a "NOTICE" text file as part of its
                  distribution, then any Derivative Works that You distribute must
                  include a readable copy of the attribution notices contained
                  within such NOTICE file, excluding those notices that do not
                  pertain to any part of the Derivative Works, in at least one
                  of the following places: within a NOTICE text file distributed
                  as part of the Derivative Works; within the Source form or
                  documentation, if provided along with the Derivative Works; or,
                  within a display generated by the Derivative Works, if and
                  wherever such third-party notices normally appear. The contents
                  of the NOTICE file are for informational purposes only and
                  do not modify the License. You may add Your own attribution
                  notices within Derivative Works that You distribute, alongside
                  or as an addendum to the Work text from the Licensor, provided
                  that such additional attribution notices cannot be construed
                  as modifying the License.
        
              You may add Your own copyright statement to Your modifications and
              may provide additional or different license terms and conditions
              for use, reproduction, or distribution of Your modifications, or
              for any such Derivative Works as a whole, provided Your use,
              reproduction, and distribution of the Work otherwise complies with
              the conditions stated in this License.
        
           5. Submission of Contributions. Unless You explicitly state otherwise,
              any Contribution intentionally submitted for inclusion in the Work
              by You to the Licensor shall be under the terms and conditions of
              this License, without any additional terms or conditions.
              Notwithstanding the above, nothing herein shall supersede or modify
              the terms of any separate license agreement you may have executed
              with Licensor regarding such Contributions.
        
           6. Trademarks. This License does not grant permission to use the trade
              names, trademarks, service marks, or product names of the Licensor,
              except as required for reasonable and customary use in describing the
              origin of the Work and reproducing the content of the NOTICE file.
        
           7. Disclaimer of Warranty. Unless required by applicable law or
              agreed to in writing, Licensor provides the Work (and each
              Contributor provides its Contributions) on an "AS IS" BASIS,
              WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
              implied, including, without limitation, any warranties or conditions
              of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
              PARTICULAR PURPOSE. You are solely responsible for determining the
              appropriateness of using or redistributing the Work and assume any
              risks associated with Your exercise of permissions under this License.
        
           8. Limitation of Liability. In no event and under no legal theory,
              whether in tort (including negligence), contract, or otherwise,
              unless required by applicable law (such as deliberate and grossly
              negligent acts) or agreed to in writing, shall any Contributor be
              liable to You for damages, including any direct, indirect, special,
              incidental, or consequential damages of any character arising as a
              result of this License or out of the use or inability to use the
              Work (including but not limited to damages for loss of goodwill,
              work stoppage, computer failure or malfunction, or any and all
              other commercial damages or losses), even if such Contributor
              has been advised of the possibility of such damages.
        
           9. Accepting Warranty or Additional Liability. While redistributing
              the Work or Derivative Works thereof, You may choose to offer,
              and charge a fee for, acceptance of support, warranty, indemnity,
              or other liability obligations and/or rights consistent with this
              License. However, in accepting such obligations, You may act only
              on Your own behalf and on Your sole responsibility, not on behalf
              of any other Contributor, and only if You agree to indemnify,
              defend, and hold each Contributor harmless for any liability
              incurred by, or claims asserted against, such Contributor by reason
              of your accepting any such warranty or additional liability.
        
           END OF TERMS AND CONDITIONS
        
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests
Requires-Dist: openai
Requires-Dist: rich
Requires-Dist: prompt-toolkit
Requires-Dist: PyYAML
Requires-Dist: json-repair
Requires-Dist: psutil
Requires-Dist: litellm
Requires-Dist: pypdf
Requires-Dist: tree-sitter
Requires-Dist: tree-sitter-languages
Dynamic: license-file

# Chatty

<p align="center">
  <img src="https://raw.githubusercontent.com/davidel/chatty/main/assets/LOGO.png" alt="Chatty" width="600">
</p>

An advanced AI Chatbot CLI with a rich terminal interface, sandboxed file system tools, multi-provider LLM support (Ollama, OpenRouter, and custom providers via LiteLLM), syntax verification, dynamic skills, and cost/token optimization.

Chatty provides a local terminal loop that allows an LLM agent to interact with your codebase and system tools in a safe, sandboxed directory. It utilizes `rich` and `prompt_toolkit` to deliver a premium user experience, complete with formatting, status indicators, and autocompletion.

---

## Features

- **Multi-Provider Flexibility**: Switch seamlessly between local Ollama instances, cloud-based OpenRouter endpoints, or any custom API provider (such as Anthropic, Chatty, DeepSeek, Groq, etc.) via LiteLLM integration.
- **Oracle Query Delegation**: Consult a more advanced oracle model dynamically during a session via the [ask_oracle](file:///tmp/chatty/src/chatty/tools/__init__.py#L668) tool or the `/oracle` slash command.
- **Sandboxed Operations**: Restricts file modifications and commands strictly to a sandbox root (default `./sandbox`), enhanced by Linux Landlock kernel-level isolation when running on Linux.
- **User-Space AST Validation**: Parses Python scripts executed via shell commands and blocks direct filesystem operations unless explicitly permitted or whitelisted by the user.
- **Interactive UI**: Status bars showing the provider, active model, token counter, active loop count, and sandbox directory path. Autocomplete and multiline inputs are fully integrated.
- **Slash Commands**: Modify settings dynamically mid-session, view tool metrics, or compress history to save tokens.
- **Dynamic & Static Skills**: Dynamically import system prompts and guidelines from external files or directories via keyword/tag triggers or statically on initialization.
- **Context Optimization**: Automates context truncation and history compression. Implements ephemeral `cache_control` tagging to maximize prompt caching efficiency for OpenRouter.
- **Google-Style Logging**: Standardized process tracking via custom `glog` styling.

---

## Installation & Setup

### Prerequisites
- Python 3.8 or higher.
- A running Ollama instance locally (optional, for local models).
- An OpenRouter API Key (optional, for remote models).

### Installation

Install directly from PyPI:

```bash
pip install chatty-agent
```

Alternatively, install from the local repository directory:

```bash
pip install .
```

### Developer Mode (Editable Install)
To develop or modify the codebase and have changes immediately visible without reinstalling:

```bash
pip install -e .
```

---

## CLI Usage & Arguments

Once installed, invoke the chatbot using the `chatty` command or via Python:

```bash
chatty [options]
# OR
python3 -m chatty [options]
```

### Full List of CLI Arguments

| Parameter | Short | Type | Default | Description |
| :--- | :--- | :--- | :--- | :--- |
| `--provider` | `-p` | string | `ollama` | Backend provider to use (e.g., `ollama`, `openrouter`, `anthropic`, `chatty`, etc.). If omitted and `--url` is set, resolves automatically to the domain of the URL. |
| `--model` | `-m` | string | *Auto-resolved* | Model identifier(s) to load. Can be specified multiple times or as comma-separated values. The first becomes the active model. Ollama: auto-detects first local model (falls back to `qwen2.5-coder:7b`). OpenRouter: dynamically defaults to the most popular free model (falls back to `google/gemini-2.5-flash:free`). Custom: required (no fallback). |
| `--oracle-model` | *None* | string | *None* | Model identifier to use as the oracle. No default is assumed (the oracle tool is only active if an oracle model is explicitly configured). |
| `--context-size` | `-c` | integer | `8192` | Target context window length constraint in tokens. |
| `--sandbox` | `-s` | string | `./sandbox` | Path to the sandboxed folder. All writes and runs are jailed inside this directory. |
| `--skills-path` | `-k` | string | *None* | Custom directory paths to scan for static/dynamic Skills (can be specified multiple times). |
| `--ondemand-skills-path` | `-o` | string | *None* | Custom directory paths to scan for on-demand Skills (can be specified multiple times). |
| `--whitelist` | `-w` | string | *None* | Add an out-of-sandbox path to the initial whitelist. Can end with `:ro` or `:rw` to set mode (defaults to `ro`). Can be specified multiple times. |
| `--static-skills` | *None* | flag | *Auto* | Load all skills statically into system instructions (defaults to `True` for OpenRouter, `False` for Ollama). |
| `--prompt-caching` | *None* | flag | `False` | Explicitly enable prompt caching for compatible models (adds `cache_control` tagging). |
| `--max-loops` | `-l` | integer | `20` | Maximum sequential tool executions allowed in a single user turn. |
| `--config-prompt` | `-f` | string | *None* | Path to a YAML or plain text file containing custom system prompt guidelines. |
| `--prompt-mode` | `-d` | string | `replace` | How to apply custom system prompt configuration (`replace` default prompt, or `integrate`/append to it). |
| `--api-key` | `-a` | string | *None* | Client API Key. Overrides `OPENROUTER_API_KEY`, `CUSTOM_API_KEY`, or `OPENAI_API_KEY` environment variables depending on the provider. |
| `--url` | `-u` | string | *None* | Custom Base URL override (required for custom providers; automatically resolves the provider name to the domain of the URL if `--provider` is not specified). |
| `--max-read-chars` | *None* | integer | `40000` | Maximum character limit when reading a text file to prevent context explosion. |
| `--max-grep-results`| *None* | integer | `100` | Limit on matching results returned from the regular expression search tool. |
| `--max-command-chars`|*None* | integer | `16000` | Maximum characters returned from stdout/stderr of executing commands. |
| `--max-history-tool-chars`|*None*| integer| `1000` | Token saving limit: compresses old/historical tool output messages to this size. |
| `--history-keep-messages`| *None*| integer| `4` | Number of recent messages to keep fully raw and uncompressed. |
| `--max-url-chars` | *None* | integer | `24000` | Limit on fetched website character outputs. |
| `--max-dir-items` | *None* | integer | `200` | Maximum number of directory items listed by directory explorer tool. |
| `--log-file` | *None* | string | `chatty.log` | File path where execution statements are logged. Set to `""` to disable. |
| `--log-level` | *None* | string | `info` | Logging verbosity (`debug`, `info`, `warning`, `error`). |
| `--headless` | *None* | flag | `False` | Run the chatbot in headless mode (no console printing or terminal interactive loop). |
| `--max-thinking-chars` | *None* | integer | `12000` | Maximum internal thinking characters before prompting the user. |
| `--max-thinking-leeway-chars` | *None* | integer | `2000` | Leeway in characters beyond the maximum before hard-aborting or prompting. |
| `--api-delay` | *None* | float | `2.5` | Minimum delay in seconds between consecutive API requests. |
| `--api-timeout` | *None* | float | `60.0` | Timeout in seconds for API requests and streams. |


---

## Custom OpenAI-Compatible & Multi-Provider Support

Chatty uses **LiteLLM** under the hood, meaning you can connect to any third-party cloud provider (like Anthropic or Chatty) or any OpenAI-compatible custom endpoint. 

### 1. OpenAI-Compatible Custom Endpoints
To connect to custom endpoints (like DeepSeek, Groq, Together AI, or local vLLM servers), specify `--url` and `--model` (along with `--api-key` if required). The provider name will automatically be resolved to the domain name of your URL:

```bash
# Run using DeepSeek V3 (automatically sets provider to 'api.deepseek.com')
chatty --url https://api.deepseek.com/v1 --api-key YOUR_DEEPSEEK_KEY --model deepseek-chat

# Run using Groq (automatically sets provider to 'api.groq.com')
chatty --url https://api.groq.com/openai/v1 --api-key YOUR_GROQ_KEY --model llama-3.3-70b-versatile

# Run using Together AI (automatically sets provider to 'api.together.xyz')
chatty --url https://api.together.xyz/v1 --api-key YOUR_TOGETHER_KEY --model Qwen/Qwen2.5-Coder-32B-Instruct
```

### 2. Native Multi-Provider Support via LiteLLM
You can also call other native providers supported by LiteLLM (such as Anthropic or Google Chatty) directly by specifying their provider name and model:

```bash
# Run using Anthropic Claude 3.5 Sonnet (expects ANTHROPIC_API_KEY env variable)
chatty --provider anthropic --model claude-3-5-sonnet

# Run using Google Chatty Pro (expects CHATTY_API_KEY env variable)
chatty --provider chatty --model chatty-2.5-pro
```

---

## Interactive Interface & Slash Commands

During a session, you can input direct queries to the model, or use **Slash Commands** to inspect/adjust configurations on the fly:

| Command | Expected Arguments | Description |
| :--- | :--- | :--- |
| `/help` | None | Displays a formatted usage table of all slash commands. |
| `/status` | None | Shows active session variables (provider, model, oracle, sandbox, tokens, loops, etc.). |
| `/tool_stats` | None | Renders execution statistics (call counts, failures, and breakdowns for tools and binaries). |
| `/provider` | `[name]` | View current provider or switch backend on the fly. Accepts standard or custom provider names (e.g., `ollama`, `openrouter`, `anthropic`, `chatty`, `api.deepseek.com`). |
| `/model` | `[ID\|name]` | View active model name or switch to another model by name or 1-based index/ID. |
| `/models` | `[add <name>\|remove <ID\|name>\|available [--refresh]\|search <query>\|info <ID\|name>]` | List, add, remove, search, or view details of LLM models. |
| `/oracle` | `[name]` | View active oracle model name or switch to another oracle model by name. |
| `/sandbox` | `[path]` | View sandbox path or change it. Instantly loads any skills found in the new sandbox. |
| `/whitelist` / `/permissions` | `[add <path> [ro\|rw] \| remove <path> \| clear]` | View or manage whitelisted out-of-sandbox paths. |
| `/skill` | `[NAME...\|clear]` | Load on-demand skill(s) or clear explicitly loaded ones. |
| `/context` | `[tokens]` | View or update target context memory window limit in tokens. |
| `/loops` | `[iterations]` | View or modify the limit of sequential agent loops allowed per turn. |
| `/api_key` | `[key]` | Configure your client token dynamically. |
| `/system` | `[text]` | Inspect or update the base system prompt instructions directly. |
| `/load` | `<path> [append\|replace]` | Read system instructions from a local YAML or text file, appending or replacing. |
| `/save` / `/save_session` | `<path>` | Save the whole status of the current conversation/session to a JSON file. |
| `/load_session` | `<path>` | Load a saved conversation/session status from a JSON file. |
| `/history` | `[N]` | Renders message records, estimated token counts, roles, and tool calls, or displays the N-th entry in full detail. |
| `/undo` | `[count]` | Reverts the last conversation turn(s), removing assistant responses, tool outputs, and the user prompt. |
| `/pop` | `<index>` | Truncates the conversation history by deleting all messages from the specified 1-based index onwards. |
| `/backups` | `<file_path>` | Lists all available timestamped backups for a given file. |
| `/restore` | `<file_path> [index_or_timestamp]` | Restores a file to a specific backup version (defaults to the latest). |
| `/tools` | None | Lists available sandboxed tools and their schema definitions. |
| `/config` | `[key=value]` | List, view, or change configuration parameters live. |
| `/clear` / `/reset`| None | Clears conversational context history. |
| `/compress` | `[N]` | Directs the model to summarize current conversational state using a structured format, resets older history, and keeps N (default 4) recent messages intact. |
| `/copy` / `/clip` | `[index]` | Copy a code block from the last AI response to the clipboard. |
| `/write` / `/save_code` | `<path> [index]` | Write a code block from the last AI response to a file. |
| `/exit` / `/quit` | None | Cleanly terminates background processes and exits Chatty. |

---

## Sandboxed File System Tools

The chatbot uses function-calling to interface with the sandbox workspace. Directly invoking command-line file manipulation programs (e.g. `cat`, `grep`, `find`) in `run_command` is blocked by safety filters in [validate_command_safety](file:///tmp/chatty/src/chatty/safety.py#L168). The agent is required to use the appropriate structured tools:

### File Manipulation Tools
- **`list_dir`**: Explores directories inside the sandbox. Truncates output above `--max-dir-items` to prevent token flooding.
- **`read_file`**: Reads text files. Accepts optional `start_line` and `end_line` parameters (1-indexed), supports displaying line numbers, and honors `--max-read-chars`.
- **`write_file`**: Writes full text contents to a file.
- **`patch_file`**: Replaces one or more unique blocks of code inside a file using Aider-style SEARCH/REPLACE blocks. Highly robust to whitespace and indentation differences (automatically adjusts output indentation to match the file). Supports chaining multiple blocks sequentially in one patch parameter to perform multiple edits in a single call.
- **`format_file`**: Styles source files using formatters: `black`/`ruff` for Python, `clang-format` for C/C++, `prettier` for frontend, or custom JSON/YAML encoders. Displays diff results.
- **`move_file`**: Renames or moves files and directories safely inside the sandbox boundaries.
- **`copy_file`**: Recursively copies file system structures.
- **`delete_file`**: Permanently removes a file. Fails on directories.
- **`delete_directory`**: Permanently removes a directory (optionally recursively).
- **`make_directory`**: Recursively builds directory trees.
- **`get_file_info`**: Retrieves file system metadata (modification dates, sizes, type, and line counts for text documents).
- **`hex_dump`**: Performs a hex dump or parses slices of binary files into integers of various widths (8/16/32/64-bit), endianness, and signedness.
- **`list_file_backups`**: Lists all available timestamped backups for a file path.
- **`read_file_backup`**: Reads the contents of a specific timestamped file backup (with optional range read and line numbering).


### Code Search & Diagnostics
- **`search_grep`**: Performs recursive regular expression string matching on files. Can report line numbers (`line_numbers: true`) to aid editing.
- **`locate_files`**: Finds files recursively matching glob configurations (e.g., `**/*.py`).
- **`run_tests`**: Runs test scripts (`pytest`, `npm test`, custom targets).

### Web & Information Retrieval
- **`search_web`**: Searches the web for a query and returns titles, URLs, and snippets. Supports multiple backends via environment variables (checked in priority order):
  - **Tavily**: Set `TAVILY_API_KEY` (highly recommended for clean, parsed AI search results).
  - **Brave Search**: Set `BRAVE_API_KEY` (independent, privacy-focused search).
  - **Google Custom Search**: Set `GOOGLE_API_KEY` and `GOOGLE_CSE_ID` (legacy Google search engine).
  - **Serper**: Set `SERPER_API_KEY` (Google search proxy).
  - **SerpApi**: Set `SERPAPI_API_KEY` (Google search proxy).
  - **Yahoo Scraper**: Default fallback if no keys are provided (unreliable for heavy use).
- **`fetch_url`**: Fetches the text content of a public URL. Automatically parses both HTML (converting it to clean plain text) and PDF documents. For scanned PDFs (which lack a text layer), it features an optional zero-overhead OCR fallback utilizing `pytesseract` and `pdf2image` dynamically if available on the host system.

### Command & Background Execution
- **`run_command`**: Runs shell commands from the sandbox directory.
  - *Safety Restrictions*: Monitored by safety checks to block commands that attempt directory escapes, or attempt to circumvent tool guidelines by calling commands like `cat`, `grep`, `find`, `sed`, `awk`, `less`, `more`, `wc`, `kill`, `pkill`, `killall`, `cp`, `mv`, `rm`, `rmdir`, `mkdir`, `ls`, `dir`.
  - *Output Controls*: Supports `output_filter` (regex matching), `head_lines`, and `tail_lines` parameters to prevent token overflow.
  - *Asynchronous Process Execution*: Commands that block or run indefinitely are automatically backgrounded by the session, returning a `Task ID` (e.g., `task_1`).
- **`check_background_command`**: Inspects status, reads output, and checks exit status code of background processes using their `task_id`.
- **`peek_task_output`**: Peeks at the currently accumulated output of a background task without blocking or changing its running status.
- **`kill_process`**: Kills a running background subprocess.
- **`sleep`**: Pauses execution for a specified number of seconds.
- **`ask_question`**: Prompts the user with a question or selectable options to confirm decisions or resolve ambiguity.
- **`ask_oracle`**: Consults a more advanced oracle model for advice or suggestions when stuck on a difficult reasoning step, logic problem, or complex code generation.

---

## Sandbox & Landlock Security Architecture

Chatty implements a layered security approach to sandbox tool executions and protect the host system.

### 1. User-Space Safety Filtering
Every command sent to `run_command` is statically parsed and verified by [validate_command_safety](file:///tmp/chatty/src/chatty/safety.py#L168). It blocks common shell commands (such as `cat`, `grep`, `find`, `cp`, `mv`, `rm`, `ls`, etc.) to force the LLM to use structured sandbox API tools rather than arbitrary shell execution.

Additionally, Python script executions (either run inline with `python -c` or loaded from `.py` files) are parsed into an Abstract Syntax Tree (AST) to extract their structural signature. If direct filesystem operations (like `open`, `write`, `os.remove`, etc.) are detected, the execution is blocked, and the chatbot prompts the user interactively to allow or deny the script, or whitelist its signature.

### 2. Kernel-Level Linux Landlock Sandboxing
On Linux (kernel version 5.13+), Chatty provides transparent, compile-on-demand process isolation using the Linux **Landlock LSM (Linux Security Module)**. 

#### Compilation Flow
When a chatbot session is initialized (see [ChatbotSession](file:///tmp/chatty/src/chatty/session.py#L216)), Chatty checks if:
1. The operating system is Linux.
2. The `--sandbox` option is enabled.
3. GCC or Clang is installed.

If these conditions are met, [compile_landlock_binary](file:///tmp/chatty/src/chatty/landlock.py#L13) builds the C-based wrapper helper [landlock_exec.c](file:///tmp/chatty/src/chatty/landlock_exec.c) into an executable binary `landlock_exec`. The binary is cached in the package directory or under the user's cache folder (`~/.cache/chatty/`) to avoid compilation overhead on subsequent startup.

#### Restrictive Access Rules
When executing a shell command via `run_command`, the execution argument is wrapped using [wrap_command_with_landlock](file:///tmp/chatty/src/chatty/landlock.py#L71), transforming it into:
```bash
landlock_exec --ro / --rw <sandbox_dir> --rw <temp_dir> -- /bin/sh -c "<command>"
```
The [landlock_exec](file:///tmp/chatty/src/chatty/landlock_exec.c) binary interacts with the Linux kernel to apply rules before spawning the target process:
* **Read-Only Paths (`--ro`)**: Allows reading from `/` (the root filesystem). This enables reading standard system executables, Python runtimes, shared libraries (`/lib`, `/usr/lib`), and general command dependencies.
* **Read-Write Paths (`--rw`)**: Specifically permits file writes, directory creation, and removals only inside the target `<sandbox_dir>` and the system's temporary directory. Any attempts to write elsewhere on the filesystem will fail with a `Permission denied` error.
* **No New Privileges**: Configures `prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0)` so that processes cannot escalate privileges.
* **Kernel Enforcement**: Enforces the ruleset using the `landlock_restrict_self` system call, sealing the sandbox rules for the current process and all future sub-processes.

#### Fallback Behavior
If Landlock is unavailable (non-Linux systems, older kernels, or missing compilers), Chatty automatically falls back to standard user-space process execution limited by the working directory (`cwd`) configuration and command regex validations.

#### Out-of-Sandbox Path Whitelisting
To allow the chatbot to access files or directories outside the sandboxed folder:
* **CLI Startup Whitelist**: Use the `--whitelist` (or `-w`) option to whitelist paths at startup.
  ```bash
  chatty -w /usr/include:ro -w /home/user/project
  ```
  Paths default to Read-Only (`ro`) unless suffixed with `:rw` (Read-Write).
* **Interactive Whitelisting**: If the agent attempts to read or write a file outside the sandbox that is not whitelisted, Chatty will prompt you interactively:
  * `[y]es`: Allow access for this operation once.
  * `[n]o`: Deny access.
  * `[a]lways`: Whitelist the specific file for the rest of the session.
  * `[p]arents`: Show a menu to select and recursively whitelist a parent directory.
* **Session Management**: Use the `/whitelist` (or `/permissions`) slash command to inspect or manage whitelisted paths on the fly.

---

## Extending Chatty with Skills

**Skills** are modular system prompt extensions and runbooks that give the LLM custom domain knowledge. Under the standard, a skill is structured as a directory containing a required `SKILL.md` file (for example, see [skills/greetings/SKILL.md](file:///tmp/chatty/src/chatty/skills/greetings/SKILL.md)) and optional helper subfolders:

```text
skills/database_guidelines/
├── SKILL.md          # Required: Main instruction file with YAML frontmatter
├── scripts/          # Optional: Helper scripts and utilities
├── examples/         # Optional: Reference implementations
├── resources/        # Optional: Additional assets or templates
└── references/       # Optional: Detailed documentation or manuals
```

### SKILL.md Structure
The `SKILL.md` must start with YAML frontmatter containing the `name` and `description` fields:
```markdown
---
name: database-operations
description: Instructions for SQL query formatting and schemas.
---
Always format queries in uppercase... (Rest of system guidelines)
```

### Discovery Locations
Chatty automatically discovers skills and plugins by searching recursive and machine-local customization roots:
1. **Workspace Customizations**: Searches for folders named `.agents/`, `.agent/`, `_agents/`, `_agent/`, or `.chatty/` by walking up parent directories from the current working directory to the repository root.
2. **Global Customizations**: Loads customizations globally from `~/.chatty/config/`.
3. **Built-in Package Skills**: Automatically loads default skills packaged with Chatty.
4. **Configuration Manifests**: Explicitly registers additional paths using `skills.json` and `plugins.json` configs placed in customization roots.

### Progressive Disclosure
To conserve LLM context window size, Chatty uses progressive disclosure:
- By default, only the **names** and **descriptions** of all discovered skills are injected into the system instructions.
- The full content of a skill is loaded into context **only** when explicitly activated.
- If a skill is running in static mode (or configured to load permanently), all skills can be loaded statically on initialization.

### The `/skill` command
* `/skill`: Lists all available skills and those currently explicitly loaded.
* `/skill name1 name2`: Explicitly activates one or more skills.
* `/skill clear`: Clears all explicitly loaded skills. Loaded skills persist through session clears (via `/clear`).

---

## Context Optimization

To operate efficiently over long conversations, Chatty implements two main context optimization mechanisms:

### 1. Smart History Pruning
Before sending messages to the LLM, the [prune_history](file:///tmp/chatty/src/chatty/session.py#L1161) method limits the total context:
- Old tool outputs exceeding `--max-history-tool-chars` characters are truncated in memory and annotated with a `[TRUNCATED]` note.
- The latest messages (configured via `--history-keep-messages`) are kept fully raw to ensure the model retains precise local context.

### 2. Prompt Caching
When `--prompt-caching` is explicitly enabled or when using static skills with OpenRouter, Chatty injects `cache_control: {"type": "ephemeral"}` metadata parameters:
- Applied to the system prompt message.
- Applied to the tools schema structure.
- Applied to the last two messages in the conversation queue.
This helps minimize token costs and reduces API response latencies for compatible model endpoints.

---

## Logging & Diagnostics

Chatty records session activity in a log file (default: `chatty.log`). The logging mechanism (configured in [setup_logging](file:///tmp/chatty/src/chatty/logging_setup.py#L42)) uses the [GlogFormatter](file:///tmp/chatty/src/chatty/logging_setup.py#L5) to output in standard Google logging syntax:

`Lyyyymmdd hh:mm:ss.uuuuuu process file:line] message`

- **`L`**: Level identifier letter (`D` for Debug, `I` for Info, `W` for Warning, `E` for Error, `F` for Critical/Fatal).
- **`yyyyymmdd hh:mm:ss.uuuuuu`**: Precise timestamps.
- **`process`**: OS Process ID.
- **`file:line`**: Source code location reporting the logging output.

*To disable file logging, run with `--log-file ""`.*

---

## Testing

Chatty comes with a comprehensive unittest suite located in `tests/`. You can execute tests from the project root:

```bash
python3 -m unittest discover tests
```

The test suite validates the following components:
- [test_caching_and_repeats.py](file:///tmp/chatty/tests/test_caching_and_repeats.py): Verifies prompt caching efficiency, EPHEMERAL headers, and handling of repeated prompts.
- [test_commands.py](file:///tmp/chatty/tests/test_commands.py): Exercises interactive slash commands (switching provider, model, modifying context parameters, system prompts).
- [test_cutoff.py](file:///tmp/chatty/tests/test_cutoff.py): Validation of message token truncation, history pruning, and prompt caching.
- [test_format.py](file:///tmp/chatty/tests/test_format.py): Verification of json, yaml, and clang-format code styling tools.
- [test_headless.py](file:///tmp/chatty/tests/test_headless.py): Validates running the chatbot session in headless mode.
- [test_landlock.py](file:///tmp/chatty/tests/test_landlock.py): Unittests for the Landlock sandboxing mechanism on Linux.
- [test_pdf_handling.py](file:///tmp/chatty/tests/test_pdf_handling.py): Validates PDF text extraction, encryption handling, scanned PDF detection, and OCR fallback logic.
- [test_logging.py](file:///tmp/chatty/tests/test_logging.py): Checks for Google-style Logging (glog) file outputs.
- [test_oracle.py](file:///tmp/chatty/tests/test_oracle.py): Tests the oracle query delegation logic, the `ask_oracle` tool, and oracle resolution.
- [test_safety.py](file:///tmp/chatty/tests/test_safety.py): Ensures commands and processes are validated for sandboxing.
- [test_sandbox_ops.py](file:///tmp/chatty/tests/test_sandbox_ops.py): Verification of file copy, move, delete, make-dir, search, and info tools.
- [test_session_persist.py](file:///tmp/chatty/tests/test_session_persist.py): Verifies saving and loading session states to/from JSON.
- [test_tool_stats.py](file:///tmp/chatty/tests/test_tool_stats.py): Checks tracking metrics for tool and external binary usage.
- [test_tools_modularity.py](file:///tmp/chatty/tests/test_tools_modularity.py): Checks the registration and separation of modular sandbox tools.
- [test_whitelist.py](file:///tmp/chatty/tests/test_whitelist.py): Asserts correct management of out-of-sandbox directory whitelisting (ro/rw permissions) and interactive whitelist prompts.
