# Familiar Security Whitepaper

**Version:** 1.0  
**Last Updated:** January 31, 2026  
**Classification:** Public

---

## Executive Summary

Familiar is a self-hosted AI agent framework designed with security as a foundational principle. Unlike cloud-hosted AI assistants, Familiar runs entirely on infrastructure you control—whether a Raspberry Pi in your home or a server in your data center.

**Key Security Properties:**

- **Data Sovereignty:** All data remains on your infrastructure. Familiar transmits only API requests to LLM providers.
- **Progressive Trust:** Users earn capabilities through demonstrated positive interactions, preventing immediate access to sensitive operations.
- **Defense in Depth:** Multiple security layers including capability-based access control, path sandboxing, command validation, and comprehensive audit logging.
- **Minimal Attack Surface:** No inbound network ports required for basic operation. No telemetry or data collection.

This document details Familiar's security architecture for security teams, compliance officers, and technical evaluators.

---

## Table of Contents

1. [Architecture Overview](#1-architecture-overview)
2. [Data Flow & Storage](#2-data-flow--storage)
3. [Authentication & Authorization](#3-authentication--authorization)
4. [Secure Tool Execution](#4-secure-tool-execution)
5. [Network Security](#5-network-security)
6. [Cryptographic Controls](#6-cryptographic-controls)
7. [Audit & Logging](#7-audit--logging)
8. [Operational Security](#8-operational-security)
9. [Incident Response](#9-incident-response)
10. [Compliance Considerations](#10-compliance-considerations)

---

## 1. Architecture Overview

### 1.1 Deployment Model

Familiar follows a **self-hosted deployment model**. The software runs entirely on customer-controlled infrastructure:

```
┌─────────────────────────────────────────────────────────────────┐
│                     CUSTOMER INFRASTRUCTURE                      │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │                      Familiar Agent                        │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐      │   │
│  │  │   Memory    │  │   Session   │  │    Audit    │      │   │
│  │  │   Store     │  │   Manager   │  │    Logs     │      │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘      │   │
│  │         │                │                │              │   │
│  │         └────────────────┼────────────────┘              │   │
│  │                          │                               │   │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐      │   │
│  │  │    Tool     │  │  Security   │  │   Channel   │      │   │
│  │  │   Registry  │  │   Layer     │  │   Gateway   │      │   │
│  │  └─────────────┘  └─────────────┘  └─────────────┘      │   │
│  └──────────────────────────┬───────────────────────────────┘   │
│                             │                                    │
│              ┌──────────────┴──────────────┐                    │
│              │      Local Storage          │                    │
│              │  ~/.familiar/ or /var/lib/   │                    │
│              └─────────────────────────────┘                    │
│                                                                  │
└──────────────────────────────┬──────────────────────────────────┘
                               │
                               │ HTTPS (outbound only)
                               ▼
                    ┌─────────────────────┐
                    │   LLM API Provider  │
                    │ (Anthropic/OpenAI)  │
                    │  or Local Ollama    │
                    └─────────────────────┘
```

### 1.2 Trust Boundaries

| Boundary | Description | Security Controls |
|----------|-------------|-------------------|
| **User ↔ Channel** | Messages from Telegram, Discord, etc. | Channel authentication, rate limiting |
| **Channel ↔ Agent** | Internal message passing | Session validation, trust level checks |
| **Agent ↔ Tools** | Tool/skill execution | Capability checks, path validation, command sanitization |
| **Agent ↔ LLM API** | API requests to LLM providers | TLS 1.3, API key authentication |
| **Agent ↔ Storage** | Local file I/O | File permissions, optional encryption |

### 1.3 Component Isolation

Familiar employs defense-in-depth through component isolation:

- **Process Isolation:** Runs as dedicated non-root user (`familiar`)
- **Filesystem Isolation:** systemd `ProtectSystem=strict`, sandboxed directories
- **Network Isolation:** No listening ports required; outbound-only by default
- **Capability Isolation:** Linux capabilities dropped; `NoNewPrivileges=true`

---

## 2. Data Flow & Storage

### 2.1 Data Categories

| Category | Description | Storage Location | Sensitivity |
|----------|-------------|------------------|-------------|
| **Conversation History** | User messages and agent responses | `history.json` | High |
| **Memory** | Long-term facts and preferences | `memory.json` | Medium-High |
| **Session Data** | Trust levels, capabilities, audit logs | `sessions/` | High |
| **Configuration** | Agent settings (no secrets) | `config.yaml` | Low |
| **Secrets** | API keys, tokens | `.env` (environment) | Critical |
| **Logs** | Operational and audit logs | `/var/log/familiar/` | Medium-High |

### 2.2 Data Flow Diagram

```
User Input (Telegram/Discord/CLI)
        │
        ▼
┌───────────────────┐
│ Channel Gateway   │ ◄── Rate limiting, basic validation
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Session Manager   │ ◄── Load/create session, check budget
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Security Layer    │ ◄── Trust level, capability checks
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Context Compiler  │ ◄── Build prompt with history, memory
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐     ┌───────────────────┐
│ LLM Provider      │────►│ External LLM API  │
└─────────┬─────────┘     └───────────────────┘
          │
          ▼
┌───────────────────┐
│ Tool Executor     │ ◄── Validate, sandbox, execute
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Audit Logger      │ ◄── Log all actions
└─────────┬─────────┘
          │
          ▼
     Response to User
```

### 2.3 Data Retention

| Data Type | Default Retention | Configurable |
|-----------|-------------------|--------------|
| Conversation History | 50 messages per user | Yes |
| Memory | Indefinite | Yes |
| Session Data | Indefinite | Yes |
| Audit Logs | 90 days | Yes |
| Operational Logs | 7 days (rotated) | Yes |

### 2.4 Data Deletion

Users can request data deletion through:
- CLI command: `familiar clear-history --user <id>`
- API: `agent.clear_history(user_id)`
- Manual: Delete files from storage directory

Session data is automatically pruned after configurable inactivity periods.

---

## 3. Authentication & Authorization

### 3.1 Progressive Trust Model

Familiar implements a **progressive trust model** where users earn capabilities through demonstrated positive behavior. This prevents compromised channels from immediately accessing sensitive operations.

```
┌─────────────────────────────────────────────────────────────────┐
│                     TRUST PROGRESSION                            │
│                                                                  │
│  STRANGER ────► KNOWN ────► TRUSTED ────► OWNER                 │
│  (0 interactions)  (10+)      (50+)     (manual grant)          │
│                                                                  │
│  Capabilities expand as trust increases:                         │
│                                                                  │
│  STRANGER: read:time, read:weather                              │
│  KNOWN:    + read:calendar, write:reminders, write:notes        │
│  TRUSTED:  + read:email, write:files, http:requests             │
│  OWNER:    + shell:execute, system:control (all capabilities)   │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 3.2 Trust Levels

| Level | Threshold | Capabilities Granted |
|-------|-----------|---------------------|
| **STRANGER** | Default | `read:time`, `read:weather` |
| **KNOWN** | 10+ positive interactions | + Calendar, reminders, notes, tasks, notifications |
| **TRUSTED** | 50+ positive, <5 negative | + Email, files, HTTP requests, browser (with confirmation) |
| **OWNER** | Manual grant only | All capabilities (still logged) |

### 3.3 Capability System

Familiar defines 22 discrete capabilities organized by risk level:

**Low Risk (STRANGER can access):**
- `read:time` - Current time/date
- `read:weather` - Weather information

**Medium Risk (KNOWN required):**
- `read:calendar`, `read:reminders`, `read:notes`, `read:tasks`
- `write:reminders`, `write:notes`, `write:calendar`, `write:tasks`
- `read:memory`, `write:memory`
- `send:notifications`

**High Risk (TRUSTED required, confirmation prompted):**
- `read:email`, `write:email`
- `read:files`, `write:files`
- `http:requests`
- `browser:control`

**Critical (OWNER only, always confirmed):**
- `shell:execute`
- `system:control`

### 3.4 Capability Checks

Every tool execution passes through capability verification:

```python
def check_skill_access(skill_name: str, session: SecureSession) -> tuple:
    """
    Returns: (allowed: bool, missing_caps: Set, needs_confirmation: bool)
    """
    required = SKILL_CAPABILITIES.get(skill_name, set())
    
    missing = set()
    needs_confirm = False
    
    for cap in required:
        if not session.has_capability(cap):
            missing.add(cap)
        elif session.requires_confirmation(cap):
            needs_confirm = True
    
    return len(missing) == 0, missing, needs_confirm
```

### 3.5 Session Management

Each user receives an isolated session containing:
- Unique session ID
- Trust level and score
- Explicitly granted/denied capabilities
- Daily budget tracking
- Interaction history
- Audit log

Sessions are stored separately per user and channel, preventing cross-contamination.

---

## 4. Secure Tool Execution

### 4.1 Shell Command Validation

All shell commands are validated against dangerous patterns before execution:

```python
DANGEROUS_SHELL_PATTERNS = [
    r'\$\(',           # Command substitution $(...)
    r'`',              # Backtick substitution
    r'\|\s*\w',        # Pipe to command
    r';\s*\w',         # Command chaining
    r'&&\s*\w',        # AND chaining
    r'\|\|\s*\w',      # OR chaining
    r'>>\s*/',         # Append redirect to absolute path
    r'>\s*/',          # Redirect to absolute path
    r'\bsudo\b',       # sudo commands
    r'\brm\s+-rf\s+/', # rm -rf with absolute path
    r'\bchmod\s+777',  # World-writable permissions
    r'\bcurl\b.*\|\s*\b(bash|sh)\b',  # curl | bash
    r'\bwget\b.*\|\s*\b(bash|sh)\b',  # wget | bash
]
```

Commands matching these patterns are rejected with an explanatory error.

### 4.2 Path Sandboxing

File operations are restricted to configured sandbox directories:

```python
def _validate_path(self, path: str, must_exist: bool = False) -> tuple:
    """Validate path is within sandboxed directories."""
    resolved = Path(path).resolve()  # Defeats ../ traversal
    
    for sandbox_dir in self.sandboxed_dirs:
        try:
            resolved.relative_to(sandbox_dir.resolve())
            return True, "", resolved
        except ValueError:
            continue
    
    return False, f"Path outside sandbox: {resolved}", None
```

Default sandboxed directories:
- User home directory
- `/tmp`
- Configured data directory

### 4.3 Tool Execution Limits

| Limit | Default | Purpose |
|-------|---------|---------|
| Max iterations per request | 15 | Prevent infinite loops |
| Shell command timeout | 60s | Prevent hanging commands |
| HTTP request timeout | 30s | Prevent hanging requests |
| Max file read size | 50KB | Prevent memory exhaustion |
| Max command length | 10,000 chars | Prevent buffer issues |

### 4.4 Atomic File Operations

File writes use atomic patterns to prevent corruption:

```python
def _atomic_write(path: Path, content: str):
    """Write file atomically using rename."""
    tmp_path = path.with_suffix(path.suffix + '.tmp')
    try:
        with open(tmp_path, 'w') as f:
            f.write(content)
            f.flush()
            os.fsync(f.fileno())
        tmp_path.replace(path)  # Atomic on POSIX
    except Exception:
        if tmp_path.exists():
            tmp_path.unlink()
        raise
```

---

## 5. Network Security

### 5.1 Network Architecture

Familiar requires **no inbound network ports** for basic operation:

```
┌─────────────────────────────────────────────────────────────────┐
│                        NETWORK FLOWS                             │
│                                                                  │
│  OUTBOUND (required):                                           │
│  ├── api.anthropic.com:443 (Claude API)                         │
│  ├── api.openai.com:443 (OpenAI API)                            │
│  └── localhost:11434 (Ollama, if local)                         │
│                                                                  │
│  OUTBOUND (optional):                                           │
│  ├── api.telegram.org:443 (Telegram channel)                    │
│  ├── discord.com:443 (Discord channel)                          │
│  └── Various webhook endpoints                                   │
│                                                                  │
│  INBOUND (optional, for dashboard):                             │
│  └── :8080 (local network only recommended)                     │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

### 5.2 TLS Configuration

All external API communications use TLS 1.2+ with certificate verification enabled. The underlying HTTP libraries (httpx, requests) enforce secure defaults.

### 5.3 Firewall Configuration

The installation script configures UFW with secure defaults:

```bash
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
# Dashboard only from local network (if enabled)
ufw allow from 192.168.0.0/16 to any port 8080
```

### 5.4 Remote Access

For remote access to a Pi-hosted instance, we recommend:

1. **Cloudflare Tunnel** (preferred) - No open ports, authenticated access
2. **WireGuard VPN** - Encrypted tunnel to home network
3. **SSH tunneling** - Port forwarding over SSH

We explicitly discourage direct port forwarding from the internet.

---

## 6. Cryptographic Controls

### 6.1 Secrets Management

Secrets are managed through environment variables, never stored in configuration files:

| Secret | Environment Variable | Purpose |
|--------|---------------------|---------|
| Anthropic API key | `ANTHROPIC_API_KEY` | LLM authentication |
| OpenAI API key | `OPENAI_API_KEY` | LLM authentication |
| Telegram token | `TELEGRAM_BOT_TOKEN` | Bot authentication |
| Discord token | `DISCORD_BOT_TOKEN` | Bot authentication |
| Webhook secret | `FAMILIAR_WEBHOOK_SECRET` | Webhook validation |
| Encryption key | `FAMILIAR_ENCRYPTION_KEY` | Data encryption (optional) |

The `.env` file has restricted permissions (`chmod 600`).

### 6.2 Encryption at Rest (Optional)

When enabled, sensitive data is encrypted using Fernet (AES-128-CBC with HMAC):

```yaml
# config.yaml
security:
  encrypt_sessions: true
  encrypt_memory: true
  encryption_key_env: FAMILIAR_ENCRYPTION_KEY
```

Key generation:
```bash
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
```

### 6.3 Session Token Generation

Session IDs are generated using cryptographically secure random numbers:

```python
session_id: str = secrets.token_hex(16)  # 128 bits of entropy
```

### 6.4 Webhook Signature Verification

Inbound webhooks are verified using HMAC-SHA256:

```python
def verify_webhook(payload: bytes, signature: str, secret: str) -> bool:
    expected = hmac.new(secret.encode(), payload, hashlib.sha256).hexdigest()
    return hmac.compare_digest(signature, expected)
```

---

## 7. Audit & Logging

### 7.1 Audit Events

Every security-relevant action is logged:

| Event | Data Captured |
|-------|---------------|
| `session_created` | User ID, channel, security mode |
| `trust_upgraded` | Previous level, new level, trigger |
| `capability_granted` | Capability, granter |
| `capability_denied` | Capability, reason |
| `tool_executed` | Tool name, input preview, user |
| `tool_blocked` | Tool name, missing capabilities |
| `budget_exceeded` | User, amount, limit |
| `confirmation_requested` | Action, token |
| `confirmation_confirmed` | Token |

### 7.2 Log Format

Logs are structured JSON for easy ingestion by SIEM systems:

```json
{
  "timestamp": "2026-01-31T21:15:00.000000+00:00",
  "level": "INFO",
  "logger": "familiar.core.security",
  "message": "Tool executed",
  "event": "tool_executed",
  "user_id": "user123",
  "channel": "telegram",
  "tool": "read_file",
  "input_preview": "{\"path\": \"/home/user/notes.txt\"}",
  "trust_level": "known",
  "request_id": "abc123"
}
```

### 7.3 PII Redaction

Sensitive data is automatically redacted from logs:

```python
REDACTION_PATTERNS = [
    (r'sk-ant-[a-zA-Z0-9-]+', '[REDACTED_API_KEY]'),
    (r'sk-[a-zA-Z0-9]+', '[REDACTED_API_KEY]'),
    (r'\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b', '[REDACTED_CC]'),
    (r'\b\d{3}-\d{2}-\d{4}\b', '[REDACTED_SSN]'),
    (r'password["\']?\s*[:=]\s*["\']?[^"\'&\s]+', 'password=[REDACTED]'),
    (r'bearer\s+[a-zA-Z0-9._-]+', 'bearer [REDACTED]'),
]
```

### 7.4 Log Retention & Rotation

Default configuration:
- Operational logs: 7 days, rotated daily
- Audit logs: 90 days (configurable to 6+ years for HIPAA)
- Max file size: 50MB before rotation
- Compressed after rotation

---

## 8. Operational Security

### 8.1 Principle of Least Privilege

Familiar runs with minimal privileges:

- **User:** Dedicated `familiar` system user (no sudo)
- **Filesystem:** Read-only access to system, write only to data directories
- **Capabilities:** All Linux capabilities dropped
- **Processes:** `NoNewPrivileges=true` prevents privilege escalation

### 8.2 Dependency Management

- Dependencies pinned in `requirements.txt`
- Regular updates recommended via `pip install --upgrade`
- Security advisories monitored for critical dependencies

### 8.3 Update Process

```bash
# Standard update
cd /opt/familiar/familiar
sudo -u familiar git pull
sudo -u familiar /opt/familiar/venv/bin/pip install -r requirements.txt
sudo systemctl restart familiar

# Verify
sudo systemctl status familiar
tail -f /var/log/familiar/familiar.log
```

### 8.4 Backup & Recovery

Recommended backup strategy:

```bash
# Daily backup
tar -czf familiar-backup-$(date +%Y%m%d).tar.gz \
  /opt/familiar/.familiar \
  /var/lib/familiar

# Retention: 30 daily, 12 monthly
```

Recovery:
```bash
sudo systemctl stop familiar
tar -xzf familiar-backup-YYYYMMDD.tar.gz -C /
sudo systemctl start familiar
```

---

## 9. Incident Response

### 9.1 Indicators of Compromise

Monitor for:
- Unexpected trust level changes
- High volume of blocked tool executions
- Budget exhaustion without corresponding activity
- Unusual API call patterns
- Failed authentication attempts

### 9.2 Response Procedures

**Suspected Compromise:**

1. Stop the service: `sudo systemctl stop familiar`
2. Preserve logs: `cp -r /var/log/familiar /tmp/familiar-incident-$(date +%s)`
3. Rotate API keys at provider consoles
4. Review audit logs for unauthorized actions
5. Reset affected user sessions
6. Investigate root cause before restarting

**API Key Exposure:**

1. Immediately rotate key at provider console
2. Update `.env` with new key
3. Restart service
4. Review logs for unauthorized usage during exposure window
5. Contact provider if suspicious activity detected

### 9.3 Security Contact

Report security vulnerabilities to: Georgescottfoley@proton.me

We follow responsible disclosure practices and aim to acknowledge reports within 48 hours.

---

## 10. Compliance Considerations

### 10.1 Shared Responsibility Model

| Control Domain | Familiar (Software) | Customer (Operator) |
|----------------|-------------------|---------------------|
| Application security | ✓ Develops secure code | Applies updates |
| Access control logic | ✓ Implements RBAC | Configures trust levels |
| Encryption capability | ✓ Provides feature | Enables and manages keys |
| Network security | ✓ Documents requirements | Implements firewall |
| API credentials | | ✓ Manages securely |
| Physical security | | ✓ Secures hardware |
| Backup & recovery | ✓ Documents procedures | Implements |
| Monitoring | ✓ Provides logging | Configures alerting |
| Incident response | ✓ Provides guidance | Executes |

### 10.2 Compliance Framework Alignment

| Framework | Familiar Capabilities |
|-----------|---------------------|
| **SOC2 Trust Principles** | Access control, audit logging, encryption |
| **HIPAA Security Rule** | Access controls, audit controls, integrity controls |
| **GDPR** | Data minimization, right to deletion, encryption |
| **CCPA** | Data inventory, deletion capability |

### 10.3 Documentation Available

- This Security Whitepaper
- HIPAA Deployment Guide
- Hardening Checklist
- Risk Assessment Template
- Shared Responsibility Matrix

---

## Appendix A: Security Configuration Reference

```yaml
# Recommended production security configuration

agent:
  security_mode: balanced  # or 'paranoid' for high-security
  sandboxed_directories:
    - /var/lib/familiar
    - /tmp/familiar

security:
  encrypt_sessions: true
  encrypt_memory: true
  encryption_key_env: FAMILIAR_ENCRYPTION_KEY
  session_timeout_hours: 24
  max_failed_attempts: 5

observability:
  audit_logging: true
  pii_redaction: true
  log_retention_days: 90
  json_format: true

resilience:
  enabled: true
  max_attempts: 3
  circuit_breaker: true
```

---

## Appendix B: Threat Model Summary

| Threat | Likelihood | Impact | Mitigations |
|--------|------------|--------|-------------|
| Compromised channel account | Medium | High | Progressive trust, capability limits |
| API key theft | Low | High | Env vars, file permissions, encryption |
| Path traversal | Low | High | Path validation, sandboxing |
| Command injection | Low | Critical | Pattern blocking, validation |
| Denial of service | Medium | Medium | Rate limiting, budgets |
| Data exfiltration via LLM | Low | Medium | Local Ollama option, audit logging |
| Physical access to Pi | Low | Critical | Disk encryption (OS-level), physical security |

---

## Document History

| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2026-01-31 | Initial release |

---

*This document describes security features of Familiar v1.3.0. Security properties depend on correct deployment and configuration. Customers are responsible for securing their deployment environment.*
