Metadata-Version: 2.4
Name: velocitymaps
Version: 0.1.5
Summary: Network Topology Discovery & Visualization using SNMP CDP/LLDP
Home-page: https://github.com/scottpeterman/velocitymaps
Author: Scott Peterman
Author-email: scottpeterman@gmail.com
License: GPLv3
Project-URL: Bug Reports, https://github.com/scottpeterman/velocitymaps/issues
Project-URL: Source, https://github.com/scottpeterman/velocitymaps
Project-URL: Documentation, https://github.com/scottpeterman/velocitymaps#readme
Keywords: network,topology,discovery,snmp,cdp,lldp,visualization,cisco,arista,juniper,aruba,pyqt6,network-automation
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: X11 Applications :: Qt
Classifier: Environment :: Win32 (MS Windows)
Classifier: Environment :: MacOS X
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Telecommunications Industry
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: System :: Networking
Classifier: Topic :: System :: Networking :: Monitoring
Classifier: Topic :: System :: Systems Administration
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: PyQt6>=6.4.0
Requires-Dist: PyQt6-WebEngine>=6.4.0
Requires-Dist: pysnmp>=6.0.0
Requires-Dist: netmiko>=4.0.0
Requires-Dist: napalm>=4.0.0
Requires-Dist: paramiko>=3.0.0
Requires-Dist: netaddr>=0.8.0
Requires-Dist: netutils>=1.0.0
Requires-Dist: ntc_templates>=3.0.0
Requires-Dist: textfsm>=1.1.0
Requires-Dist: networkx>=2.6.0
Requires-Dist: matplotlib>=3.5.0
Requires-Dist: igraph>=0.10.0
Requires-Dist: n2g>=0.3.0
Requires-Dist: PyYAML>=6.0
Requires-Dist: ruamel.yaml>=0.17.0
Requires-Dist: lxml>=4.9.0
Requires-Dist: openpyxl>=3.0.0
Requires-Dist: numpy>=1.21.0
Requires-Dist: requests>=2.28.0
Requires-Dist: cryptography>=3.4.0
Requires-Dist: bcrypt>=4.0.0
Requires-Dist: colorama>=0.4.4
Requires-Dist: rich>=12.0.0
Requires-Dist: click>=8.0.0
Requires-Dist: ttp>=0.9.0
Requires-Dist: ttp-templates>=0.3.0
Requires-Dist: secure-cartography>=0.9.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-qt>=4.0.0; extra == "dev"
Requires-Dist: black>=22.0.0; extra == "dev"
Requires-Dist: flake8>=4.0.0; extra == "dev"
Requires-Dist: mypy>=0.950; extra == "dev"
Provides-Extra: junos
Requires-Dist: junos-eznc>=2.6.0; extra == "junos"
Requires-Dist: ncclient>=0.6.0; extra == "junos"
Provides-Extra: arista
Requires-Dist: pyeapi>=1.0.0; extra == "arista"
Provides-Extra: procurve
Requires-Dist: napalm-procurve>=0.6.0; extra == "procurve"
Provides-Extra: full
Requires-Dist: junos-eznc>=2.6.0; extra == "full"
Requires-Dist: ncclient>=0.6.0; extra == "full"
Requires-Dist: pyeapi>=1.0.0; extra == "full"
Requires-Dist: napalm-procurve>=0.6.0; extra == "full"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

VelocityMaps SNMP Network Discovery Suite
[![PyPI version](https://badge.fury.io/py/velocitymaps.svg)](https://badge.fury.io/py/velocitymaps)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

A comprehensive network topology discovery solution using SNMP (CDP/LLDP) with a PyQt6 graphical interface. Recursively discovers network devices, maps connections, and generates topology data for visualization.

## Table of Contents

- [Overview](#overview)
  - [Architecture](#architecture)
  - [Components](#components)
  - [Installation](#installation)
  - [Configuration](#configuration)
  - [Usage](#usage)
  - [Output Formats](#output-formats)
  - [GUI Reference](#gui-reference)
  - [Troubleshooting](#troubleshooting)
  - [Design Decisions](#design-decisions)

---

## Overview

This solution provides automated network topology discovery by:

1. **Querying seed devices** via SNMP to discover neighbors (CDP and LLDP)
   2. **Recursively crawling** discovered neighbors up to a configurable depth
   3. **Resolving interface mappings** from ifIndex to interface names
   4. **Generating topology JSON** with bidirectional connection data
   5. **Providing a GUI** for real-time monitoring and control

### Key Features

- **SNMPv2c and SNMPv3 support** via unified credential manager
  - **Credential caching** eliminates redundant authentication testing
  - **Multi-vendor support**: Cisco, Arista, Juniper (extensible)
  - **Interface table resolution** for accurate port naming
  - **Multiple domain suffix stripping** for clean hostnames
  - **CDP + LLDP fusion** with deduplication
  - **Real-time progress tracking** in GUI
  - **Color-coded log output** with filtering

---
## Screenshots

### Discovery Interface

<p align="center">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/discovery_dark.png" alt="Discovery - Dark Theme" width="45%">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/discovery_light.png" alt="Discovery - Light Theme" width="45%">
</p>

### Topology Generation

<p align="center">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/topology_dark.png" alt="Topology - Dark Theme" width="45%">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/topology_light.png" alt="Topology - Light Theme" width="45%">
</p>

### Interactive Map Viewer

<p align="center">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/maps_light.png" alt="Map Viewer" width="80%">
</p>

### Built-in Editors

<p align="center">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/seeds_editor_light.png" alt="Seeds Editor" width="45%">
  <img src="https://raw.githubusercontent.com/scottpeterman/velocitymaps/main/screenshots/creds_edior_light.png" alt="Credentials Editor" width="45%">
</p>

## Architecture

### High-Level Flow

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                           SNMP Discovery Pipeline                            │
└─────────────────────────────────────────────────────────────────────────────┘

  ┌──────────────┐     ┌──────────────────┐     ┌───────────────────┐
  │  Seed CSV    │────▶│  ndp_recursive   │────▶│  Discovery JSON   │
  │  + Creds     │     │  (Phase 1)       │     │  per device       │
  └──────────────┘     └──────────────────┘     └─────────┬─────────┘
                              │                           │
                              │ Recursive                 │
                              │ Discovery                 ▼
                              │                  ┌───────────────────┐
                              ▼                  │  snmp_to_topology │
                       ┌─────────────┐           │  (Phase 2)        │
                       │   Queue     │           └─────────┬─────────┘
                       │  Depth 0..N │                     │
                       └─────────────┘                     ▼
                                                 ┌───────────────────┐
                                                 │  Topology JSON    │
                                                 │  (network.json)   │
                                                 └───────────────────┘
```

### Discovery Process

```
                    ┌─────────────────────────────────────┐
                    │         Start Discovery              │
                    └─────────────────┬───────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────────┐
                    │    Load Seed Devices from CSV       │
                    └─────────────────┬───────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────────┐
                    │    Initialize Credential Manager    │
                    │    (Load from YAML)                 │
                    └─────────────────┬───────────────────┘
                                      │
                    ┌─────────────────▼───────────────────┐
             ┌──────│         For Each Depth Level        │◀─────┐
             │      └─────────────────┬───────────────────┘      │
             │                        │                          │
             │                        ▼                          │
             │      ┌─────────────────────────────────────┐      │
             │      │      For Each Device in Queue       │      │
             │      └─────────────────┬───────────────────┘      │
             │                        │                          │
             │                        ▼                          │
             │      ┌─────────────────────────────────────┐      │
             │      │  1. Test/Cache Credential           │      │
             │      │  2. Query Interface Table           │      │
             │      │  3. Query CDP Neighbors             │      │
             │      │  4. Query LLDP Neighbors            │      │
             │      │  5. Save device.json, cdp.json,     │      │
             │      │     lldp.json                       │      │
             │      └─────────────────┬───────────────────┘      │
             │                        │                          │
             │                        ▼                          │
             │      ┌─────────────────────────────────────┐      │
             │      │  Add Valid Neighbors to Next Queue  │──────┘
             │      │  (if depth < max_depth)             │
             │      └─────────────────────────────────────┘
             │
             │      ┌─────────────────────────────────────┐
             └─────▶│        All Depths Complete          │
                    └─────────────────┬───────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────────┐
                    │     Save discovery_summary.json     │
                    └─────────────────────────────────────┘
```

### Credential Resolution Flow

```
┌─────────────────────────────────────────────────────────────────┐
│                  Credential Manager Flow                         │
└─────────────────────────────────────────────────────────────────┘

    Request Credential
    for Device IP
          │
          ▼
    ┌───────────────┐     Yes    ┌─────────────────┐
    │ In Cache?     │───────────▶│ Return Cached   │
    └───────┬───────┘            │ Credential      │
            │ No                 └─────────────────┘
            ▼
    ┌───────────────────────────────────────┐
    │ Test Credentials in Order:            │
    │   1. SNMPv3 credentials (if present)  │
    │   2. SNMPv2c credentials              │
    └───────────────────┬───────────────────┘
                        │
                        ▼
    ┌───────────────────────────────────────┐
    │ For each credential:                  │
    │   - Query sysName                     │
    │   - If success: cache + return        │
    │   - If fail: try next                 │
    └───────────────────┬───────────────────┘
                        │
            ┌───────────┴───────────┐
            ▼                       ▼
    ┌───────────────┐       ┌───────────────┐
    │ Found Working │       │ No Working    │
    │ Credential    │       │ Credential    │
    └───────┬───────┘       └───────┬───────┘
            │                       │
            ▼                       ▼
    ┌───────────────┐       ┌───────────────┐
    │ Add to Cache  │       │ Return Error  │
    │ Return Result │       └───────────────┘
    └───────────────┘
```

---

## Components

### 1. `ndp_discover.py` - Discovery Engine

The core recursive SNMP discovery script.

**Responsibilities:**
- Load seed devices from CSV
  - Manage SNMP credential testing and caching
  - Query CDP and LLDP neighbor tables
  - Build interface index-to-name mappings
  - Recursive neighbor discovery up to max depth
  - Save per-device JSON results

**Key Classes/Functions:**
- `SNMPCredentialManager` - Unified v2c/v3 credential handling
  - `discover_device()` - Main device discovery routine
  - `get_cdp_neighbors()` - CDP neighbor extraction
  - `get_lldp_neighbors()` - LLDP neighbor extraction
  - `get_interface_table()` - Interface mapping (ifIndex → name)

### 2. `snmp_to_topology_gui.py` - Topology Generator

Converts discovery results into unified topology format.

**Responsibilities:**
- Load device summaries from discovery output
  - Build topology from LLDP data (primary)
  - Augment with CDP data (deduplicated)
  - Ensure bidirectional connections
  - Normalize interface names across vendors
  - Detect connected components
  - Annotate with location data (optional)

**Output Format:**
```json
{
  "device-name": {
    "node_details": {
      "ip": "10.0.0.1",
      "platform": "Cisco IOSv IOS 15.9(3)M8",
      "location": "datacenter-1"
    },
    "peers": {
      "peer-name": {
        "ip": "10.0.0.2",
        "platform": "Arista vEOS-lab EOS 4.33.1F",
        "connections": [
          ["Gi0/1", "Eth1"],
          ["Gi0/2", "Eth2"]
        ]
      }
    }
  }
}
```

### 3. `snmp_discovery_gui.py` - PyQt6 GUI

Graphical interface for the discovery pipeline.

**Features:**
- **Discovery Tab**: Configure and run discovery
  - **Topology Tab**: Generate topology from results
  - **Results Tab**: Browse discovered devices

**GUI Components:**
- Real-time stats (total/success/failed/depth/queue)
  - Progress bar tracking depth completion
  - Queue table showing device status
  - Color-coded log viewer with filtering
  - Auto-scroll and verbose toggle

### 4. `snmp_credentials.py` - Credential Manager

Manages SNMP v2c and v3 credentials with caching.

**Features:**
- YAML configuration with environment variable substitution
  - Priority ordering (v3 preferred by default)
  - Per-device credential caching
  - Statistics tracking (hits/misses/attempts)

---

## Installation

### Prerequisites

```bash
# Python 3.10+
python --version

# Required packages
pip install pysnmp>=6.0.0
pip install PyQt6
pip install pyyaml
```

### MIB Setup

The discovery requires compiled MIBs for CDP and LLDP:

```
project/
├── compiled_mibs/
│   ├── cisco/
│   │   └── CISCO-CDP-MIB.py
│   ├── IF-MIB.py
│   └── LLDP-MIB.py
```

MIBs can be compiled using `mibdump.py` from pysmi or obtained from vendor sources.

### Directory Structure

```
snmp_maps/
├── ndp_recursive_6.py          # Discovery engine
├── snmp_to_topology_gui.py     # Topology generator
├── snmp_discovery_gui.py       # PyQt6 GUI
├── snmp_credentials.py         # Credential manager
├── credentials.yaml            # SNMP credentials
├── devices.csv                 # Seed devices
├── compiled_mibs/              # Compiled MIB files
│   ├── cisco/
│   └── ...
└── results/                    # Discovery output
    ├── device-1/
    │   ├── device.json
    │   ├── cdp.json
    │   └── lldp.json
    ├── device-2/
    └── discovery_summary.json
```

---

## Configuration

### Credentials File (`credentials.yaml`)

```yaml
# SNMPv2c credentials
v2c:
  - name: readonly
    community: public
    
  - name: production
    community: ${SNMP_COMMUNITY}  # Environment variable

# SNMPv3 credentials  
v3:
  - name: admin-v3
    user: snmpadmin
    auth_protocol: SHA
    auth_password: ${SNMP_AUTH_PASS}
    priv_protocol: AES
    priv_password: ${SNMP_PRIV_PASS}
    
  - name: noauth-v3
    user: monitor
    security_level: noAuthNoPriv
```

### Seed Devices CSV

```csv
Hostname,Vendor,Model,OS Version,Serial Number,Site,Location,Rack,Status,Tenant,IP Address
wan-core-1,cisco,7200,15.2(4)M11,FTX1234,DC1,Row-A,Rack-1,Active,,172.16.100.1
spine-1,arista,vEOS,4.33.1F,SN001,DC1,Row-B,Rack-2,Active,,172.16.100.10
```

**Required columns:** `Hostname`, `Vendor`  
**Optional columns:** `IP Address` (fallback), `Site` (location annotation)

### Domain Suffixes

Domain suffixes are stripped from hostnames for clean folder naming:

- **Discovery format:** `lab.local,home.com` (no leading dot)
  - **Topology format:** `.lab.local,.home.com` (leading dot)

The GUI automatically converts between formats.

---

## Usage

### GUI Mode

```bash
python snmp_discovery_gui.py
```

1. **Discovery Tab:**
   - Set seed devices CSV path
   - Set credentials YAML path
   - Set output directory
   - Enter domain suffixes (comma-separated)
   - Set max depth (0 = seed only)
   - Enable/disable verbose mode
   - Click "Start Discovery"

   2. **Topology Tab:**
      - Results directory auto-populated from discovery
      - Set output JSON filename
      - Set domain suffixes (with dots)
      - Click "Generate Topology"

   3. **Results Tab:**
      - Browse to results directory
      - Click "Load Results"
      - Click devices to view JSON details

### Command Line Mode

**Discovery:**
```bash
python ndp_discover.py [-v] <devices.csv> <credentials.yaml> <output_dir> <domains> [max_depth]

# Example
python ndp_discover.py -v devices.csv credentials.yaml ./results 'lab.local,home.com' 5
```

**Topology Generation:**
```bash
python snmp_to_topology_gui.py <discovery_dir> [output_file] [domain_suffixes] [locations_csv]

# Example
python snmp_to_topology_gui.py ./results network.json '.lab.local,.home.com' devices.csv
```

---

## Output Formats

### Per-Device Discovery Output

**`device.json`** - Device metadata and interface table:
```json
{
  "hostname": "usa-rtr-1",
  "fqdn": "usa-rtr-1.lab.local",
  "ip": "172.16.100.2",
  "vendor": "cisco",
  "credential": "v2c:readonly",
  "credential_version": "v2c",
  "depth": 1,
  "timestamp": "2024-12-03T20:00:00",
  "status": "success",
  "sysDescr": "Cisco IOS Software...",
  "interface_table": {
    "1": {"ifName": "Gi0/0", "ifDescr": "GigabitEthernet0/0", "ifAlias": "UPLINK"},
    "2": {"ifName": "Gi0/1", "ifDescr": "GigabitEthernet0/1", "ifAlias": ""}
  }
}
```

**`cdp.json`** - CDP neighbor data:
```json
{
  "neighbor_count": 2,
  "neighbors": [
    {
      "index": "3.1",
      "local_port": "Gi0/0",
      "device_id": "wan-core-1",
      "platform": "Cisco 7206VXR",
      "remote_port": "Gi1/0",
      "ip_address": "172.16.100.1"
    }
  ]
}
```

**`lldp.json`** - LLDP neighbor data:
```json
{
  "neighbor_count": 3,
  "neighbors": [
    {
      "index": "0.2.1",
      "local_port_index": "2",
      "system_name": "eng-spine-1",
      "system_description": "Arista Networks EOS version 4.33.1F...",
      "chassis_id": "50:01:00:01:00:00",
      "port_id": "Ethernet1",
      "port_description": "INT::usa-rtr-1::Gi0/1",
      "management_address": "172.16.200.1"
    }
  ]
}
```

### Topology Output

**`network.json`** - Complete topology:
```json
{
  "wan-core-1": {
    "node_details": {
      "ip": "172.16.100.1",
      "platform": "Cisco 7200 IOS 15.2(4)M11",
      "location": "DC1"
    },
    "peers": {
      "usa-rtr-1": {
        "ip": "172.16.100.2",
        "platform": "Cisco IOSv IOS 15.9(3)M8",
        "connections": [
          ["Et1/1", "Gi0/0"]
        ]
      }
    }
  }
}
```

---

## GUI Reference

### Log Message Prefixes

The topology generator uses parseable prefixes for GUI color-coding:

| Prefix | Color | Description |
|--------|-------|-------------|
| `[SECTION]` | Blue (bold) | Section headers |
| `[INFO]` | Light gray | Informational messages |
| `[DEVICE]` | Green | Per-device processing |
| `[STAT]` | Cyan | Statistics |
| `[WARN]` | Orange (bold) | Warnings |
| `[ERROR]` | Red (bold) | Errors |
| `[SUCCESS]` | Light green | Success messages |
| `[SUMMARY]` | Purple (bold) | Summary lines |

### Discovery Log Prefixes

| Prefix | Color | Description |
|--------|-------|-------------|
| `[VERBOSE]` | Gray | Debug messages (filterable) |
| `[CRED]` | Purple | Credential manager messages |

---

## Troubleshooting

### Common Issues

**1. No working credential found**
```
ERROR: No working credential found (2 tested)
```
- Verify SNMP is enabled on device
  - Check community string / v3 credentials
  - Verify network connectivity (UDP 161)
  - Check ACLs on device

**2. DNS resolution failed**
```
WARNING: DNS resolution failed for device.lab.local
FALLBACK: Using fallback IP: 172.16.100.1
```
- Ensure DNS is configured or provide IP in CSV
  - The script will use CDP/LLDP management addresses as fallback

**3. No LLDP neighbors**
```
Found 0 valid LLDP neighbors
```
- Verify LLDP is enabled on device
  - Some devices (older Cisco) only support CDP
  - Check LLDP holdtime hasn't expired

**4. Interface not resolved**
```
[STAT] Skipped (no local port): 5
```
- Interface table may be incomplete
  - LLDP local port index doesn't match ifIndex
  - Enable verbose mode to debug interface mapping

**5. Topology shows disconnected components**
```
[WARN] 2 disconnected network segments
```
- Some devices may not have been discovered
  - Check if devices are unreachable
  - Verify max_depth is sufficient

### Debug Mode

Enable verbose output for detailed debugging:

```bash
# CLI
python ndp_discover.py -v ...

# GUI
Check "Verbose Mode" checkbox
```

Verbose output includes:
- SNMP walk iterations and results
  - Credential testing details
  - Interface resolution attempts
  - Neighbor queue processing

---

## Design Decisions

### Why Cache-First Credential Lookup?

In large networks, the same credential often works for many devices. Testing credentials is expensive (SNMP timeouts). The cache-first approach:
- Tests each credential only once per IP
  - Eliminates redundant authentication on subsequent queries
  - Provides 50%+ cache hit rates in typical deployments

### Why LLDP Primary, CDP Secondary?

LLDP is vendor-neutral and provides richer metadata (management addresses, system capabilities). CDP is Cisco-specific but still valuable:
- LLDP: Primary source, processed first
  - CDP: Augments LLDP, deduplicated to avoid duplicates

### Why Bidirectional Topology?

Network connections are inherently bidirectional. Storing only one direction would require lookups in both directions. Bidirectional storage:
- Simplifies topology queries
  - Enables easy path finding
  - Matches how networks actually work

### Why Interface Table Resolution?

LLDP provides local port as an ifIndex number, not a name. Without resolution:
- You'd see: `local_port: "3"`
  - With resolution: `local_port: "Gi0/1"`

The interface table query (IF-MIB) provides the mapping.

### Why Multiple Domain Suffix Support?

Enterprise networks often span multiple DNS domains:
- `device.corp.example.com`
  - `device.lab.example.com`
  - `device.dc1.example.com`

Multiple suffix support ensures clean hostname extraction regardless of domain.

---

## License

GPLv3 - see license file

## Author

Scott Peterman - Principal Infrastructure Engineer

