velocitymaps 0.1.5

Network Topology Discovery & Visualization using SNMP CDP/LLDP

VelocityMaps SNMP Network Discovery Suite

A comprehensive network topology discovery solution using SNMP (CDP/LLDP) with a PyQt6 graphical interface. Recursively discovers network devices, maps connections, and generates topology data for visualization.

Table of Contents

• Overview
  • Architecture
  • Components
  • Installation
  • Configuration
  • Usage
  • Output Formats
  • GUI Reference
  • Troubleshooting
  • Design Decisions

---

Overview

This solution provides automated network topology discovery by:

1. Querying seed devices via SNMP to discover neighbors (CDP and LLDP) 2. Recursively crawling discovered neighbors up to a configurable depth 3. Resolving interface mappings from ifIndex to interface names 4. Generating topology JSON with bidirectional connection data 5. Providing a GUI for real-time monitoring and control

Key Features

• SNMPv2c and SNMPv3 support via unified credential manager
  • Credential caching eliminates redundant authentication testing
  • Multi-vendor support: Cisco, Arista, Juniper (extensible)
  • Interface table resolution for accurate port naming
  • Multiple domain suffix stripping for clean hostnames
  • CDP + LLDP fusion with deduplication
  • Real-time progress tracking in GUI
  • Color-coded log output with filtering

---

Screenshots

Discovery Interface

Topology Generation

Interactive Map Viewer

Built-in Editors

Architecture

High-Level Flow

┌─────────────────────────────────────────────────────────────────────────────┐
│                           SNMP Discovery Pipeline                            │
└─────────────────────────────────────────────────────────────────────────────┘

  ┌──────────────┐     ┌──────────────────┐     ┌───────────────────┐
  │  Seed CSV    │────▶│  ndp_recursive   │────▶│  Discovery JSON   │
  │  + Creds     │     │  (Phase 1)       │     │  per device       │
  └──────────────┘     └──────────────────┘     └─────────┬─────────┘
                              │                           │
                              │ Recursive                 │
                              │ Discovery                 ▼
                              │                  ┌───────────────────┐
                              ▼                  │  snmp_to_topology │
                       ┌─────────────┐           │  (Phase 2)        │
                       │   Queue     │           └─────────┬─────────┘
                       │  Depth 0..N │                     │
                       └─────────────┘                     ▼
                                                 ┌───────────────────┐
                                                 │  Topology JSON    │
                                                 │  (network.json)   │
                                                 └───────────────────┘

Discovery Process

                    ┌─────────────────────────────────────┐
                    │         Start Discovery              │
                    └─────────────────┬───────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────────┐
                    │    Load Seed Devices from CSV       │
                    └─────────────────┬───────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────────┐
                    │    Initialize Credential Manager    │
                    │    (Load from YAML)                 │
                    └─────────────────┬───────────────────┘
                                      │
                    ┌─────────────────▼───────────────────┐
             ┌──────│         For Each Depth Level        │◀─────┐
             │      └─────────────────┬───────────────────┘      │
             │                        │                          │
             │                        ▼                          │
             │      ┌─────────────────────────────────────┐      │
             │      │      For Each Device in Queue       │      │
             │      └─────────────────┬───────────────────┘      │
             │                        │                          │
             │                        ▼                          │
             │      ┌─────────────────────────────────────┐      │
             │      │  1. Test/Cache Credential           │      │
             │      │  2. Query Interface Table           │      │
             │      │  3. Query CDP Neighbors             │      │
             │      │  4. Query LLDP Neighbors            │      │
             │      │  5. Save device.json, cdp.json,     │      │
             │      │     lldp.json                       │      │
             │      └─────────────────┬───────────────────┘      │
             │                        │                          │
             │                        ▼                          │
             │      ┌─────────────────────────────────────┐      │
             │      │  Add Valid Neighbors to Next Queue  │──────┘
             │      │  (if depth < max_depth)             │
             │      └─────────────────────────────────────┘
             │
             │      ┌─────────────────────────────────────┐
             └─────▶│        All Depths Complete          │
                    └─────────────────┬───────────────────┘
                                      │
                                      ▼
                    ┌─────────────────────────────────────┐
                    │     Save discovery_summary.json     │
                    └─────────────────────────────────────┘

Credential Resolution Flow

┌─────────────────────────────────────────────────────────────────┐
│                  Credential Manager Flow                         │
└─────────────────────────────────────────────────────────────────┘

    Request Credential
    for Device IP
          │
          ▼
    ┌───────────────┐     Yes    ┌─────────────────┐
    │ In Cache?     │───────────▶│ Return Cached   │
    └───────┬───────┘            │ Credential      │
            │ No                 └─────────────────┘
            ▼
    ┌───────────────────────────────────────┐
    │ Test Credentials in Order:            │
    │   1. SNMPv3 credentials (if present)  │
    │   2. SNMPv2c credentials              │
    └───────────────────┬───────────────────┘
                        │
                        ▼
    ┌───────────────────────────────────────┐
    │ For each credential:                  │
    │   - Query sysName                     │
    │   - If success: cache + return        │
    │   - If fail: try next                 │
    └───────────────────┬───────────────────┘
                        │
            ┌───────────┴───────────┐
            ▼                       ▼
    ┌───────────────┐       ┌───────────────┐
    │ Found Working │       │ No Working    │
    │ Credential    │       │ Credential    │
    └───────┬───────┘       └───────┬───────┘
            │                       │
            ▼                       ▼
    ┌───────────────┐       ┌───────────────┐
    │ Add to Cache  │       │ Return Error  │
    │ Return Result │       └───────────────┘
    └───────────────┘

---

Components

1. ndp_discover.py - Discovery Engine

The core recursive SNMP discovery script.

Responsibilities:

• Load seed devices from CSV
  • Manage SNMP credential testing and caching
  • Query CDP and LLDP neighbor tables
  • Build interface index-to-name mappings
  • Recursive neighbor discovery up to max depth
  • Save per-device JSON results

Key Classes/Functions:

• SNMPCredentialManager - Unified v2c/v3 credential handling
  • discover_device() - Main device discovery routine
  • get_cdp_neighbors() - CDP neighbor extraction
  • get_lldp_neighbors() - LLDP neighbor extraction
  • get_interface_table() - Interface mapping (ifIndex → name)

2. snmp_to_topology_gui.py - Topology Generator

Converts discovery results into unified topology format.

Responsibilities:

• Load device summaries from discovery output
  • Build topology from LLDP data (primary)
  • Augment with CDP data (deduplicated)
  • Ensure bidirectional connections
  • Normalize interface names across vendors
  • Detect connected components
  • Annotate with location data (optional)

Output Format:

{
  "device-name": {
    "node_details": {
      "ip": "10.0.0.1",
      "platform": "Cisco IOSv IOS 15.9(3)M8",
      "location": "datacenter-1"
    },
    "peers": {
      "peer-name": {
        "ip": "10.0.0.2",
        "platform": "Arista vEOS-lab EOS 4.33.1F",
        "connections": [
          ["Gi0/1", "Eth1"],
          ["Gi0/2", "Eth2"]
        ]
      }
    }
  }
}

3. snmp_discovery_gui.py - PyQt6 GUI

Graphical interface for the discovery pipeline.

Features:

• Discovery Tab: Configure and run discovery
  • Topology Tab: Generate topology from results
  • Results Tab: Browse discovered devices

GUI Components:

• Real-time stats (total/success/failed/depth/queue)
  • Progress bar tracking depth completion
  • Queue table showing device status
  • Color-coded log viewer with filtering
  • Auto-scroll and verbose toggle

4. snmp_credentials.py - Credential Manager

Manages SNMP v2c and v3 credentials with caching.

Features:

• YAML configuration with environment variable substitution
  • Priority ordering (v3 preferred by default)
  • Per-device credential caching
  • Statistics tracking (hits/misses/attempts)

---

Installation

Prerequisites

# Python 3.10+
python --version

# Required packages
pip install pysnmp>=6.0.0
pip install PyQt6
pip install pyyaml

MIB Setup

The discovery requires compiled MIBs for CDP and LLDP:

project/
├── compiled_mibs/
│   ├── cisco/
│   │   └── CISCO-CDP-MIB.py
│   ├── IF-MIB.py
│   └── LLDP-MIB.py

MIBs can be compiled using mibdump.py from pysmi or obtained from vendor sources.

Directory Structure

snmp_maps/
├── ndp_recursive_6.py          # Discovery engine
├── snmp_to_topology_gui.py     # Topology generator
├── snmp_discovery_gui.py       # PyQt6 GUI
├── snmp_credentials.py         # Credential manager
├── credentials.yaml            # SNMP credentials
├── devices.csv                 # Seed devices
├── compiled_mibs/              # Compiled MIB files
│   ├── cisco/
│   └── ...
└── results/                    # Discovery output
    ├── device-1/
    │   ├── device.json
    │   ├── cdp.json
    │   └── lldp.json
    ├── device-2/
    └── discovery_summary.json

---

Configuration

Credentials File (credentials.yaml)

# SNMPv2c credentials
v2c:
  - name: readonly
    community: public
    
  - name: production
    community: ${SNMP_COMMUNITY}  # Environment variable

# SNMPv3 credentials  
v3:
  - name: admin-v3
    user: snmpadmin
    auth_protocol: SHA
    auth_password: ${SNMP_AUTH_PASS}
    priv_protocol: AES
    priv_password: ${SNMP_PRIV_PASS}
    
  - name: noauth-v3
    user: monitor
    security_level: noAuthNoPriv

Seed Devices CSV

Hostname,Vendor,Model,OS Version,Serial Number,Site,Location,Rack,Status,Tenant,IP Address
wan-core-1,cisco,7200,15.2(4)M11,FTX1234,DC1,Row-A,Rack-1,Active,,172.16.100.1
spine-1,arista,vEOS,4.33.1F,SN001,DC1,Row-B,Rack-2,Active,,172.16.100.10

Required columns: Hostname, Vendor
Optional columns: IP Address (fallback), Site (location annotation)

Domain Suffixes

Domain suffixes are stripped from hostnames for clean folder naming:

• Discovery format: lab.local,home.com (no leading dot)
  • Topology format: .lab.local,.home.com (leading dot)

The GUI automatically converts between formats.

---

Usage

GUI Mode

python snmp_discovery_gui.py

1. Discovery Tab:

   • Set seed devices CSV path
   • Set credentials YAML path
   • Set output directory
   • Enter domain suffixes (comma-separated)
   • Set max depth (0 = seed only)
   • Enable/disable verbose mode
   • Click "Start Discovery"

   2. Topology Tab:

      • Results directory auto-populated from discovery
      • Set output JSON filename
      • Set domain suffixes (with dots)
      • Click "Generate Topology"

   3. Results Tab:

      • Browse to results directory
      • Click "Load Results"
      • Click devices to view JSON details

Command Line Mode

Discovery:

python ndp_discover.py [-v] <devices.csv> <credentials.yaml> <output_dir> <domains> [max_depth]

# Example
python ndp_discover.py -v devices.csv credentials.yaml ./results 'lab.local,home.com' 5

Topology Generation:

python snmp_to_topology_gui.py <discovery_dir> [output_file] [domain_suffixes] [locations_csv]

# Example
python snmp_to_topology_gui.py ./results network.json '.lab.local,.home.com' devices.csv

---

Output Formats

Per-Device Discovery Output

device.json - Device metadata and interface table:

{
  "hostname": "usa-rtr-1",
  "fqdn": "usa-rtr-1.lab.local",
  "ip": "172.16.100.2",
  "vendor": "cisco",
  "credential": "v2c:readonly",
  "credential_version": "v2c",
  "depth": 1,
  "timestamp": "2024-12-03T20:00:00",
  "status": "success",
  "sysDescr": "Cisco IOS Software...",
  "interface_table": {
    "1": {"ifName": "Gi0/0", "ifDescr": "GigabitEthernet0/0", "ifAlias": "UPLINK"},
    "2": {"ifName": "Gi0/1", "ifDescr": "GigabitEthernet0/1", "ifAlias": ""}
  }
}

cdp.json - CDP neighbor data:

{
  "neighbor_count": 2,
  "neighbors": [
    {
      "index": "3.1",
      "local_port": "Gi0/0",
      "device_id": "wan-core-1",
      "platform": "Cisco 7206VXR",
      "remote_port": "Gi1/0",
      "ip_address": "172.16.100.1"
    }
  ]
}

lldp.json - LLDP neighbor data:

{
  "neighbor_count": 3,
  "neighbors": [
    {
      "index": "0.2.1",
      "local_port_index": "2",
      "system_name": "eng-spine-1",
      "system_description": "Arista Networks EOS version 4.33.1F...",
      "chassis_id": "50:01:00:01:00:00",
      "port_id": "Ethernet1",
      "port_description": "INT::usa-rtr-1::Gi0/1",
      "management_address": "172.16.200.1"
    }
  ]
}

Topology Output

network.json - Complete topology:

{
  "wan-core-1": {
    "node_details": {
      "ip": "172.16.100.1",
      "platform": "Cisco 7200 IOS 15.2(4)M11",
      "location": "DC1"
    },
    "peers": {
      "usa-rtr-1": {
        "ip": "172.16.100.2",
        "platform": "Cisco IOSv IOS 15.9(3)M8",
        "connections": [
          ["Et1/1", "Gi0/0"]
        ]
      }
    }
  }
}

---

GUI Reference

Log Message Prefixes

The topology generator uses parseable prefixes for GUI color-coding:

Prefix	Color	Description
[SECTION]	Blue (bold)	Section headers
[INFO]	Light gray	Informational messages
[DEVICE]	Green	Per-device processing
[STAT]	Cyan	Statistics
[WARN]	Orange (bold)	Warnings
[ERROR]	Red (bold)	Errors
[SUCCESS]	Light green	Success messages
[SUMMARY]	Purple (bold)	Summary lines

Discovery Log Prefixes

Prefix	Color	Description
[VERBOSE]	Gray	Debug messages (filterable)
[CRED]	Purple	Credential manager messages

---

Troubleshooting

Common Issues

1. No working credential found

ERROR: No working credential found (2 tested)

• Verify SNMP is enabled on device
  • Check community string / v3 credentials
  • Verify network connectivity (UDP 161)
  • Check ACLs on device

2. DNS resolution failed

WARNING: DNS resolution failed for device.lab.local
FALLBACK: Using fallback IP: 172.16.100.1

• Ensure DNS is configured or provide IP in CSV
  • The script will use CDP/LLDP management addresses as fallback

3. No LLDP neighbors

Found 0 valid LLDP neighbors

• Verify LLDP is enabled on device
  • Some devices (older Cisco) only support CDP
  • Check LLDP holdtime hasn't expired

4. Interface not resolved

[STAT] Skipped (no local port): 5

• Interface table may be incomplete
  • LLDP local port index doesn't match ifIndex
  • Enable verbose mode to debug interface mapping

5. Topology shows disconnected components

[WARN] 2 disconnected network segments

• Some devices may not have been discovered
  • Check if devices are unreachable
  • Verify max_depth is sufficient

Debug Mode

Enable verbose output for detailed debugging:

# CLI
python ndp_discover.py -v ...

# GUI
Check "Verbose Mode" checkbox

Verbose output includes:

• SNMP walk iterations and results
  • Credential testing details
  • Interface resolution attempts
  • Neighbor queue processing

---

Design Decisions

Why Cache-First Credential Lookup?

In large networks, the same credential often works for many devices. Testing credentials is expensive (SNMP timeouts). The cache-first approach:

• Tests each credential only once per IP
  • Eliminates redundant authentication on subsequent queries
  • Provides 50%+ cache hit rates in typical deployments

Why LLDP Primary, CDP Secondary?

LLDP is vendor-neutral and provides richer metadata (management addresses, system capabilities). CDP is Cisco-specific but still valuable:

• LLDP: Primary source, processed first
  • CDP: Augments LLDP, deduplicated to avoid duplicates

Why Bidirectional Topology?

Network connections are inherently bidirectional. Storing only one direction would require lookups in both directions. Bidirectional storage:

• Simplifies topology queries
  • Enables easy path finding
  • Matches how networks actually work

Why Interface Table Resolution?

LLDP provides local port as an ifIndex number, not a name. Without resolution:

• You'd see: local_port: "3"
  • With resolution: local_port: "Gi0/1"

The interface table query (IF-MIB) provides the mapping.

Why Multiple Domain Suffix Support?

Enterprise networks often span multiple DNS domains:

• device.corp.example.com
  • device.lab.example.com
  • device.dc1.example.com

Multiple suffix support ensures clean hostname extraction regardless of domain.

---

License

GPLv3 - see license file

Author

Scott Peterman - Principal Infrastructure Engineer