sfacts 2.3.0

Modern network discovery and fact collection toolkit with secure credential management and structured logging

Simple Facts - Network Discovery & Fact Collection Toolkit

Automated network inventory toolkit that syncs your actual network data to NetBox IPAM in minutes. Discover devices, collect facts, and eliminate manual documentation drift. Built on Netmiko, TextFSM, and PyNetBox. Recently updated with Prefect.

Features

• 🔍 Network Discovery: Automatic device discovery with SSH port scanning
• 🔧 Device Identification: Auto-detect device types using Netmiko SSHDetect
• 📊 Fact Collection: Collect structured device information using TextFSM
• 🏢 NetBox Integration: Complete IPAM/DCIM synchronization with VRF support
• 🔄 Prefect Workflows: Modern workflow orchestration with automated pipelines and beautiful UI
• ** Secure Credentials**: OS-native keyring storage with .env file support
• 🚀 High Performance: Parallel processing with configurable workers
• 📝 Structured Logging: Professional logging with Loguru
• 🌐 SSH Tunnel Support: Test mode for containerlab and remote environments
• 📁 Flexible Output: Custom output directories with --output-dir option
• 🧹 Code Quality: Ruff linting and formatting for clean, maintainable code
• ⚙️ CI/CD Pipeline: GitLab CI/CD with lint, test, security, and build stages
• 🏗️ Modern Architecture: Built with uv, Loguru, and industry standards

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Code Building Blocks                          │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                         sfacts/                                 │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │  sfacts/core/                                           │    │
│  │  • scanner.py      - Network discovery & port scanning  │    │
│  │  • collector.py    - Fact collection & TextFSM parsing  │    │
│  │  • netbox/         - NetBox sync modules               │    │
│  │    - base_objects.py  - Core object definitions        │    │
│  │    - bulk_sync.py     - Bulk sync operations            │    │
│  │    - device_sync.py   - Device synchronization          │    │
│  │    - interface_sync.py - Interface sync                 │    │
│  │    - ipam_sync.py      - IPAM/VLAN/VRF sync             │    │
│  │    - mac_sync.py       - MAC address sync               │    │
│  │    - vrf_sync.py       - VRF synchronization            │    │
│  │  • utils/         - Auth, logging, helpers              │    │
│  └─────────────────────────────────────────────────────────┘    │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │  sfacts/tasks/                                          │    │
│  │  • collect.py      - Fact collection task               │    │
│  │  • netbox.py       - NetBox sync task                   │    │
│  └─────────────────────────────────────────────────────────┘    │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │  sfacts/flows/                                           │    │
│  │  • modular_flows.py - Prefect flow definitions          │    │
│  └─────────────────────────────────────────────────────────┘    │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │  sfacts/cli/                                             │    │
│  │  • main.py         - CLI entry point                     │    │
│  └─────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                      Prefect Orchestration                      │
│  • serve.py        - Deploy Prefect flows                      │
│  • deployments.py  - Flow deployment configuration             │
│  • UI @ localhost:4200 for workflow management                 │
└─────────────────────────────────────────────────────────────────┘

Data Flow:
CLI → sfacts/core/ → sfacts/tasks/ → NetBox API
Prefect flows orchestrate tasks automatically

CLI Commands

Command	Purpose	Example
sfacts discover	Network device discovery	sfacts discover --subnet 192.168.1.0/24
sfacts facts	Enhanced fact collection	sfacts facts --input discovery_results.json
sfacts export	Export to Nornir inventory	sfacts export --output-dir inventory
sfacts netbox sync	Sync facts to NetBox IPAM/DCIM	sfacts netbox sync --site-name "Production"
sfacts netbox test	Test NetBox API connection	sfacts netbox test
sfacts netbox status	Show NetBox instance statistics	sfacts netbox status
sfacts netbox purge	Delete NetBox data (DESTRUCTIVE)	sfacts netbox purge --dry-run
sfacts netbox validate	Validate facts for NetBox sync	sfacts netbox validate

Discovery Modes

• Subnet Discovery: Scan network ranges for SSH-enabled devices
• SSH Tunnel Mode: Connect through SSH tunnels for testing (containerlab, etc.)

Installation

This project uses uv for fast Python package management. The project requires Python 3.8 or higher (Python 3.9+ recommended for Prefect workflow support).

Install uv (if not already installed)

curl -LsSf https://astral.sh/uv/install.sh | sh

Install dependencies

uv sync

Alternative: Using pip (legacy)

pip install -r requirements.txt

Quick Start

1. Install Dependencies

uv sync

2. Set up credentials (choose one method):

Option A: Environment Variables

export NETOPS_USERNAME=your_username
export NETOPS_PASSWORD=your_password

Option B: Secure Keyring Storage

# Store credentials securely in OS keyring
uv run python -c "
from sfacts.utils.auth import CredentialManager
cm = CredentialManager()
cm.store_credentials('your_username', 'your_password')
"

3. Configure NetBox Integration (Optional)

Create a .env file for NetBox integration:

# Copy template and configure
cp .env.template .env

# Edit .env with your NetBox details
NETBOX_URL=https://your-netbox-instance.com
NETBOX_TOKEN=your-api-token-here

4. Discover Network Devices

Subnet Discovery

# Discover devices in a network range
uv run sfacts discover --subnet 192.168.1.0/24

# Save results to specific directory
uv run sfacts discover --subnet 192.168.1.0/24 --output-dir ./network_scans

# High-performance scanning with custom output
uv run sfacts discover --subnet 10.0.0.0/16 --workers 50 --timeout 15 --output-dir ./results

SSH Tunnel Mode (for testing/containerlab)

# Connect through SSH tunnels
uv run sfacts discover --tunnel-host localhost --tunnel-ports 2101-2116

5. Collect Device Facts

# Collect facts from discovered devices
uv run sfacts facts --input disco_results_*.json

# Save facts to specific directory
uv run sfacts facts --input disco_results.json --output-dir ./network_facts

# Direct subnet fact collection with custom output
uv run sfacts facts --subnet 192.168.1.0/24 --workers 50 --output-dir ./results

6. NetBox Integration (Optional)

Test NetBox Connection

# Test API connectivity
uv run sfacts netbox test

# Check NetBox instance status and data statistics
uv run sfacts netbox status

Synchronize to NetBox IPAM/DCIM

# Dry-run to preview changes
uv run sfacts netbox sync --site-name "Production" --dry-run

# Sync device facts to NetBox
uv run sfacts netbox sync --site-name "Production"

# Validate facts data before sync
uv run sfacts netbox validate

NetBox Data Management

# Preview what would be deleted (SAFE)
uv run sfacts netbox purge --dry-run

# Delete all NetBox data (DESTRUCTIVE - requires confirmation)
uv run sfacts netbox purge

# Delete specific site data only
uv run sfacts netbox purge --site-name "Lab Environment" --dry-run

7. Prefect Workflow Orchestration (Optional)

Install Prefect workflows

uv sync --extra workflows

Start Prefect server and deployments

# Terminal 1: Start Prefect server
prefect server start

# Terminal 2: Start deployments
uv run python serve.py

Access Prefect UI

• Open http://localhost:4200
• Run deployments from the UI

Six deployments available:

1. discover-network - Find devices (auto-detects subnet/tunnel)
2. collect-facts - Gather facts (auto-fetches from latest discovery)
3. sync-to-netbox - Push to NetBox (auto-fetches from latest facts)
4. test-netbox - Test NetBox API connection
5. netbox-status - Show NetBox instance statistics
6. purge-netbox - Purge NetBox data (destructive)

Example workflow:

# Run from Prefect UI (http://localhost:4200):
# 1. discover-network with target: "192.168.121.0/24"
# 2. collect-facts (automatically uses latest discovery)
# 3. sync-to-netbox (automatically uses latest facts)

Benefits:

• ✅ Zero manual JSON copying between steps
• ✅ Beautiful UI with artifact visualization
• ✅ Flexible: run steps independently or skip discovery
• ✅ Data persistence via Prefect artifacts

See Prefect Modular Workflows for complete documentation.

Documentation

• Usage Guide - Detailed usage instructions and examples
• NetBox Integration - Complete NetBox IPAM/DCIM integration guide
• NetBox Dependency Map - NetBox sync system architecture and data flow
• Prefect Integration - Legacy Prefect flows documentation
• Prefect Modular Workflows - New modular deployment architecture
• CI/CD Pipeline - GitLab CI/CD pipeline setup and configuration
• Inventory Structure - Complete inventory configuration reference
• Troubleshooting - Common issues and solutions

Output Structure

output/
├── disco_results_YYYYMMDD_HHMMSS.json        # Network discovery results
├── facts_results_YYYYMMDD_HHMMSS.json        # Device facts with TextFSM parsing
└── report/
    └── network_report_YYYYMMDD_HHMMSS.md     # Comprehensive network report

Environment Variables

The following environment variables are supported:

Device Authentication:

• NETOPS_USERNAME - SSH username for device authentication
• NETOPS_PASSWORD - SSH password for device authentication
• NETOPS_SECRET - Enable secret for privileged access (optional)

NetBox Integration:

• NETBOX_URL - NetBox instance URL (e.g., https://netbox.company.com)
• NETBOX_TOKEN - NetBox API token for authentication
• NETBOX_VERIFY_SSL - SSL certificate verification (default: true)

Development

Code Quality Tools

This project uses modern Python development tools:

# Install development dependencies
uv sync --dev

# Run linting and formatting
uv run ruff check sfacts/ tests/
uv run ruff check --fix sfacts/ tests/
uv run ruff format sfacts/ tests/

# Type checking
uv run mypy sfacts/

# Security scanning
uv run bandit -r sfacts/

# Run tests with coverage
uv run pytest

CI/CD Pipeline

The project includes a GitLab CI/CD pipeline (.gitlab-ci.yml) with four stages:

1. Lint - Ruff linting, formatting checks, MyPy type checking
2. Test - Unit tests with coverage reporting
3. Security - Dependency vulnerability scanning, Bandit security analysis, code quality metrics
4. Build - Python package creation (main branch and tags)

See CI/CD Pipeline for full setup details.

Key Technologies

• uv - Fast Python package management
• Prefect - Modern workflow orchestration and scheduling
• Loguru - Structured logging with colors and context
• Keyring - Secure OS-native credential storage
• python-dotenv - Environment variable management from .env files
• Netmiko - Multi-vendor SSH device connections
• TextFSM - Structured parsing with ntc-templates
• pynetbox - NetBox API client for IPAM/DCIM integration
• Rich - Beautiful progress bars and terminal output

Additional Features

Nornir Inventory Export

Export collected facts to Nornir inventory format for integration with Nornir-based automation:

# Export latest facts to Nornir inventory
uv run sfacts export

# Export specific facts file
uv run sfacts export --input-file facts_results_20250124.json

# Include credentials in defaults.yaml
uv run sfacts export --include-credentials

See inventory/README.md for details on using exported inventory files.

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.