Data importer from NetBox 2.10.x to Nautobot
Project description
Nautobot NetBox Importer
A plugin for Nautobot.
Installation
The plugin is available as a Python package in PyPI and can be installed with pip:
pip install nautobot-netbox-importer
The plugin is compatible with Nautobot 1.0.0b3 and later and can handle JSON data exported from NetBox 2.10.3 thru 2.10.8 at present.
Once installed, the plugin needs to be enabled in your nautobot_config.py
:
PLUGINS = ["nautobot_netbox_importer"]
Usage
Getting a data export from NetBox
From the NetBox root directory, run the following command to produce a JSON file (here, /tmp/netbox_data.json
) describing the contents of your NetBox database:
python netbox/manage.py dumpdata \
--traceback --format=json \
--exclude admin.logentry --exclude sessions.session \
--exclude extras.ObjectChange --exclude extras.Script --exclude extras.Report \
> /tmp/netbox_data.json
Importing the data into Nautobot
From within the Nautobot application environment, run nautobot-server import_netbox_json <json_file> <netbox_version>
, for example nautobot-server import_netbox_json /tmp/netbox_data.json 2.10.3
.
Importing change logging (ObjectChange) records
Because the database change log can be a massive amount of data, and often this historical information does not need to be imported, ObjectChange
records are not included in the database export command above and are not handled by the import_netbox_json
command. To import ObjectChange
records specifically, after the previous Netbox import process has succeeded, you can do the following.
Getting a data export from NetBox with ONLY ObjectChange items
python netbox/manage.py dumpdata extras.ObjectChange\
--traceback --format=json \
> /tmp/netbox_only_objectchange.json
Importing the ObjectChanges into Nautobot
From within the Nautobot application environment, run nautobot-server import_netbox_objectchange_json <json_file_without_objectchanges> <json_file_only_objectchanges> <netbox_version>
, for example nautobot-server import_netbox_objectchange_json imp/script/import_netbox_json.json imp/script/netbox_only_objectchange.json 2.10.3
.
Contributing
Most of the internal logic of this plugin is based on the DiffSync library, which in turn is built atop Pydantic. A basic understanding of these two libraries will be helpful to those wishing to contribute to this project.
Pull requests are welcomed and automatically built and tested against multiple version of Python and multiple versions of Nautobot through TravisCI.
The project is packaged with a light development environment based on docker-compose
to help with the local development of the project and to run the tests within TravisCI.
The project is following Network to Code software development guideline and is leveraging:
- Black, Pylint, Bandit and pydocstyle for Python linting and formatting.
- Django unit test to ensure the plugin is working properly.
- Poetry for packaging and dependency management.
Development Environment
The development environment can be used in 2 ways. First, with a local poetry environment if you wish to develop outside of Docker. Second, inside of a docker container.
Invoke tasks
The PyInvoke library is used to provide some helper commands based on the environment. There are a few configuration parameters which can be passed to PyInvoke to override the default configuration:
nautobot_ver
: the version of Nautobot to use as a base for any built docker containers (default: develop-latest)project_name
: the default docker compose project name (default: nautobot-netbox-importer)python_ver
: the version of Python to use as a base for any built docker containers (default: 3.6)local
: a boolean flag indicating if invoke tasks should be run on the host or inside the docker containers (default: False, commands will be run in docker containers)compose_dir
: the full path to a directory containing the project compose filescompose_files
: a list of compose files applied in order (see Multiple Compose files for more information)
Using PyInvoke these configuration options can be overridden using several methods. Perhaps the simplest is simply setting an environment variable INVOKE_NAUTOBOT-NETBOX-IMPORTER_VARIABLE_NAME
where VARIABLE_NAME
is the variable you are trying to override. The only exception is compose_files
, because it is a list it must be overridden in a yaml file. There is an example invoke.yml
in this directory which can be used as a starting point.
Local Poetry Development Environment
- Copy
development/creds.example.env
todevelopment/creds.env
(This file will be ignored by git and docker) - Uncomment the
POSTGRES_HOST
,REDIS_HOST
, andNAUTOBOT_ROOT
variables indevelopment/creds.env
- Create an invoke.yml with the following contents at the root of the repo:
---
nautobot_netbox_importer:
local: true
compose_files:
- "docker-compose.requirements.yml"
- Run the following commands:
poetry shell
poetry install
pip install nautobot
export $(cat development/dev.env | xargs)
export $(cat development/creds.env | xargs)
- You can now run nautobot-server commands as you would from the Nautobot documentation for example to start the development server:
nautobot-server runserver 0.0.0.0:8080 --insecure
Nautobot server can now be accessed at http://localhost:8080.
Docker Development Environment
This project is managed by Python Poetry and has a few requirements to setup your development environment:
- Install Poetry, see the Poetry Documentation for your operating system.
- Install Docker, see the Docker documentation for your operating system.
Once you have Poetry and Docker installed you can run the following commands to install all other development dependencies in an isolated python virtual environment:
poetry shell
poetry install
invoke start
Nautobot server can now be accessed at http://localhost:8080.
CLI Helper Commands
The project includes a CLI helper based on invoke to help setup the development environment. The commands are listed below in 3 categories dev environment
, utility
and testing
.
Each command can be executed with invoke <command>
. Environment variables INVOKE_NAUTOBOT_NETBOX_IMPORTER_PYTHON_VER
and INVOKE_NAUTOBOT_NETBOX_IMPORTER_NAUTOBOT_VER
may be specified to override the default versions. Each command also has its own help invoke <command> --help
Docker dev environment
build Build Nautobot docker image.
debug Start Nautobot and its dependencies in debug mode.
destroy Destroy all containers and volumes.
restart Gracefully restart all containers.
start Start Nautobot and its dependencies in detached mode.
stop Stop Nautobot and its dependencies.
Utility
check-migrations Check for missing migrations.
cli Launch a bash shell inside the running Nautobot container.
createsuperuser Create a new Nautobot superuser account (default: "admin"), will prompt for password.
generate-packages Generate all Python packages inside docker and copy the file locally under dist/.
makemigrations Perform makemigrations operation in Django.
migrate Perform migrate operation in Django.
nbshell Launch an interactive nbshell session.
post-upgrade Performs Nautobot common post-upgrade operations using a single entrypoint.
vscode Launch Visual Studio Code with the appropriate Environment variables to run in a container.
Testing
bandit Run bandit to validate basic static code security analysis.
black Check Python code style with Black.
flake8 Check for PEP8 compliance and other style issues.
hadolint Check Dockerfile for hadolint compliance and other style issues.
pydocstyle Run pydocstyle to validate docstring formatting adheres to standards.
pylint Run pylint code analysis.
tests Run all tests for this plugin.
unittest Run Django unit tests for the plugin.
unittest-coverage Report on code test coverage as measured by 'invoke unittest'.
Questions
For any questions or comments, please check the FAQ first and feel free to swing by the #nautobot slack channel. Sign up here
Screenshots
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for nautobot-netbox-importer-1.3.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0227e74a53179b57e42a3f2bdd95a24843f33abeb0a92ab389cab7f9417f19fa |
|
MD5 | 334afd6ac6d617236586ee166e243734 |
|
BLAKE2b-256 | 5dbb27ab57a14e0b997507521efb6e3b19f30cfaa46ef6b29819386404623f9d |
Hashes for nautobot_netbox_importer-1.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c92fa2c6910bba92685cc8c16f097e7b79340534ce9bb0faca8bbca2fee78a02 |
|
MD5 | 5d4f93cea94aeeb44a29b999035e2538 |
|
BLAKE2b-256 | 18ffbe8765730575f88067ce3fbce32a3d34fa1ab54eafb1fc8f01b4123b17d0 |