Skip to main content
Join the official 2020 Python Developers SurveyStart the survey!

Configuration manager for embedded devices, implemented as a reusable django-app

Project description

https://travis-ci.org/openwisp/django-netjsonconfig.svg https://coveralls.io/repos/openwisp/django-netjsonconfig/badge.svg Requirements Status https://badge.fury.io/py/django-netjsonconfig.svg

Configuration manager for embedded devices, implemented as a reusable django-app.

Based on the NetJSON format and the netjsonconfig library.

adhoc interface
preview

Current features

  • configuration management for embedded devices supporting different firmwares:
  • configuration editor based on JSON-Schema editor
  • advanced edit mode: edit NetJSON DeviceConfiguration objects for maximum flexibility
  • configuration templates: reduce repetition to the minimum
  • configuration variables: reference ansible-like variables in the configuration and templates
  • template tags: tag templates to automate different types of auto-configurations (eg: mesh, WDS, 4G)
  • simple HTTP resources: allow devices to automatically download configuration updates
  • VPN management: easily create VPN servers and clients

Project goals

  • automate configuration management for embedded devices
  • allow to minimize repetition by using templates
  • provide base logic that can be extended by third-party apps (see Extending django-netjsonconfig)
  • provide ways to support more firmwares by adding custom backends
  • keep the core as simple as possible

Deploy it in production

An automated installer is available at ansible-openwisp2.

Dependencies

  • Python >=3.6
  • OpenSSL

Install stable version from pypi

Install from pypi:

pip install django-netjsonconfig

Install development version

Install tarball:

pip install https://github.com/openwisp/django-netjsonconfig/tarball/master

Alternatively you can install via pip using git:

pip install -e git+git://github.com/openwisp/django-netjsonconfig#egg=django-netjsonconfig

If you want to contribute, install your cloned fork:

git clone git@github.com:<your_fork>/django-netjsonconfig.git
cd django-netjsonconfig
python setup.py develop

Setup (integrate in an existing django project)

Add django_netjsonconfig, django.contrib.admin, sortedm2m and reversion to INSTALLED_APPS in the following order:

INSTALLED_APPS = [
    # other apps
    'openwisp_utils.admin_theme',
    'django_netjsonconfig',
    # ensure the django admin comes after django-netjsonconfig
    'django.contrib.admin',
    'sortedm2m',
    'reversion'  # optional, can be removed if not needed
    # ...
]

Add the controller URLs to your main urls.py:

urlpatterns = [
    # ... other urls in your project ...

    # controller URLs
    # used by devices to download/update their configuration
    # keep the namespace argument unchanged
    url(r'^', include('django_netjsonconfig.controller.urls', namespace='controller')),
    # common URLs
    # shared among django-netjsonconfig components
    # keep the namespace argument unchanged
    url(r'^', include('django_netjsonconfig.urls', namespace='netjsonconfig')),
]

Then run:

./manage.py migrate

Installing for development

Install sqlite:

sudo apt-get install sqlite3 libsqlite3-dev openssl libssl-dev

Install your forked repo:

git clone git://github.com/<your_fork>/django-netjsonconfig
cd django-netjsonconfig/
python setup.py develop

Install test requirements:

pip install -r requirements-test.txt

Create database:

cd tests/
./manage.py migrate
./manage.py createsuperuser

Launch development server:

./manage.py runserver

You can access the admin interface at http://127.0.0.1:8000/admin/.

Run tests with:

./runtests.py

How to use configuration variables

Sometimes the configuration is not exactly equal on all the devices, some parameters are unique to each device or need to be changed by the user.

In these cases it is possible to use configuration variables in conjunction with templates, this feature is also known as configuration context, think of it like a dictionary which is passed to the function which renders the configuration, so that it can fill variables according to the passed context.

The different ways in which variables are defined are described below.

Predefined device variables

Each device gets the following attributes passed as configuration variables:

  • id
  • key
  • name
  • mac_address

User defined device variables

In the device configuration section, you can access the context field by clicking on “Advanced Options (show)”.

advanced options (show)

Then you can define the variables as a key, value dictionary (JSON formatted) as shown below.

context

Template default values

It’s possible to specify the default values of variables defined in a template.

This allows to achieve 2 goals:

  1. pass schema validation without errors (otherwise it would not be possible to save the template in the first place)
  2. provide good default values that are valid in most cases but can be overridden in the device if needed

These default values will be overridden by the User defined device variables.

To do this, click on “Advanced Options (show)” in the edit template page:

advanced options (show)

Then you can define the default values of the variables:

default values

Global variables

Variables can also be defined globally using the NETJSONCONFIG_CONTEXT setting.

Example usage of variables

Here’s a typical use case, the WiFi SSID and WiFi password. You don’t want to define this for every device, but you may want to allow operators to easily change the SSID or WiFi password for a specific device without having to re-define the whole wifi interface to avoid duplicating information.

This would be the template:

{
    "interfaces": [
        {
            "type": "wireless",
            "name": "wlan0",
            "wireless": {
                "mode": "access_point",
                "radio": "radio0",
                "ssid": "{{wlan0_ssid}}",
                "encryption": {
                    "protocol": "wpa2_personal",
                    "key": "{{wlan0_password}}",
                    "cipher": "auto"
                }
            }
        }
    ]
}

These would be the default values in the template:

{
    "wlan0_ssid": "SnakeOil PublicWiFi",
    "wlan0_password": "Snakeoil_pwd!321654"
}

The default values can then be overridden at device level if needed, eg:

{
    "wlan0_ssid": "Room 23 ACME Hotel",
    "wlan0_password": "room_23pwd!321654"
}

Signals

config_modified

Path: django_netjsonconfig.signals.config_modified

Arguments:

  • instance: instance of Config which got its config modified

This signal is emitted every time the configuration of a device is modified.

It does not matter if Config.status is already modified, this signal will be emitted anyway because it signals that the device configuration has changed.

It is not triggered when the device is created for the first time.

This signal is used to trigger the update of the configuration on devices, when the push feature is enabled (requires Device credentials).

config_status_changed

Path: django_netjsonconfig.signals.config_status_changed

Arguments:

  • instance: instance of Config which got its status changed

This signal is emitted only when the configuration status of a device has changed.

checksum_requested

Path: django_netjsonconfig.signals.checksum_requested

Arguments:

  • instance: instance of Device for which its configuration checksum has been requested
  • request: the HTTP request object

This signal is emitted when a device requests a checksum via the controller views.

The signal is emitted just before a successful response is returned, it is not sent if the response was not successful.

config_download_requested

Path: django_netjsonconfig.signals.config_download_requested

Arguments:

  • instance: instance of Device for which its configuration has been requested for download
  • request: the HTTP request object

This signal is emitted when a device requests to download its configuration via the controller views.

The signal is emitted just before a successful response is returned, it is not sent if the response was not successful.

Settings

NETJSONCONFIG_BACKENDS

type: tuple
default:
(
  ('netjsonconfig.OpenWrt', 'OpenWRT'),
  ('netjsonconfig.OpenWisp', 'OpenWISP'),
)

Available configuration backends. For more information, see netjsonconfig backends.

NETJSONCONFIG_VPN_BACKENDS

type: tuple
default:
(
  ('django_netjsonconfig.vpn_backends.OpenVpn', 'OpenVPN'),
)

Available VPN backends for VPN Server objects. For more information, see OpenVPN netjsonconfig backend.

A VPN backend must follow some basic rules in order to be compatible with django-netjsonconfig:

  • it MUST allow at minimum and at maximum one VPN instance
  • the main NetJSON property MUST match the lowercase version of the class name, eg: when using the OpenVpn backend, the system will look into config['openvpn']
  • it SHOULD focus on the server capabilities of the VPN software being used

NETJSONCONFIG_DEFAULT_BACKEND

type: str
default: NETJSONCONFIG_BACKENDS[0][0]

The preferred backend that will be used as initial value when adding new Config or Template objects in the admin.

This setting defaults to the raw value of the first item in the NETJSONCONFIG_BACKENDS setting, which is netjsonconfig.OpenWrt.

Setting it to None will force the user to choose explicitly.

NETJSONCONFIG_DEFAULT_VPN_BACKEND

type: str
default: NETJSONCONFIG_VPN_BACKENDS[0][0]

The preferred backend that will be used as initial value when adding new Vpn objects in the admin.

This setting defaults to the raw value of the first item in the NETJSONCONFIG_VPN_BACKENDS setting, which is django_netjsonconfig.vpn_backends.OpenVpn.

Setting it to None will force the user to choose explicitly.

NETJSONCONFIG_REGISTRATION_ENABLED

type: bool
default: True

Whether devices can automatically register through the controller or not.

This feature is enabled by default.

Autoregistration must be supported on the devices in order to work, see openwisp-config automatic registration for more information.

NETJSONCONFIG_CONSISTENT_REGISTRATION

type: bool
default: True

Whether devices that are already registered are recognized when reflashed or reset, hence keeping the existing configuration without creating a new one.

This feature is enabled by default.

Autoregistration must be enabled also on the devices in order to work, see openwisp-config consistent key generation for more information.

NETJSONCONFIG_REGISTRATION_SELF_CREATION

type: bool
default: True

Whether devices that are not already present in the system are allowed to register or not.

Turn this off if you still want to use auto-registration to avoid having to manually set the device UUID and key in its configuration file but also want to avoid indiscriminate registration of new devices without explicit permission.

NETJSONCONFIG_SHARED_SECRET

type: str
default: ""

A secret key which must be used by devices to perform automatic registration.

This key MUST be explicitly set in production (if settings.DEBUG is False), otherwise an ImproperlyConfigured exception will be raised on startup.

NETJSONCONFIG_CONTEXT

type: dict
default: {}

Additional context that is passed to the default context of each device object.

NETJSONCONFIG_CONTEXT can be used to define system-wide configuration variables.

For technical information about how variables are handled in the lower levels of OpenWISP, see netjsonconfig context: configuration variables.

NETJSONCONFIG_DEFAULT_AUTO_CERT

type: bool
default: True

The default value of the auto_cert field for new Template objects.

The auto_cert field is valid only for templates which have type set to VPN and indicates whether a new x509 certificate should be created automatically for each configuration using that template.

The automatically created certificates will also be removed when they are not needed anymore (eg: when the VPN template is removed from a configuration object).

NETJSONCONFIG_CERT_PATH

type: str
default: /etc/x509

The filesystem path where x509 certificate will be installed when downloaded on routers when auto_cert is being used (enabled by default).

NETJSONCONFIG_COMMON_NAME_FORMAT

type: str
default: {mac_address}-{name}

Defines the format of the common_name attribute of VPN client certificates that are automatically created when using VPN templates which have auto_cert set to True.

NETJSONCONFIG_MANAGEMENT_IP_DEVICE_LIST

type: bool
default: True

In the device list page, the column IP will show the management_ip if available, defaulting to last_ip otherwise.

If this setting is set to False the management_ip won’t be shown in the device list page even if present, it will be shown only in the device detail page.

You may set this to False if for some reason the majority of your user doesn’t care about the management ip address.

NETJSONCONFIG_BACKEND_DEVICE_LIST

type: bool
default: True

In the device list page, the column backend and the backend filter are shown by default.

If this setting is set to False these items will be removed from the UI.

You may set this to False if you are using only one configuration backend and having this UI element doesn’t add any value to your users.

NETJSONCONFIG_HARDWARE_ID_ENABLED

type: bool
default: False

The field hardware_id can be used to store a unique hardware id, for example a serial number.

If this setting is set to True then this field will be shown first in the device list page and in the add/edit device page.

This feature is disabled by default.

NETJSONCONFIG_HARDWARE_ID_OPTIONS

type: dict
default:
{
    'blank': not NETJSONCONFIG_HARDWARE_ID_ENABLED,
    'null': True,
    'max_length': 32,
    'unique': True,
    'verbose_name': _('Serial number'),
    'help_text': _('Serial number of this device')
}

Options for the model field hardware_id.

  • blank: wether the field is allowed to be blank
  • null: wether an empty value will be stored as NULL in the database
  • max_length: maximum length of the field
  • unique: wether the value of the field must be unique
  • verbose_name: text for the human readable label of the field
  • help_text: help text to be displayed with the field

NETJSONCONFIG_HARDWARE_ID_AS_NAME

type: bool
default: True

When the hardware ID feature is enabled, devices will be referenced with their hardware ID instead of their name.

If you still want to reference devices by their name, set this to False.

Extending django-netjsonconfig

django-netjsonconfig provides a set of models, admin classes and generic views which can be imported, extended and reused by third party apps.

To extend django-netjsonconfig, you MUST NOT add it to settings.INSTALLED_APPS, but you must create your own app (which goes into settings.INSTALLED_APPS), import the base classes from django-netjsonconfig and add your customizations.

In order to help django find the static files and templates of django-netjsonconfig, you need to perform the steps described below.

1. Add EXTENDED_APPS

Add the following to your settings.py:

EXTENDED_APPS = ('django_netjsonconfig', 'django_x509',)

2. Add openwisp_utils.staticfiles.DependencyFinder

Add openwisp_utils.staticfiles.DependencyFinder to STATICFILES_FINDERS in your settings.py:

STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'openwisp_utils.staticfiles.DependencyFinder',
]

3. Add openwisp_utils.loaders.DependencyLoader

Add openwisp_utils.loaders.DependencyLoader to TEMPLATES in your settings.py:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'OPTIONS': {
            'loaders': [
                'django.template.loaders.filesystem.Loader',
                'django.template.loaders.app_directories.Loader',
                'openwisp_utils.loaders.DependencyLoader',
            ],
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    }
]

Extending models

This example provides an example of how to extend the base models of django-netjsonconfig by adding a relation to another django model named Organization.

# models.py of your custom ``config`` app
from django.db import models
from sortedm2m.fields import SortedManyToManyField
from taggit.managers import TaggableManager

from django_netjsonconfig.base.config import AbstractConfig, TemplatesVpnMixin
from django_netjsonconfig.base.tag import AbstractTaggedTemplate, AbstractTemplateTag
from django_netjsonconfig.base.template import AbstractTemplate
from django_netjsonconfig.base.vpn import AbstractVpn, AbstractVpnClient

# the model ``organizations.Organization`` is omitted for brevity
# if you are curious to see a real implementation, check out django-organizations
# https://github.com/bennylope/django-organizations

class OrganizationMixin(models.Model):
    organization = models.ForeignKey('organizations.Organization')

    class Meta:
        abstract = True


class Config(OrganizationMixin, TemplatesVpnMixin, AbstractConfig):
    templates = SortedManyToManyField('config.Template',
                                      related_name='config_relations',
                                      blank=True)
    vpn = models.ManyToManyField('config.Vpn',
                                 through='config.VpnClient',
                                 related_name='vpn_relations',
                                 blank=True)

    def clean(self):
        # your own validation logic here...
        pass

    class Meta(AbstractConfig.Meta):
        abstract = False


class TemplateTag(AbstractTemplateTag):
    class Meta(AbstractTemplateTag.Meta):
        abstract = False


class TaggedTemplate(AbstractTaggedTemplate):
    tag = models.ForeignKey('config.TemplateTag',
                            related_name='%(app_label)s_%(class)s_items',
                            on_delete=models.CASCADE)

    class Meta(AbstractTaggedTemplate.Meta):
        abstract = False


class Template(OrganizationMixin, AbstractTemplate):
    tags = TaggableManager(through='config.TaggedTemplate', blank=True)
    vpn = models.ForeignKey('config.Vpn', blank=True, null=True)

    def clean(self):
        # your own validation logic here...
        pass

    class Meta(AbstractTemplate.Meta):
        abstract = False


class Vpn(OrganizationMixin, AbstractVpn):
    class Meta(AbstractVpn.Meta):
        abstract = False


class VpnClient(AbstractVpnClient):
    config = models.ForeignKey('config.Config', on_delete=models.CASCADE)
    vpn = models.ForeignKey('config.Vpn', on_delete=models.CASCADE)
    cert = models.OneToOneField('django_x509.Cert',
                                on_delete=models.CASCADE,
                                blank=True,
                                null=True)

    class Meta(AbstractVpnClient.Meta):
        abstract = False

Extending the admin

Following the previous Organization example, you can avoid duplicating the admin code by importing the base admin classes and registering your models with.

# admin.py of your app
# these are your custom models, they must be imported before the abstract admin classes
from .models import Config, Template, Vpn

from django.contrib import admin
from django_netjsonconfig.base.admin import (AbstractConfigAdmin,
                                             AbstractConfigForm,
                                             AbstractTemplateAdmin,
                                             AbstractVpnAdmin,
                                             AbstractVpnForm,
                                             BaseForm)


class ConfigForm(AbstractConfigForm):
    class Meta(AbstractConfigForm.Meta):
        model = Config


class ConfigAdmin(AbstractConfigAdmin):
    form = ConfigForm


class TemplateForm(BaseForm):
    class Meta(BaseForm.Meta):
        model = Template


class TemplateAdmin(AbstractTemplateAdmin):
    form = TemplateForm


class VpnForm(AbstractVpnForm):
    class Meta(AbstractVpnForm.Meta):
        model = Vpn


class VpnAdmin(AbstractVpnAdmin):
    form = VpnForm


admin.site.register(Config, ConfigAdmin)
admin.site.register(Template, TemplateAdmin)
admin.site.register(Vpn, VpnAdmin)

Extending controller views

If your use case doesn’t vary a lot from the base one, you may also want to try to reuse the controller views:

# your_config_app.controller.views
from ..models import Device, Vpn
from django_netjsonconfig.controller.generics import (BaseDeviceChecksumView, BaseDeviceDownloadConfigView,
                                                      BaseDeviceRegisterView, BaseDeviceReportStatusView,
                                                      BaseVpnChecksumView, BaseVpnDownloadConfigView)

class DeviceChecksumView(BaseDeviceChecksumView):
    model = Device


class DeviceDownloadConfigView(BaseDeviceDownloadConfigView):
    model = Device


class DeviceReportStatusView(BaseDeviceReportStatusView):
    model = Device


class DeviceRegisterView(BaseDeviceRegisterView):
    model = Device


class VpnChecksumView(BaseVpnChecksumView):
    model = Vpn


class VpnDownloadConfigView(BaseVpnDownloadConfigView):
    model = Vpn


device_checksum = DeviceChecksumView.as_view()
device_download_config = DeviceDownloadConfigView.as_view()
device_report_status = DeviceReportStatusView.as_view()
device_register = DeviceRegisterView.as_view()
vpn_checksum = VpnChecksumView.as_view()
vpn_download_config = VpnDownloadConfigView.as_view()

Controller URLs

If you are not making drastic changes to the controller views, you can avoid duplicating the URL logic by using the get_controller_urls function. Put this in your controller urls.py:

# your_config_app.controller.urls
from django_netjsonconfig.utils import get_controller_urls
from . import views

urlpatterns = get_controller_urls(views)

Extending AppConfig

You may want to reuse the AppConfig class of django-netjsonconfig too:

from django_netjsonconfig.apps import DjangoNetjsonconfigApp


class MyOwnConfig(DjangoNetjsonconfigApp):
    name = 'yourapp.config'
    label = 'config'

    def __setmodels__(self):
        from .models import Config, VpnClient  # these are your custom models
        self.config_model = Config
        self.vpnclient_model = VpnClient

Real world extensions of django-netjsonconfig

For full working examples of django proejcts which extend django-netjsonconfig, see:

Screenshots

configuration item
bridge
radio
wpa enterprise
preview
adhoc interface

Contributing

  1. Announce your intentions in the OpenWISP Mailing List
  2. Fork this repo and install it
  3. Follow PEP8, Style Guide for Python Code
  4. Write code
  5. Write tests for your code
  6. Ensure all tests pass
  7. Ensure test coverage does not decrease
  8. Document your changes
  9. Send pull request

Changelog

See CHANGES.

License

See LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for django-netjsonconfig, version 0.12
Filename, size File type Python version Upload date Hashes
Filename, size django_netjsonconfig-0.12-py2.py3-none-any.whl (412.8 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size django-netjsonconfig-0.12.tar.gz (390.5 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page