Identifiable information scrambler for Nokia SR OS (id-scrambler)

Overview

The Nokia id-scrambler application scrambles cleartext files containing identifiable information, such as the output from the admin show configuration (admin display-config in classic CLI) or the admin tech-support state (admin tech-support output-level state in classic CLI) commands. This allows an administrator to review and edit the scrambled cleartext output before sending it to Nokia technical support, ensuring all identifiable data has been scrambled.

The following identifiable information is scrambled:

• quoted strings that may contain descriptions or names of items such as:
  • BGP communities
  • domain names
  • interfaces
  • local users
  • policies
  • services
  • the router
  • VPRNs
• MAC addresses
• IP addresses and prefixes
• autonomous system numbers
• log events

Scrambling

The application reads cleartext input line by line, replaces cleartext strings that match patterns in a YAML rule file with a HMAC-SHA256 scrambled string, and writes cleartext output.

Strings that match a pattern are scrambled with hash = HMAC-SHA256(key, string) to generate a 64-character hexadecimal hash. The resulting scrambled string consists of the scramble-prefix, followed by the first eight characters of the hash, followed by the optional scramble-suffix. For example, STR-8f96d576.

WARNING: Review the scrambled output carefully.

The same key produces the same scrambled string for the same input. This ensures that a cleartext string like an IP address has the same scrambled string and can be correlated in the same output or multiple outputs without revealing its real value.

Requirements

• Python 3.11 or newer, tested on Python 3.11 through 3.14
• PyYAML 6.0 or newer
• tqdm 4.70.0 or newer

The application can be downloaded from:

• GitHub
• PyPi

Installation

Nokia recommends using Python virtual environments, execute these commands to active the virtual environment:

python3 -m venv .venv
source .venv/bin/activate

Installation from GitHub

To install the application and its dependencies after cloning from GitHub, execute this command:

pip install .

Installation from PyPi

To install the application and its dependencies from PyPi, execute this command:

pip install id-scrambler

To upgrade to the latest release from PyPi, execute this command:

pip install --upgrade id-scrambler

Docker installation

Docker container images for x86 and ARM platforms are available on the GitHub Container Registry.

To install the 64-bit x86 image, execute this command:

sudo docker pull ghcr.io/nokia-steward/id-scrambler:1.0.0-amd64

To install the 64-bit ARM image, execute this command:

sudo docker pull ghcr.io/nokia-steward/id-scrambler:1.0.0-arm64

Usage

Usage: id_scrambler [OPTION]... [FILE]...
Scramble strings that match patterns in a cleartext file.

With no FILE, or when FILE is -, read standard input.

  -c, --config        the YAML configuration file, if not specified the file with
                      a ".yml" suffix in the current working directory is used
  -d, --debug         enable debug mode and send output to stderr
  -h, --help          display this help text and exit
  -k, --key           the key to be used in the hash, 32 to 128 bytes,
                      if not specified a random 32-byte key is generated
  -o, --output        the output file name, if not specified a ".scrm" suffix is
                      added to the input file name, if "stdout" then the output
                      is sent to stdout
  -v, --version       display version information and exit

Scrambling rules and options

Scrambling rules are defined in a YAML file. When -c/--config is omitted, the application uses the file with a .yml suffix in the current working directory.

Each top-level key is a rule name. Rule ordering is significant: the first rule that matches a string of text is evaluated, so ignore rules with no scramble-prefix are ordered first to match text that must remain cleartext. See the sr_os_rules.yml file for the SR OS scrambling rules that are included with the application.

Each scrambling rule can be enabled or disabled, depending on your requirements. All rules are enabled by default, ensuring that the maximum amount of information is scrambled. For example, the IP and ASN rules could be disabled to allow Nokia technical support to load the configuration file on a router when replicating a support issue, while still scrambling other identifiable information within quoted strings.

Caution: Disabling the quoted-string rule is not recommended, as it scrambles the largest amount of identifiable information.

description (no default)

This informational option is ignored by the application. It can be used for a free-form description or for other information about the rule. YAML # characters can also be used to annotate rules.

enabled (default true)

When enabled: true is set or ommitted, the rule is enabled. Rules can be disabled by setting the value to false.

ignore-match-prefix (no default)

When an ignore-match-prefix is defined and the match pattern begins with that prefix, it is ignored before hashing so only the remaining string is hashed. This makes the hash independent of the prefix, and the ignored prefix is displayed at the beginning of the scrambled string output. For example, matching peer-as 65150 with ignore-match-prefix: 'peer-as ' and scramble-prefix: 'ASN-' displays peer-as ASN-<hash of 65150>. The first time such a match occurs, it is retained as a dynamic runtime rule in memory that matches that exact value. For example, when a string 65150 is present in the input, it is scrambled like ASN-<hash of 65150> without the peer-as prefix. The dynamic rule uses the parent rule's scramble-prefix, scramble-suffix, and output-padding.

match-pattern (no default)

The pattern to match as defined by the pattern-format.

output-padding (default false)

When output-padding: true is set, the output is extended so that the scrambled string is the same length as the cleartext string. If the cleartext string is longer than the 64-character hash, the scrambled string is padded with lowercase alphanumeric characters (a-z0-9). The length of thescramble-prefix and scramble-suffix count towards the scrambled string length.

pattern-format (default yang)

Two pattern-format types are supported:

• yang: the XML Schema (XSD) regex flavor used by YANG pattern statements, including multifragment concatenation with +, so that patterns can be copied from a YANG model
• posix: Python regular expression, without anchoring

scramble-group (no default - whole match)

The scramble-group defines a capture group index so that only the string in that capture group is scrambled. The rest of the string in the match-pattern matched is displayed unchanged. Scramble groups are useful when the surrounding string is needed to locate the value unambiguously but must be preserved. For example, matching the string filter-id="4001" with match-pattern: '="([^"]*)"', scramble-group: 1, and scramble-prefix: 'STR-' displays filter-id="STR-<hash of 4001>", leaving the =" and " delimiters in the output.

scramble-prefix (no default)

The scramble-prefix defines the prefix of the scrambled string, for example STR-. The first 8-characters of the hash are appended to the prefix to generate the scrambled string. When output-padding is enabled, the output is padded so the scrambled string is the same length as the cleartext string.

Omit to ignore a match so that it is not scrambled and is displayed in cleartext.

scramble-suffix (no default)

The scramble-suffix defines the suffix of the scrambled string, for example, characters needed to reconstruct the cleartext format.

Scrambling examples

Example configuration files in model-driven and classic format, and other basic examples are included in the examples directory. As a quick example to get started, execute these commands to scramble an example file and display the differences:

id_scrambler examples/md-config.cfg
diff -u0 examples/md-config.cfg examples/md-config.cfg.scrm

Key requirements

A key with the -k/--key option may be entered with printable UTF-8 characters and must be 32 to 128 bytes long. If a key is not specified, a random 32-byte key is generated.

WARNING: Use a key with entropy when it is entered and not generated by the application.

Scambling a file

The most common use case is when the application defaults can be used to scramble a file. Execute this command:

id_scrambler myinputfile

The YAML rules in the present workding directory will be used with a generated key, and output will be written to ./myinputfile.scrm. Additional application options may be specified using the command line options.

Scrambling input interactively

Interactive scrambling is useful to quickly scramble a few lines that are copied and pasted, or to see how an input line is scrambled. To scramble input interactively, execute this command and then paste or enter the cleartext to be scrambled:

id_scrambler -o stdout

For example:

id_scrambler -o stdout
autonomous-system 64511        // pasted input
autonomous-system ASN-304becc1 // scrambled output
stdin : Scrambled 1 string     // total for the input, reported at end of input

Scrambling debug mode

Debug mode can be enabled to determine why input matches or does not match a scrambling rule. To enable debug mode, execute this command and then paste or enter the cleartext to be scrambled:

id_scrambler -o stdout -d

Example:

id_scrambler -o stdout -d
id_scrambler: loaded 30 rule(s) from ./sr_os_rules.yml
id_scrambler:   rule 'ignore-echo' [enabled, ignore]
id_scrambler:   rule 'ignore-command-names' [enabled, ignore]
id_scrambler:   rule 'ignore-commit-history-logs' [enabled, ignore]
id_scrambler:   rule 'ignore-numeric-entries' [enabled, ignore]
id_scrambler:   rule 'ignore-forwarding-classes' [enabled, ignore]
id_scrambler:   rule 'ignore-timos-banner' [enabled, ignore]
id_scrambler:   rule 'ignore-ipv4-well-knowns' [enabled, ignore]
id_scrambler:   rule 'ignore-ipv4-loopbacks' [enabled, ignore]
id_scrambler:   rule 'ignore-ipv6-well-knowns' [enabled, ignore]
id_scrambler:   rule 'ignore-common-keywords' [enabled, ignore]
id_scrambler:   rule 'ignore-log-messages' [disabled, ignore]
id_scrambler:   rule 'mac-address' [enabled, scramble]
id_scrambler:   rule 'ipv4-prefix' [enabled, scramble]
id_scrambler:   rule 'ipv4-address' [enabled, scramble]
id_scrambler:   rule 'ipv6-prefix' [enabled, scramble]
id_scrambler:   rule 'ipv6-address' [enabled, scramble]
id_scrambler:   rule 'autonomous-system' [enabled, scramble]
id_scrambler:   rule 'peer-as' [enabled, scramble]
id_scrambler:   rule 'local-as' [enabled, scramble]
id_scrambler:   rule 'as-colon' [enabled, scramble]
id_scrambler:   rule 'route-distinguisher' [enabled, scramble]
id_scrambler:   rule 'rd-entry' [enabled, scramble]
id_scrambler:   rule 'primary-config' [enabled, scramble]
id_scrambler:   rule 'secondary-config' [enabled, scramble]
id_scrambler:   rule 'tertiary-config' [enabled, scramble]
id_scrambler:   rule 'dns-domain' [enabled, scramble]
id_scrambler:   rule 'by-user-from' [enabled, scramble]
id_scrambler:   rule 'by-user-parentheses' [enabled, scramble]
id_scrambler:   rule 'configuration-comment' [enabled, scramble]
id_scrambler:   rule 'quoted-string' [enabled, scramble]
id_scrambler: no key supplied, generated a random 32-byte key
/configure router "Base" bgp group "IBGP"         // pasted input
CLEARTEXT : /configure router "Base" bgp group "IBGP"
RULE: ignore-common-keywords MATCHED: '"Base"' IGNORED (kept cleartext)
RULE: quoted-string MATCHED: '"IBGP"' REPLACED WITH: '"STR-2714a2f2"'
SCRAMBLED : /configure router "Base" bgp group "STR-2714a2f2"
/configure router "Base" bgp group "STR-2714a2f2" // scrambled output
stdin : Scrambled 1 string                        // total for the input

The input and output may also be a file.

Scrambling using a Docker image

To run the Docker image to scramble a file, execute this command:

sudo docker run --rm -v "$PWD:/work" -u "$(id -u):$(id -g)" ghcr.io/nokia-steward/id-scrambler:1.0.0-amd64 -c /usr/share/id_scrambler/sr_os_rules.yml myinputfile

To run the Docker image interactively, optionally with the -d option to enable debugging, execute this command:

sudo docker run --rm -i ghcr.io/nokia-steward/id-scrambler:1.0.0-amd64 -c /usr/share/id_scrambler/sr_os_rules.yml -o stdout

Input file formatting

Some SR OS output uses DOS format with CRLF line endings. The application removes all unprintable characters, such as ^M (carrige return) and ^G (bell), from the output. This removal will cause significant differences in the output when using tools like diff.

Nokia recommends converting the input file from DOS to Unix format before scrambling. This can be done using tools like dos2unix, or by removing the ^M and ^G characters with tools such as sed or vi.

Reporting issues

Nokia welcomes all issues, suggestions, and enhancements. Please submit these through the Nokia support process. While issues raised on GitHub may be considered for project inclusion by Nokia, direct pull requests are not accepted.

Known limitations

• When a scrambled string is longer than the cleartext string, output in show and tools commands is misaligned.
• Truncated output, for example an IP address 10.13* will not match a pattern and will not be scrambled.
• Some patterns in quoted strings may match a rule before the quoted-string rule at the end, and are not scrambled. For example: file-url "sftp://10.10.10.1/test" scrambles to file-url "sftp://IPv4-A-d69e5409/test" because it matches the ipv4-address rule.
• YANG/XSD \p{...} translation covers \p{L}, \p{N}, \p{Nd}. Inside a character class these fall back to ASCII ranges (Python cannot nest classes).
• XSD character-class subtraction and the \i/\c XML name escapes are not supported and raise a configuration error.
• YANG patterns are used as searches rather than whole-string matches. The translated expression is wrapped in ASCII alphanumeric/underscore token boundaries, not ^ and $ anchors.