Skip to main content

Template Text Parser

Project description

Downloads PyPI versions Documentation status

Template Text Parser

TTP is a Python library for semi-structured text parsing using templates.

Why?

To save ones time on transforming raw text into structured data and beyond.

How?

Regexes, regexes everywhere... but, dynamically formed out of TTP templates with added capabilities to simplify the process of getting desired outcome.

What?

In essence TTP can help to:

  • Prepare, sort and load text data for parsing
  • Parse text using regexes dynamically derived out of templates
  • Process matches on the fly using broad set of built-in or custom functions
  • Combine match results in a structure with arbitrary hierarchy
  • Transform results in desired format to ease consumption by humans or machines
  • Return results to various destinations for storage or further processing

Reference documentation for more information.

TTP Networktocode Slack channel

Collection of TTP Templates

Example - as simple as it can be

Simple interfaces configuration parsing example

Code
from ttp import ttp
import pprint

data = """
interface Loopback0
 description Router-id-loopback
 ip address 192.168.0.113/24
!
interface Vlan778
 description CPE_Acces_Vlan
 ip address 2002::fd37/124
 ip vrf CPE1
!
"""

template = """
interface {{ interface }}
 ip address {{ ip }}/{{ mask }}
 description {{ description }}
 ip vrf {{ vrf }}
"""

parser = ttp(data, template)
parser.parse()
pprint.pprint(parser.result(), width=100)

# prints:
# [[[{'description': 'Router-id-loopback',
#     'interface': 'Loopback0',
#     'ip': '192.168.0.113',
#     'mask': '24'},
#    {'description': 'CPE_Acces_Vlan',
#     'interface': 'Vlan778',
#     'ip': '2002::fd37',
#     'mask': '124',
#     'vrf': 'CPE1'}]]]

Example - a bit more complicated

For this example lets say we want to parse BGP peerings output, but combine state with configuration data, at the end we want to get pretty looking text table printed to screen.

Code
template="""
<doc>
This template first parses "show bgp vrf CUST-1 vpnv4 unicast summary" commands
output, forming results for "bgp_state" dictionary, where peer ip is a key.

Following that, "show run | section bgp" output parsed by group "bgp_cfg". That
group uses nested groups to form results structure, including absolute path
"/bgp_peers*" with path formatter to produce a list of peers under "bgp_peers"
path.

For each peer "hostname" and local bgp "local_asn" added using previous matches.
Additionally, group lookup function used to lookup peer state from "bgp_state"
group results, adding found data to peer results.

Finally, "bgp_peers" section of results passed via "tabulate_outputter" to
from and print this table to terminal:

hostname           local_asn    vrf_name    peer_ip    peer_asn    uptime    state    description    afi    rpl_in           rpl_out
-----------------  -----------  ----------  ---------  ----------  --------  -------  -------------  -----  ---------------  ---------------
ucs-core-switch-1  65100        CUST-1      192.0.2.1  65101       00:12:33  300      peer-1         ipv4   RPL-1-IMPORT-v4  RPL-1-EXPORT-V4
ucs-core-switch-1  65100        CUST-1      192.0.2.2  65102       03:55:01  idle     peer-2         ipv4   RPL-2-IMPORT-V6  RPL-2-EXPORT-V6

Run this script with "python filename.py"
</doc>

<vars>
hostname="gethostname"
chain_1 = [
    "set('vrf_name')",
    "lookup('peer_ip', group='bgp_state', update=True)"
]
</vars>

<group name="bgp_state.{{ peer }}" input="bgp_state">
{{ peer }}  4 65101      20      21       43    0    0 {{ uptime }} {{ state }}
</group>

<group name="bgp_cfg" input="bgp_config">
router bgp {{ asn | record(asn) }}
  <group name="vrfs.{{ vrf_name }}" record="vrf_name">
  vrf {{ vrf_name }}
    <group name="/bgp_peers*" chain="chain_1">
    neighbor {{ peer_ip }}
      {{ local_asn | set(asn) }}
      {{ hostname | set(hostname) }}
      remote-as {{ peer_asn }}
      description {{ description }}
      address-family {{ afi }} unicast
        route-map {{ rpl_in }} in
        route-map {{ rpl_out }} out
	</group>
  </group>
</group>

<output
name="tabulate_outputter"
format="tabulate"
path="bgp_peers"
returner="terminal"
headers="hostname, local_asn, vrf_name, peer_ip, peer_asn, uptime, state, description, afi, rpl_in, rpl_out"
/>
"""

data_bgp_state = """
ucs-core-switch-1#show bgp vrf CUST-1 vpnv4 unicast summary
Neighbor   V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.0.2.1  4 65101      32      54       42    0    0 00:12:33       300
192.0.2.2  4 65101      11      45       99    0    0 03:55:01       idle
"""

data_bgp_config = """
ucs-core-switch-1#show run | section bgp
router bgp 65100
  vrf CUST-1
    neighbor 192.0.2.1
      remote-as 65101
      description peer-1
      address-family ipv4 unicast
        route-map RPL-1-IMPORT-v4 in
        route-map RPL-1-EXPORT-V4 out
    neighbor 192.0.2.2
      remote-as 65102
      description peer-2
      address-family ipv4 unicast
        route-map RPL-2-IMPORT-V6 in
        route-map RPL-2-EXPORT-V6 out
"""

from ttp import ttp

parser = ttp()
parser.add_template(template)
parser.add_input(data=data_bgp_state, input_name="bgp_state")
parser.add_input(data=data_bgp_config, input_name="bgp_config")
parser.parse()

Contributions

Feel free to submit an issue, report a bug or ask a question, feature requests are welcomed. Or buy Author a coffee

Additional resources

List of additional resources:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ttp-0.9.5.tar.gz (69.1 kB view details)

Uploaded Source

Built Distribution

ttp-0.9.5-py2.py3-none-any.whl (85.8 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file ttp-0.9.5.tar.gz.

File metadata

  • Download URL: ttp-0.9.5.tar.gz
  • Upload date:
  • Size: 69.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.9.13 Windows/10

File hashes

Hashes for ttp-0.9.5.tar.gz
Algorithm Hash digest
SHA256 234414f4d3039d2d1cde09993f89f8db1b34d447f76c6a402555cefac2e59c4e
MD5 43b9fc3bf54caa4f7cfffc96b36dca24
BLAKE2b-256 6e4fb95e48d3b653e1b1494a3a5b15614db5a340826282c5389e38cb2248da39

See more details on using hashes here.

File details

Details for the file ttp-0.9.5-py2.py3-none-any.whl.

File metadata

  • Download URL: ttp-0.9.5-py2.py3-none-any.whl
  • Upload date:
  • Size: 85.8 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.9.13 Windows/10

File hashes

Hashes for ttp-0.9.5-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 2c9fcf560b3f696e9fdd3554dc8e4622cbb10cac1d4fca13a7cf608c4a7fd137
MD5 b0a3431d7dc88c94c633de453a71230a
BLAKE2b-256 0338d475e7bde8d192ca5d64ccc0988f3d58f36211bd68c32b6c5883332a8abf

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page