lscl – Logstash configuration language handling

lscl is a Python module for parsing and rendering Logstash configurations in its own language, named LSCL for “LogStash Configuration Language”.

The project is present at the following locations:

• Official website and documentation at lscl.touhey.pro;

• lscl repository on Gitlab;

• lscl project on PyPI.

As described in Reading Logstash configurations and Rendering Logstash configurations, you can use this module to create, parse, update and render Logstash pipelines. Here is an example where lscl is used to add an add_field operation on an existing pipeline:

from lscl.lang import LsclAttribute, LsclBlock
from lscl.parser import parse_lscl
from lscl.renderer import render_as_lscl


SOURCE = """
input {
    stdin { }
}
filter {
    dissect {
        mapping => {
            "message" => "[%{ts}] %{message}"
        }
    }
}
output {
    elasticsearch { codec => rubydebug }
}
"""

content = parse_lscl(SOURCE)

# Find the 'filter' block at top level.
# If the block is not found, create it.
for el in content:
    if isinstance(el, LsclBlock) and el.name == "filter":
        break
else:
    el = LsclBlock(name="filter")
    content.append(el)

# Add the add_field filter.
el.content.append(
    LsclBlock(
        name="mutate",
        content=[
            LsclAttribute(name="add_field", content={"mytag": "myvalue"}),
        ],
    )
)

print(render_as_lscl(content), end="")

The script will output the following:

input {
  stdin {}
}
filter {
  dissect {
    mapping => {
      message => "[%{ts}] %{message}"
    }
  }
  mutate {
    add_field => {
      mytag => myvalue
    }
  }
}
output {
  elasticsearch {
    codec => rubydebug
  }
}