pysigma-backend-loki 0.4.0

pySigma Loki backend

pySigma Loki Backend

This is the Loki backend for pySigma. It provides the package sigma.backends.loki with the LogQLBackend class.

It supports the following output formats:

• default: plain Loki LogQL queries
• ruler: creates Loki LogQL queries in the ruler (YAML) format for generating alerts

It includes the following pipeline transformations in sigma.pipelines.loki:

• SetCustomAttributeTransformation: adds a specified custom attribute to a rule, which can be used to introduce a stream selector or parser expression into the generated query
  • The LokiCustomAttributes enum contains the relevant custom attribute names used by the backend

Further, it contains the processing pipelines in sigma.pipelines.loki:

• loki_log_parser: converts field names to logfmt labels used by Grafana
• loki_promtail_sysmon_message: parse and adjust field names for Windows sysmon data produced by promtail
  • Note: most rules lack the sysmon service tag, and hence this pipeline should be used in combination with the generic sysmon pipeline

This backend is currently maintained by:

• Nick Moore
• Mostafa Moradian

Installation

To get started developing/testing pySigma-backend-loki, these steps may help you get started:

1. Install poetry
2. Clone this repository and open a terminal/shell in the top-level directory
3. Run poetry install to install the Python dependencies
4. Run poetry shell to activate the poetry environment
5. Check it all works by running poetry run pytest
6. (Optional) If you wish to validate the generated rules using sigma_backend_tester.py, install LogCLI

Work in progress

These features are currently either WIP or are planned to be implemented in the near future.

• Various processing pipelines for other applications and log sources
• Generating more accurate log stream selectors based on logsource
• Translate field names in Sigma signatures into relevant labels for Loki using pipelines

Won't implement (probably)

These features are not easily supported by the backend, and hence are unlikely to be implemented.

• More complex keyword/line filter searches than ANDs of ORs