Easily test logstash filter
Project description
logstash-filter-test
Logstash is a program that collects json documents from various inputs, transform them according to a configuration file (a filter), and sends them to various outputs. This script helps you make sure that your filter does what you expect by writing a test suite.
Installation
You can install using pip:
pip install logstash-filter-test
or just copy logstash_filter_run.py
and logstash_filter_test.py
. There are
no dependencies. Python 3.5+ and 2.7 are supported.
Example
Here is a simple test suite file:
testcases.js:
[
// Test a message with a value
[
{
"message": "<<2018-06-11 13:45:39,127+0300>> TAU = 6.2831853"
},
{
"date": "2018-06-11 13:45:39,127+0300",
"@timestamp": "2018-06-11T10:45:39.127Z",
"msg_text": "TAU = 6.2831853",
"metric": "TAU",
"value": 6.2831853,
}
],
]
filter.conf:
filter {
grok {
match => {
"message" => [
"^<<%{TIMESTAMP_ISO8601:date}>> %{GREEDYDATA:msg_text}$"
]
}
}
date {
match => ["date", "ISO8601"]
}
grok {
match => {
"msg_text" => [
"^%{WORD:metric} = %{NUMBER:value:float}"
]
}
tag_on_failure => []
}
}
Put them in a directory, run ./logstash_filter_test.py
and it will inform you
if the testcases passed, and if not, why.
testcases.js
can be a simple json file. It has the extension .js
because
javascript comments and trailing commas are allowed, thanks to
jstyleson. It is a list of
test cases. Each test case is a list of size 2. The first item is the input
json document. The second item is compared to the output json document. All
fields that are defined must be equal to the fields in the output document.
The output document may include other fields. To test that an output field
doesn't exist, use "field": null
.
Testing from Python
If you don't like the testcase file format, it's easy to test by yourself:
>>> from logstash_filter_run import logstash_filter_run
>>> logstash_filter_run([{"a": 3}], 'filter { mutate { copy => { "a" => "b" } } }')
[{'@timestamp': '2018-07-10T18:26:58.411Z',
'@version': '1',
'a': 3,
'b': 3,
'host': 'myhost'}]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file logstash-filter-test-0.1.1.tar.gz
.
File metadata
- Download URL: logstash-filter-test-0.1.1.tar.gz
- Upload date:
- Size: 4.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4720bdbf2b41935ffaea578e127df78088e190a1fad8ee3c00aaf03323531d31 |
|
MD5 | ff9d84c13de7809429184a7839b55e5f |
|
BLAKE2b-256 | 7092d9260199cf63ca6cfce8addf0215efb90b85669dc36ecbe4aaa2edfa7a77 |
File details
Details for the file logstash_filter_test-0.1.1-py3-none-any.whl
.
File metadata
- Download URL: logstash_filter_test-0.1.1-py3-none-any.whl
- Upload date:
- Size: 7.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ec91afffb395f92eb7e907e08592de6caff4042ed03750979327232be619ee5 |
|
MD5 | d760b7d8deef42533172c4411bdbd55d |
|
BLAKE2b-256 | ab687c87878dfc38040d77cc60d4ea3adefe94d8226c89781b5d9b3dfb56ced7 |