sslogread

Python reader module for the Speedy Structured C++17 logging library

See github repository for more details.

Installation

From pypi

python3 -m pip install sslogread

From sources

git clone https://github.com/dfeneyrou/sslog
cd sslog
mkdir build && cd build
cmake .. -DSSLOG_BUILD_PYTHON_READER=ON
make -j
python3 -m pip install python/dist/sslogread-*.whl

sslogread.load

This function (the unique one of the module) is the mandatory first step to work with logs.
It reads the meta information of the logs and creates the sslogread.LogSession objects with enables logs manipulation.

Declaration

def load(path)

The returned value is a LogSession object if the loading succeeded. In case of failure, an exception with textual explanation is thrown.

The LogSession objects contains only two methods that are described in the following chapters:

Example

import sslogread

try:
    session = sslogread.load("/path/to/my/log_folder")
except Exception as e:
    print("Loading failed:", e)

sslogread.LogSession.query

This is the main service of the LogSession object. Logs are first filtered then returned in a structured form.

Declaration

The parameters are a list of dictionaries, each of them representing a filtering rule.
A logic OR is performed between them

def LogSession.query({ level_min:"trace", level_max:"off", category:"", thread:"", format:"", must_have_buffer:False, arguments:[],
                     no_category:"", no_thread:"", no_format:"", must_not_have_buffer:False }, { ... }, ... )

A filter rule is a dictionary containing one or many of the following key-value pairs (logic AND between them):

[!IMPORTANT] Filtering on arguments is not pattern based. Names and values must be exact. The syntax is: <name><operator><value> with <operator> among =, ==, >, >=, <, <=
The <operator><value> can be ommitted to filter on the existence of a named argument.
Ex: "user==georges", "id=315", "weight<=50.0", "weight"

The returned value is a chronologic list of NamedTuples. The sslogread.Log structure is:

Examples

Display all the logs (no filtering) in text:

result = session.query()
for log in result:
   print("%d) %s" % (log['timestampUtcNs'], log['format']))

Keep only the logs with a binary buffer and on a category matching *engine*:

result = session.query( { 'buffer_size_min': 1, 'category':'*engine*' } )

Select the logs on a category matching *engine* but not on a thread matching worker*

result = session.query( { 'no_thread': "worker*", 'category':'*engine*' } )

Select the logs which possess an argument named id and another named user

result = session.query( { 'arguments': ["id", "user"] } )

[!WARNING] The argument name shall be exact, no pattern matching for argument's names and values

Select the logs which possess an argument named user which has a value github_agent

result = session.query( { 'arguments': ["user==github_agent"] } ) # Note: '=' or '==' are identical

Select the logs on category transaction and which possess an argument named id which has a value higher than 356 and lower or equal to 1000

result = session.query( { 'category': "transaction", 'arguments': ["id>356", "id<=1000"] } )

Select the logs with level >= error, with category name containing 'car', which contain both an argument named 'color' and one named 'wheels' with value >= 4

result = session.query( { 'level_min':'error', 'category':'*car*', 'arguments':['color', 'wheels>=4'] } )

Select the logs on category transaction and which possess an argument named id which has a value higher than 356 and lower or equal to 1000
OR the logs with level >= error, with category name containing 'car', which contain both an argument named 'color' and one named 'wheels' with value >= 4

result = session.query( { 'category': "transaction", 'arguments': ["id>356", "id<=1000"] },
                        { 'level_min':'error', 'category':'*car*', 'arguments':['color', 'wheels>=4'] } )

sslogread.LogSession.get_strings

This method returns a list of strings matching both a content pattern and a selection of string flags.
It is basically a query on strings.

Declaration

def LogSession.get_strings(pattern="", in_category=False, in_thread=False, in_format=False, in_arg_name=False, in_arg_value=False, in_arg_unit=False)

[!TIP] If the pattern is empty, then all strings are accepted for this criteria.
If no flag on the string type is set, then all strings are accepted for these criteria.

Example

// Get strings used as a category, matching the pattern "/scheduler/*/worker"
strings = session.get_strings(pattern="/scheduler/*/worker", in_category=True)

// Get all thread name strings
strings = session.get_strings(in_thread=True)

// Get all format strings or argument value strings, containing the word "User"
strings = session.get_strings(pattern="*User*", in_format=True, in_arg_value=True)

License

Released under the MIT license.