Skip to main content

# Analyzer SQream Logs

The log analyzer has the ability to help us find the correct logs faster and in a more intuitive way.

  • Version: 0.0.3

Requirements

  • Python 3.9+

How to use the tool

run either:

./sq_log_analyzer.py –help python3 sq_log_analyzer.py –help

This will show the full syntax with all options and acronyms, as well as, help details if you are not sure of the use.

Please note, using –help (or -h) will override any other arguments, which can help when in doubt

Flags

1.Load logs

The tool needs to know which files to use, we can specify a directory or a specific file:

–file-path (-p), requires a specific file path specify the full path (e.g. /media/weka_delivery/ti_folder/sqream_20230830_results.csv) specify just file name, the tool will search in the current directory Usage: –file-path=”sqream_20230830_results.csv” or –file-path=”/media/weka_delivery/ti_folder/sqream_20230830_results.csv”

—dir-path (-P), require a specific directory path specify the full path (e.g. /media/weka_delivery/ti_folder/10082023) specify just file name, the tool will search in the current directory Usage: –file-path=”10082023” or –file-path=”/media/weka_delivery/ti_folder/10082023”

the tool will search in all the subdirectories for sqream_logs files

Use the file filters (–file-date-filter/-fdf or –file-month-filter/-fmf) to help the tool search only the relevant files

2.Filters

Use as many filters as relevant to your search:

–from-date/–to-date: date and time filters –message-filter: any message in the logs, including, queries, known phrases (e.g. query before parsing) –info-level: INFO, ERROR or DEBUG (most logs are info) –ip-address: the server ip address –worker-port: the worker port –statement-id: the statement id if you are looking for a specific statement –connection-id: the connection id –service-name: service name –message-type: the number of the message, like 200 for execution plan

3.Additional Parameters

The tool support querying the results using mysql syntax:

use –query=”specific query” or –query-file=”file_path.sql” for query in a file

if you use column names with space in the name (like “statement id”), you must use the query file as the name must come in double quotes.

The tool support saving results to file:

use –output-dir=”directory_name” to save the file as out.csv in the directory use –output-path=”file_path.csv” to save to a specific file

please note, using the same directory dir will override old files

Important

  • The tool currently supports only the SQreamDB logs (the one inside the cluster)

  • The tool will only read files with the pattern: sqream_yyyymmdd_*.log

  • When specifying a directory or a file, it will search in the current directory unless a full path is specified.

  • The name of the table if you add a query is df, always

  • When running a query a warning may appear (“UserWarning: the ‘timedelta’ type is not supported”), ignore it

  • Queries may take long to run, so make sure it runs on the smallest dataset possible

Metadata

Release files for pysqream-log-analyzer 0.0.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pysqream-log-analyzer 0.0.3
File Size Uploaded Provenance
pysqream_log_analyzer-0.0.3.tar.gz 3.6 kB Without provenance Details

Built distribution (wheel)

Table of built distributions (wheels) for pysqream-log-analyzer 0.0.3
File Interpreter ABI Platform
pysqream_log_analyzer-0.0.3-py3-none-any.whl Python 3 none any Details

Total release size: 7.3 kB

Release files / pysqream_log_analyzer-0.0.3.tar.gz

Download URL pysqream_log_analyzer-0.0.3.tar.gz
Size 3.6 kB
Tags Source
SHA-256 checksum
How to use checksums
540ebf10d1de4411320077a6805ee33edfdc69a80b3e001c5a3ae85e491be4e2
BLAKE2b-256 checksum
How to use checksums
e496ec5372484cbda37ae47c5b71e6392e5176bbcb7ad25fa53fcddda9d25078
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.9.18

Release files / pysqream_log_analyzer-0.0.3-py3-none-any.whl

Download URL pysqream_log_analyzer-0.0.3-py3-none-any.whl
Size 3.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
642c919c008c34a11ac1663f3d0fc339f37a1733570c2b84050fe280136709b9
BLAKE2b-256 checksum
How to use checksums
f100f27e9b175d8f7a8201c819d4b5c7dc7edb28130a179763f065c3ded5f72f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.9.18

Security for pysqream-log-analyzer 0.0.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page