snowkill 0.4.5

A tool for realtime monitoring of running, queued and blocked queries in Snowflake

SnowKill

SnowKill is a near real-time query monitoring tool for Snowflake Data Cloud.

SnowKill helps to detect potential problems with queries which are currently running. It analyzes query stats and plans, detects bad patterns, generates notifications and optionally "kills" some queries automatically.

The core logic of SnowKill relies on internal REST API calls instead of SQL queries. It does not require an active warehouse to run, which makes it possible to maintain the constant monitoring almost free of charge.

SnowKill has programmatic access to query plan from "Query Profile" page in SnowSight. SnowKill also has access to information about locks and tries to report the exact reason for transaction collisions.

SnowKill operates on present data, which normally allows it to react much faster relative to conventional monitoring tools operating on past data from QUERY_HISTORY and GET_QUERY_OPERATOR_STATS.

How does it work?

1. Load list of queries which are currently RUNNING, QUEUED or BLOCKED.
2. Load additional information about query plans and active locks, if necessary.
3. Check queries against list of fully customizable conditions.
4. Optionally terminate matched queries exceeding specific thresholds.
5. Detect and skip previously reported queries, avoid duplicates.
6. Send notifications about newly matched queries (via Slack, Email, etc.).

Notification example

Quick links

• Getting started
• Implementation details
• Examples
• Deployment best practices

Built-in conditions:

• Blocked Duration
• Cartesian Join Explosion
• Estimated Scan Duration
• Execute Duration
• Join Explosion
• Storage Spilling
• Queued Duration
• Union without ALL

Built-in formatters:

• Markdown
• Slack

Built-in storages:

• Postgres table
• Snowflake table

Future plans

• More conditions, formatters, storages.
• Automated testing on push.
• Maybe some conditions for groups of queries, e.g. "more than 4 queries are queued on warehouse".

Issues? Questions? Feedback?

Please use GitHub "Issues" to report bugs and technical problems.

Please use GitHub "Discussions" to ask questions and provide feedback.

Created by

Vitaly Markov, 2023

Enjoy!