Skip to main content

Amazon Redshift workload triage: captures SYS/SVV telemetry from producer and consumer clusters into local DuckDB, groups repeat query patterns, and ranks physical-design fixes by total time cost.

Project description

Infraredshift

Amazon Redshift workload triage. It captures SYS/SVV telemetry from your producer and consumer clusters into a local DuckDB file, groups queries that repeat with different literals into one pattern, and ranks physical-design fixes by the total time they actually cost you.

The point is triage. A cluster running fifteen thousand slow queries does not have fifteen thousand problems — it usually has a few dozen query shapes run over and over. Infraredshift finds the shapes, totals their real cost, and tells you which ones are worth fixing first.

Everything runs locally. Your SQL, your telemetry, and your credentials never leave the machine.


Install

pip install infraredshift

To capture live from Redshift, add the connector:

pip install "infraredshift[redshift]"

On Windows/ARM, where the native connector has no wheel, use the JDBC path:

pip install "infraredshift[jdbc]"

Python 3.10+.

No executables are installed. The wheel is pure Python (py3-none-any) and contains no .exe, .dll or compiled extension of any kind. Deliberately, it also installs no console-script shims — on Windows pip generates those as .exe files (the same mechanism behind pip.exe), drops them in a directory that is often not on PATH, and warns about it. On a locked-down laptop an unexplained .exe is a question you have to answer, so there are none.

Everything runs as a module:

python -m analyzer                  # the desktop app
python -m analyzer.loader           # the loader CLI
python -m analyzer.ingest_redshift  # direct capture

First run

python -m analyzer

The app opens on a local sign-in screen. This is not a Redshift login — it creates an access code and PIN that encrypt your Redshift credentials at rest on this machine, using Windows DPAPI scoped to your Windows user. Nobody else signing in to this machine can decrypt them, and they are never written to a .env file or exported into environment variables.

There is nothing to configure before this step. If you have no data yet, the app opens with empty screens and tells you what to load.

Getting data in

You need three things: a cluster profile, credentials, and a load.

1. Describe your clusters. Create redshift_cluster_profiles.json next to the app, or point at one with REDSHIFT_ANALYZER_PROFILE_PATH:

{
  "format": "redshift-query-anatomy-cluster-profiles",
  "version": 1,
  "profiles": [
    {
      "profile": "REDSHIFT_PRODUCER",
      "display_name": "Producer",
      "namespace_id": "your-producer-namespace-uuid",
      "port": "5439",
      "primary_database": "dev",
      "floor_seconds": "300"
    },
    {
      "profile": "REDSHIFT_CONSUMER_1",
      "display_name": "Reporting",
      "namespace_id": "your-consumer-namespace-uuid",
      "port": "5439",
      "primary_database": "dev",
      "floor_seconds": "30"
    }
  ]
}

floor_seconds is the capture threshold for that cluster — queries faster than this are not captured. A busy producer earns a high floor (300s); a quieter consumer can afford 30s. This file holds no credentials, so it is safe to share with teammates.

2. Enter credentials in the app under Local Credentials. They are encrypted immediately.

3. Load. Use the Data Loader tab, or:

python -m analyzer.loader refresh --days 7

The first load is the slow one. After that, loads are incremental.

No cluster handy? The app ships a mock warehouse, so you can open every screen and see real-shaped data before connecting anything.


What you get

Workload Triage — one bubble per repeat pattern. Horizontal axis is how often it runs, vertical is what you choose (runtime, rows, spill). Big bubble in the upper right is where your time is going. Each pattern carries a verdict: fix the query, fix the tables, or leave it alone.

Table Heat Map — every table as a square, coloured by distribution and sort health. Missing DISTKEY, unsorted, stale statistics. Spectrum tables get their own view coloured by whether a partition key exists.

Query Decomposer — takes one slow query and splits it into the stages it actually runs, so you can see which join or scan is the expensive one.

Behind Views — the base tables a query really touches once views are expanded, which is usually more than the SQL suggests.


Commands

Command Does
python -m analyzer Open the desktop app
python -m analyzer.loader refresh --days 7 Incremental capture
python -m analyzer.ingest_redshift --external-tables Spectrum/external metadata (opt-in)

Large Spectrum catalogs

SVV_EXTERNAL_COLUMNS returns one row per column, so a catalog with millions of external tables is tens of millions of rows. If yours is large, restrict the capture in the producer profile:

"external_schemas": "curated, analytics",
"external_table_patterns": "fact_*, dim_*"

The filter runs on Redshift, so the rows never cross the wire. * and ? are the wildcards; a literal _ in a table name means itself.


Privacy

Analysis is local. The app makes no outbound calls except to your own Redshift clusters. Captured SQL, telemetry, and credentials stay in your DuckDB file and your DPAPI-encrypted secrets file on this machine.


License

MIT. See LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

infraredshift-0.1.1.tar.gz (811.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

infraredshift-0.1.1-py3-none-any.whl (717.8 kB view details)

Uploaded Python 3

File details

Details for the file infraredshift-0.1.1.tar.gz.

File metadata

  • Download URL: infraredshift-0.1.1.tar.gz
  • Upload date:
  • Size: 811.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.13

File hashes

Hashes for infraredshift-0.1.1.tar.gz
Algorithm Hash digest
SHA256 80b1d605c8dd8a3871c84f83150bfe67ebd38cbf9a612dd8b8328cd7308bc35c
MD5 a3e395803c19b8119d8b26ba8739567b
BLAKE2b-256 b1bd0374854847ba144875ec9caae36dcd382f9963caf0648fad64ff56108aad

See more details on using hashes here.

File details

Details for the file infraredshift-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: infraredshift-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 717.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.13

File hashes

Hashes for infraredshift-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 44dbe1c972061d5c85dad82587c1b4c807d2b1b5af9abb1187b0bd96e26fdb10
MD5 07333da3026cc6bf8470fad393c76cb2
BLAKE2b-256 807b5d44aa8cd6ddea8107dd33d5b7aeee23c5feaca766820cae6782d42139d7

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page