Skip to main content

Amazon Redshift workload triage: captures SYS/SVV telemetry from producer and consumer clusters into local DuckDB, groups repeat query patterns, and ranks physical-design fixes by total time cost.

Project description

Infraredshift

Amazon Redshift workload triage. It captures SYS/SVV telemetry from your producer and consumer clusters into a local DuckDB file, groups queries that repeat with different literals into one pattern, and ranks physical-design fixes by the total time they actually cost you.

The point is triage. A cluster running fifteen thousand slow queries does not have fifteen thousand problems — it usually has a few dozen query shapes run over and over. Infraredshift finds the shapes, totals their real cost, and tells you which ones are worth fixing first.

Everything runs locally. Your SQL, your telemetry, and your credentials never leave the machine.


Install

pip install infraredshift

To capture live from Redshift, add the connector:

pip install "infraredshift[redshift]"

On Windows/ARM, where the native connector has no wheel, use the JDBC path:

pip install "infraredshift[jdbc]"

Python 3.10+.


First run

infraredshift

The app opens on a local sign-in screen. This is not a Redshift login — it creates an access code and PIN that encrypt your Redshift credentials at rest on this machine, using Windows DPAPI scoped to your Windows user. Nobody else signing in to this machine can decrypt them, and they are never written to a .env file or exported into environment variables.

There is nothing to configure before this step. If you have no data yet, the app opens with empty screens and tells you what to load.

Getting data in

You need three things: a cluster profile, credentials, and a load.

1. Describe your clusters. Create redshift_cluster_profiles.json next to the app, or point at one with REDSHIFT_ANALYZER_PROFILE_PATH:

{
  "format": "redshift-query-anatomy-cluster-profiles",
  "version": 1,
  "profiles": [
    {
      "profile": "REDSHIFT_PRODUCER",
      "display_name": "Producer",
      "namespace_id": "your-producer-namespace-uuid",
      "port": "5439",
      "primary_database": "dev",
      "floor_seconds": "300"
    },
    {
      "profile": "REDSHIFT_CONSUMER_1",
      "display_name": "Reporting",
      "namespace_id": "your-consumer-namespace-uuid",
      "port": "5439",
      "primary_database": "dev",
      "floor_seconds": "30"
    }
  ]
}

floor_seconds is the capture threshold for that cluster — queries faster than this are not captured. A busy producer earns a high floor (300s); a quieter consumer can afford 30s. This file holds no credentials, so it is safe to share with teammates.

2. Enter credentials in the app under Local Credentials. They are encrypted immediately.

3. Load. Use the Data Loader tab, or:

infraredshift-loader refresh --days 7

The first load is the slow one. After that, loads are incremental.

No cluster handy? The app ships a mock warehouse, so you can open every screen and see real-shaped data before connecting anything.


What you get

Workload Triage — one bubble per repeat pattern. Horizontal axis is how often it runs, vertical is what you choose (runtime, rows, spill). Big bubble in the upper right is where your time is going. Each pattern carries a verdict: fix the query, fix the tables, or leave it alone.

Table Heat Map — every table as a square, coloured by distribution and sort health. Missing DISTKEY, unsorted, stale statistics. Spectrum tables get their own view coloured by whether a partition key exists.

Query Decomposer — takes one slow query and splits it into the stages it actually runs, so you can see which join or scan is the expensive one.

Behind Views — the base tables a query really touches once views are expanded, which is usually more than the SQL suggests.


Commands

Command Does
infraredshift Open the desktop app
infraredshift-loader refresh --days 7 Incremental capture
infraredshift-capture --external-tables Spectrum/external metadata (opt-in)

Large Spectrum catalogs

SVV_EXTERNAL_COLUMNS returns one row per column, so a catalog with millions of external tables is tens of millions of rows. If yours is large, restrict the capture in the producer profile:

"external_schemas": "curated, analytics",
"external_table_patterns": "fact_*, dim_*"

The filter runs on Redshift, so the rows never cross the wire. * and ? are the wildcards; a literal _ in a table name means itself.


Privacy

Analysis is local. The app makes no outbound calls except to your own Redshift clusters. Captured SQL, telemetry, and credentials stay in your DuckDB file and your DPAPI-encrypted secrets file on this machine.


License

MIT. See LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

infraredshift-0.1.0.tar.gz (810.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

infraredshift-0.1.0-py3-none-any.whl (717.8 kB view details)

Uploaded Python 3

File details

Details for the file infraredshift-0.1.0.tar.gz.

File metadata

  • Download URL: infraredshift-0.1.0.tar.gz
  • Upload date:
  • Size: 810.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.13

File hashes

Hashes for infraredshift-0.1.0.tar.gz
Algorithm Hash digest
SHA256 25288d8aca5beee643313de8069eed4a54de25741dfb9302329603be3c6ee638
MD5 c49475e27decff0062b901722e9938c7
BLAKE2b-256 6c8d47d12076bca7ba086c8cfca2101280ad9db6e560b5ec9c30b54cb76d961f

See more details on using hashes here.

File details

Details for the file infraredshift-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: infraredshift-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 717.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.13

File hashes

Hashes for infraredshift-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ea13d69f2e34459472caaccb331241ad7194fc9d7ac355a8fe14bfe576092a02
MD5 e3f63c0c0c7014d70c5c00fec733dd98
BLAKE2b-256 1c5c84406db9667950253f12ed24369933e409fdf34b6f3a8c11e3d583a27dbf

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page