analysis3054 0.4.9

Advanced time-series analytics and forecasting toolkit for commodity and power trading

Analysis3054

Analysis3054 is a data pipeline + dashboard toolkit for energy, commodities, and demand planning. The analysis code lives in this repo; dashboards + update scripts live in a separate dashboard workspace.

Paths & environment

export ANALYSIS3054_CODEX_HOME=~/Analysis3054-Codex
export ANALYSIS3054_DASHBOARD_HOME=~/analysis3054-dashboards

Workspace layout

$ANALYSIS3054_DASHBOARD_HOME/
  scripts/
  dashboard_repos/<dashboard>/
    data/
      build_dashboard.py
      update_data.py
      requirements.txt
      *.csv (inputs)
      *.html (outputs)

Automation scripts

Automated update scripts live in $ANALYSIS3054_DASHBOARD_HOME/scripts. Finder-friendly wrappers live alongside them in $ANALYSIS3054_DASHBOARD_HOME/ (run_update_*.command).

Install (repo)

python -m venv .venv
source .venv/bin/activate
pip install -e .

Per-dashboard dependencies live in each requirements.txt:

pip install -r $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/requirements.txt

All dashboards load Plotly JS from the Plotly 3.3.1 CDN (export HTML inlines Plotly for offline use).

Dashboards

1) Vehicle Miles & Fuel Usage (Kayross + DTN)

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/

Inputs (4 CSVs):

• kayross_us.csv columns: VALUE_DATE, VALUE_DIESEL, VALUE_GASOLINE, REGION
• kayross_eu.csv columns: VALUE_DATE, COUNTRY, VALUE_DIESEL_CONSUMPTION, VALUE_GASOLINE_CONSUMPTION
• dtn_refined_fuels.csv (PADD) columns: effectiveDateTime, region, grade, sumNetVolume, coverageFactor
• dtn_refined_fuels_rack.csv (rack/state) columns: effectiveDateTime, rackId, rackName, state, region, grade, sumNetVolume

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/build_dashboard.py \
  --na-csv $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/kayross_us.csv \
  --eu-csv $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/kayross_eu.csv \
  --padd-csv $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/dtn_refined_fuels.csv \
  --rack-csv $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/dtn_refined_fuels_rack.csv

Outputs:

• $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/kayross_dashboard.html
• $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/kayross_dashboard_export.html (inline Plotly)

Chronos2 forecast:

KAYROSS_ENABLE_CHRONOS2=1 python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/build_dashboard.py

DTN updater (optional):

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/kayross/data/update_data.py --update-dtn

DTN auth (any one):

• DTN_API_KEY (preferred)
• DTN_ACCESS_TOKEN
• or OAuth: DTN_CLIENT_ID + DTN_CLIENT_SECRET (+ optional DTN_AUDIENCE)

Pandas inputs:

• build_dashboard.py accepts DataFrames via build_payload(...). If you want in-memory DataFrames, copy that script and call build_payload/build_html directly.

Notes:

• DTN diesel grades: #1 Diesel, #2 Diesel (plus common variants).
• DTN gasoline grades: Premium, Regular.
• PADD volumes apply coverage factor then divide by 42000 (KBD). Rack/state data skips coverage factor and still divides by 42000.

2) CFTC Disaggregated COT Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/cftc_disagg/data/

Build (downloads data if cache missing):

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/cftc_disagg/data/build_dashboard.py

Outputs:

• $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/cftc_disagg/data/cftc_disagg_dashboard.html
• $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/cftc_disagg/data/cftc_disagg_dashboard_export.html
• $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/cftc_disagg/data/cftc_disagg_dashboard_data.json
• $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/cftc_disagg/data/cftc_disagg_filtered.csv

3) Port TEU Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/port_teu/data/

Inputs:

• port_of_la_container_stats_2021_present.csv
• port_long_beach_teu.csv
• port_ny_teu.csv
• port_houston_teu.csv
• port_savannah_teu.csv

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/port_teu/data/build_dashboard.py --data-dir $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/port_teu/data

Update (Gemini disabled; Houston/Savannah are manual):

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/port_teu/data/update_data.py \
  --houston-csv /path/to/port_houston.csv \
  --savannah-csv /path/to/port_savannah.csv

4) Rail Traffic Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/rail_traffic/data/

Inputs (manual):

• rail_traffic_north_american.csv
• rail_traffic_us.csv
• rail_traffic_canada.csv
• rail_traffic_mexico.csv

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/rail_traffic/data/build_dashboard.py --data-dir $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/rail_traffic/data

Update:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/rail_traffic/data/update_data.py \
  --north-american-csv /path/to/north_american.csv \
  --us-csv /path/to/us.csv \
  --canada-csv /path/to/canada.csv \
  --mexico-csv /path/to/mexico.csv

5) Macro Drivers Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/macro/data/

Inputs:

• FRED snapshots: fred_diesel_monthly.csv, fred_diesel_quarterly.csv
• Macro focus: fred_macro_focus_*
• EIA snapshots: eia_distillate_product_supplied.csv, eia_steo_monthly.csv, eia_steo_quarterly.csv
• Forecast snapshots: distillate_forecast*.csv

Update (FRED + optional EIA refresh):

FRED_API_KEY=... python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/macro/data/update_data.py

Skip EIA refresh:

SKIP_EIA_FETCH=true python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/macro/data/update_data.py

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/macro/data/build_dashboard.py

Forecast model override:

DISTILLATE_FORECAST_MODEL=chronos2 python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/macro/data/build_dashboard.py

6) Europe Energy Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/europe_energy/data/

Update (Spain + UK sources):

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/europe_energy/data/update_data.py

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/europe_energy/data/build_dashboard.py

7) DFO Generator Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/dfo_generators/data/

Inputs: EIA-860/923 raw tabs (place into this folder). The update helper rebuilds:

• dfo_generators_inventory.csv
• dfo_generators_costs.csv

Update + Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/dfo_generators/data/update_data.py

8) EIA Electricity Runner (EIA-860/923)

Folder: $ANALYSIS3054_CODEX_HOME/analysis3054/data/eia_electricity/

This runner refreshes the raw EIA-860/923 CSVs and can rebuild the DFO dashboard.

Run:

python $ANALYSIS3054_CODEX_HOME/analysis3054/data/eia_electricity/update_data.py

Options:

python $ANALYSIS3054_CODEX_HOME/analysis3054/data/eia_electricity/update_data.py --output-dir $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/dfo_generators/data
python $ANALYSIS3054_CODEX_HOME/analysis3054/data/eia_electricity/update_data.py --year-860 2024 --year-923 2024
python $ANALYSIS3054_CODEX_HOME/analysis3054/data/eia_electricity/update_data.py --skip-dfo

9) Refinery Product Mix Dashboard

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/refinery/data/

Inputs:

• la_refinery_latest.csv (optional updater)
• tx_refineries_2021_present.csv
• refinery_data.csv
• refinery_forecast.csv
• refinery_product_forecast.csv
• tx_refinery_name_map.csv

Update:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/refinery/data/update_data.py --refresh-la

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/refinery/data/build_dashboard.py

10) TX Refinery Receipts & Deliveries

Folder: $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/tx_refineries_transfers/data/

Inputs:

• tx_refineries_received_2021_present.csv
• tx_refineries_delivered_2021_present.csv
• tx_refineries_2021_present.csv
• tx_refinery_name_map.csv

Build:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/tx_refineries_transfers/data/build_dashboard.py

Update:

python $ANALYSIS3054_DASHBOARD_HOME/dashboard_repos/tx_refineries_transfers/data/update_data.py \
  --received-csv /path/to/tx_received.csv \
  --delivered-csv /path/to/tx_delivered.csv

Notes

• Each dashboard folder is designed to be copied as-is.
• Update scripts avoid Gemini-based parsing; provide manual CSVs where noted.
• If you want offline HTML, use the *_export.html outputs where available.
• Legacy analysis3054/data/* folders in the code repo are no longer used by automation; remove them only after confirming no one relies on module default paths or the EIA electricity runner.