Your single pane of glass for real-time analytics into MySQL/MariaDB & ProxySQL
Project description
Dolphie
Your single pane of glass for real-time analytics into MySQL/MariaDB & ProxySQL
Installation
Requires Python 3.8.1+
Using PyPi
$ pip install dolphie
Using Poetry
$ curl -sSL https://install.python-poetry.org | python3 -
$ poetry install
Using Homebrew
If you are a Homebrew user, you can install dolphie via
$ brew install dolphie
Using Docker
$ docker pull ghcr.io/charles-001/dolphie:latest
$ docker run -dit --name dolphie ghcr.io/charles-001/dolphie:latest
$ docker exec -it dolphie dolphie -h host.docker.internal -u root --ask-pass
Usage
positional arguments:
uri Use a URI string for credentials (mysql/proxysql) - format: mysql://user:password@host:port (port is optional with default 3306, or 6032 for ProxySQL)
options:
--help show this help message and exit
--host-setup Start Dolphie by showing the Host Setup modal instead of automatically connecting
-C , --cred-profile Credential profile to use. See below for more information
-u , --user Username
-p , --password Password
-h , --host Hostname/IP address
-P , --port Port (socket has precedence)
-S , --socket Socket file
-c , --config-file Dolphie's config file to use. Options are read from these files in the given order: ['/etc/dolphie.cnf', '/etc/dolphie/dolphie.cnf', '~/.dolphie.cnf']
-m , --mycnf-file MySQL config file path to use. This should use [client] section [default: ~/.my.cnf]
-l , --login-path Specify login path to use with mysql_config_editor's file ~/.mylogin.cnf for encrypted login credentials [default: client]
-r , --refresh-interval
How much time to wait in seconds between each refresh [default: 1]
--host-cache-file Resolve IPs to hostnames when your DNS is unable to. Each IP/hostname pair should be on its own line using format ip=hostname [default: ~/dolphie_host_cache]
--host-setup-file Specify location of file that stores the available hosts to use in host setup modal [default: ~/dolphie_hosts]
--heartbeat-table (MySQL only) If your hosts use pt-heartbeat, specify table in format db.table to use the timestamp it has for replication lag instead of Seconds_Behind_Master from SHOW REPLICA STATUS
--ssl-mode Desired security state of the connection to the host. Supports: REQUIRED/VERIFY_CA/VERIFY_IDENTITY [default: OFF]
--ssl-ca Path to the file that contains a CA (certificate authority)
--ssl-cert Path to the file that contains a certificate
--ssl-key Path to the file that contains a private key for the certificate
--panels What panels to display on startup separated by a comma. Supports: dashboard,processlist,graphs,replication,metadata_locks,ddl,proxysql_hostgroup_summary,proxysql_mysql_query_rules,proxysql_command_stats [default: dashboard,processlist]
--graph-marker What marker to use for graphs (available options: https://tinyurl.com/dolphie-markers) [default: braille]
--pypi-repo What PyPi repository to use when checking for a new version default: [https://pypi.org/pypi/dolphie/json]
-H , --hostgroup This is used for creating tabs and connecting to them for hosts you specify in Dolphie's config file under a hostgroup section. As an example, you'll have a section called [cluster1] then below it you will list each host on a new line in the format key=host (keys have no meaning). Hosts support optional port (default is whatever port parameter is) in the format host:port. You can also name the tabs by suffixing ~tab_name to the host (i.e. 1=host~tab_name)
-R, --record Enables recording of Dolphie's data to a replay file. Note: This can use significant disk space. Monitor accordingly!
-D, --daemon Starts Dolphie in daemon mode. This will not show the TUI and is designed be put into the background with whatever solution you decide to use. Automatically enables --record. This mode is solely used for recording data to a replay file
--daemon-log-file Full path of the log file for daemon mode
--replay-file Specify the full path of the replay file to load and enable replay mode
--replay-dir Directory to store replay data files
--replay-retention-hours
Number of hours to keep replay data. Data will be purged every hour [default: 48]
--exclude-notify-vars
Dolphie will let you know when a global variable has been changed. If you have variables that change frequently and you don't want to see them, you can specify which ones with this option separated by a comma (i.e. --exclude-notify-vars=variable1,variable2)
--show-trxs-only (MySQL only) Start with only showing threads that have an active transaction
--additional-columns Start with additional columns in Processlist panel
--debug-options Display options that are set and what they're set by (command-line, dolphie config, etc) then exit. WARNING: This will show passwords and other sensitive information in plain text
-V, --version Display version and exit
Order of precedence for methods that pass options to Dolphie:
1. Command-line
2. Credential profile (set by --cred-profile)
3. Environment variables
4. Dolphie's config (set by --config-file)
5. ~/.mylogin.cnf (mysql_config_editor)
6. ~/.my.cnf (set by --mycnf-file)
Credential profiles can be defined in Dolphie's config file as a way to store credentials for easy access.
A profile can be created by adding a section in the config file with the format: [credential_profile_<name>]
When using a credential profile, do not include the prefix 'credential_profile' (i.e. -C production)
The following options are supported in credential profiles:
user
password
socket
ssl_mode REQUIRED/VERIFY_CA/VERIFY_IDENTITY
ssl_ca
ssl_cert
ssl_key
mycnf_file
login_path
MySQL my.cnf file supports these options under [client] section:
host
user
password
port
socket
ssl_mode REQUIRED/VERIFY_CA/VERIFY_IDENTITY
ssl_ca
ssl_cert
ssl_key
Login path file supports these options:
host
user
password
port
socket
Environment variables support these options:
DOLPHIE_USER
DOLPHIE_PASSWORD
DOLPHIE_HOST
DOLPHIE_PORT
DOLPHIE_SOCKET
DOLPHIE_SSL_MODE REQUIRED/VERIFY_CA/VERIFY_IDENTITY
DOLPHIE_SSL_CA
DOLPHIE_SSL_CERT
DOLPHIE_SSL_KEY
Dolphie's config supports these options under [dolphie] section:
(bool) host_setup
(str) credential_profile
(str) user
(str) password
(str) host
(int) port
(str) socket
(str) ssl_mode
(str) ssl_ca
(str) ssl_cert
(str) ssl_key
(str) mycnf_file
(str) login_path
(str) host_cache_file
(str) host_setup_file
(int) refresh_interval
(str) heartbeat_table
(str) startup_panels
(str) graph_marker
(str) pypi_repository
(str) hostgroup
(bool) show_trxs_only
(bool) show_additional_query_columns
(bool) record_for_replay
(bool) daemon_mode
(str) daemon_mode_log_file
(str) replay_file
(str) replay_dir
(int) replay_retention_hours
(str) exclude_notify_global_vars
Supported ProxySQL versions
- ProxySQL 2.6+ (could work on previous versions but not tested)
Note: Use admin
user instead of stats
user so you can use all features
Supported MariaDB versions
- MariaDB 5.5/10.0/11.0+
- RDS MariaDB
Supported MySQL versions
- MySQL/Percona Server 5.6/5.7/8.x/9.x
- RDS MySQL & Aurora/Azure
MySQL Grants required
Least privilege
- PROCESS (only if you switch to using processlist via
P
command) - SELECT to
performance_schema
+pt-heartbeat table
(if used) - REPLICATION CLIENT/REPLICATION SLAVE
Recommended
- PROCESS (only if you switch to using processlist via
P
command) - Global SELECT access (good for explaining queries, listing all databases, etc)
- REPLICATION CLIENT/REPLICATION SLAVE
- SUPER (required if you want to kill queries)
Record & Replay
Have you ever wished you could view the process list and various other related statistics from a specific moment in time? Perhaps during a database stall that led to an incident, and your monitoring tools failed to identify the root cause? Well, you're in luck! Dolphie has a Replay system that lets you do just that.
Starting with version 6.0.0, you can instruct Dolphie to record its data (via --record
) into a local SQLite database file that's compressed with ZSTD. When you're ready to replay this data, simply pass the --replay-file
option, and you can interact with it as if you were watching it live! Within the Replay interface, you can navigate with the controls: back, forward, play/pause, or seek to a specific datetime. While some features are disabled during replay, the essential functionalities remain intact. For a complete list of available commands, press ?
to access the help menu.
Note that this feature can incur a significant amount of disk space depending on how busy your server is and what you set --replay-retention-hours
+ --refresh-interval
. Adjust these values to suit your needs and monitor the disk space usage accordingly. You can also not mix Dolphie versions with a replay file. If you try to, it will rename the current replay file and create a new one to prevent any potential version conflicts.
Daemon Mode
Also introduced in version 6.0.0 is the ability to run Dolphie in daemon mode (via --daemon
option). It's designed to run in the background continuously recording data for future replay. It’s more resource-efficient than running Dolphie live, transforming it into a passive, always-on monitoring service. It eliminates Textual's TUI and creates a log file for messages while also displaying them in the console.
You have flexibility in how you run Dolphie in the background; personally, I prefer using systemctl
, but alternatives like nohup
or tmux
can be viable options.
Here's some examples of log messages you might see:
[INFO] Starting Dolphie v6.0.0 in daemon mode with a refresh interval of 1s
[INFO] Log file: /var/log/dolphie/dolphie.log
[INFO] Connected to MySQL with Process ID 324
[INFO] Replay SQLite file: /var/lib/dolphie/replays/localhost/daemon.db (24 hours retention)
[INFO] Connected to SQLite
[INFO] Replay database metadata - Host: localhost:3306, Version: 8.0.34 (Percona Server), Dolphie: 6.0.0
[INFO] ZSTD compression dictionary trained with 10 samples (size: 52.56KB)
[WARNING] Read-only mode changed: R/W -> RO
[INFO] Global variable innodb_io_capacity changed: 1000 -> 2000
Credential Profiles
Credential profiles can be defined in Dolphie's config file as a way to store credentials for easy access. A profile can be created by adding a section in the config file with the format: [credential_profile_<name>]
The following options are supported in credential profiles:
- user
- password
- socket
- ssl_mode REQUIRED/VERIFY_CA/VERIFY_IDENTITY
- ssl_ca
- ssl_cert
- ssl_key
- mycnf_file
- login_path
Example:
[credential_profile_dev]
user = dev_user
password = dev_password
[credential_profile_prod]
mycnf_file = /secure/path/to/prod.cnf
To use a credential profile, you can specify it with -C
/--cred-profile
option without using the prefix credential_profile
(i.e. -C prod
) when starting Dolphie. Hostgroups can also use credential profiles (see below)
Hostgroups
Hostgroups are a way to easily connect to multiple hosts at once. To set this up, you will create a section in Dolphie's config file with the name you want the hostgroup to be and list each host on a new line in the format key=<json>
(keys have no meaning). Hosts support optional port (default is whatever port
parameter is) in the format host:port
. Once ready, you will use the parameter hostgroup
or Host Setup
modal to see it in action!
Note: Colors can be used in the tab name by using the format [color]text[/color]
(i.e. [red]production[/red]
). You can also use emojis supported by Rich (can see them by running python -m rich.emoji
) by using the format :emoji:
(i.e. :ghost:
). Rich supports the normal emoji shortcodes.
Example:
[cluster1]
1={"host": "host1", "tab_title": "[yellow]host1[/yellow] :ghost:", "credential_profile": "dev"}
2={"host": "host2", "tab_title": "[blue]host2[/blue] :ghost:", "credential_profile": "dev"}
3={"host": "host3:3307", "tab_title": "[red]production[/red]", "credential_profile": "prod"}
4={"host": "host4"}
Feedback
I welcome all questions, bug reports, and requests. If you enjoy Dolphie, please let me know! I'd love to hear from you :dolphin:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.