GitData

GitData gives you one set of commands for exploring files and databases. List what is inside a source with gitdata ls, then preview its data with gitdata scan.

Install

GitData supports Python 3.9 through 3.12.

python3 -m pip install gitdata-lib

Oracle support is optional:

python3 -m pip install 'gitdata-lib[oracle]'

Explore a data source

The same two commands work across CSV files, SQLite databases, MariaDB/MySQL databases, and Oracle databases:

gitdata ls <source>
gitdata scan <source>

Use -l for a detailed listing, -a to include system objects, and --limit to choose how many rows to preview.

CSV

List the columns in a CSV file:

gitdata ls examples/locations-headered.csv

Preview its rows:

gitdata scan --limit 10 examples/locations-headered.csv

SQLite

Start with the database file, then move into a table or column:

# List tables
gitdata ls examples/inspection.sqlite

# Show table details
gitdata ls -l examples/inspection.sqlite/customers

# Preview rows
gitdata scan --limit 5 examples/inspection.sqlite/customers

# Examine one column
gitdata scan --limit 5 examples/inspection.sqlite/customers.email

SQLite references follow this shape:

path/to/database.sqlite
path/to/database.sqlite/table
path/to/database.sqlite/table.column

MariaDB and MySQL

Keep passwords in environment variables instead of putting them in database references:

export GITDATA_DB_PASSWORD='your-password'

SERVER='mysql://user@db.example.com?password_env=GITDATA_DB_PASSWORD'
DATABASE='mysql://user@db.example.com/sales?password_env=GITDATA_DB_PASSWORD'
TABLE='mysql://user@db.example.com/sales.orders?password_env=GITDATA_DB_PASSWORD'

Explore the server, database, and table with the same commands:

# List databases
gitdata ls "$SERVER"

# List tables
gitdata ls "$DATABASE"

# Show columns and indexes
gitdata ls -l "$TABLE"

# Preview rows
gitdata scan --limit 10 "$TABLE"

MariaDB/MySQL references follow this shape:

mysql://user@host
mysql://user@host/database
mysql://user@host/database.table

If the username is omitted, GitData uses the current operating-system username. Use gitdata ls -a when you also want to see system databases or tables.

Try the included MariaDB example

The repository includes examples/mariadb.sql. Load it into a disposable MariaDB container:

docker run --name gitdata-example-mariadb \
  -e MARIADB_ROOT_PASSWORD=example \
  -p 3307:3306 \
  -d mariadb:10.7

until docker exec gitdata-example-mariadb \
  mariadb-admin ping -h 127.0.0.1 -uroot -pexample --silent
do
  sleep 1
done

docker exec -i gitdata-example-mariadb \
  mariadb -uroot -pexample < examples/mariadb.sql

export GITDATA_EXAMPLE_PASSWORD=example
gitdata ls 'mysql://root@127.0.0.1:3307/gitdata_example?password_env=GITDATA_EXAMPLE_PASSWORD'
gitdata scan --limit 5 'mysql://root@127.0.0.1:3307/gitdata_example.customers?password_env=GITDATA_EXAMPLE_PASSWORD'

Remove the example when finished:

docker rm -f gitdata-example-mariadb

Command reference

gitdata ls <source>          list databases, tables, or columns
gitdata ls -l <source>       show a detailed listing
gitdata ls -a <source>       include system objects
gitdata scan <source>        preview and profile data
gitdata scan --limit N ...   preview at most N rows

Run a command with --help for all options:

gitdata ls --help
gitdata scan --help

Oracle

Install the optional Oracle driver, then keep the password in an environment variable:

python3 -m pip install 'gitdata-lib[oracle]'
export ORACLE_PASSWORD='your-password'

DATABASE='oracle://app@db.example.com:1521/FREEPDB1?password_env=ORACLE_PASSWORD'
TABLE="${DATABASE}#orders"

Explore the service and its tables with the same commands:

# List tables owned by the connected user
gitdata ls "$DATABASE"

# Show table counts and sizes
gitdata ls -l "$DATABASE"

# Show column details
gitdata ls -l "$TABLE"

# Preview rows
gitdata scan --limit 10 "$TABLE"

Oracle references follow this shape:

oracle://user@host:port/service?password_env=ENVIRONMENT_VARIABLE
oracle://user@host:port/service?password_env=ENVIRONMENT_VARIABLE#table

Inline passwords are not accepted. Ordinary Oracle table and column names are shown in lowercase. Quoted identifiers are shown with their double quotes so names that differ only by Oracle quoting remain distinct; use the displayed spelling after # when selecting one (the quotes may be URL-encoded as %22).