Indexima migration schema based on yoyo
Project description
yoyo-indexima
Versions following Semantic Versioning
Overview
Indexima migration schema based on yoyo and pyhive.
The little story
In the land of database migration tool, i have tried:
- flyway
- liquidbase with hive extention
Both either did not support hive (as flyway), or indexima did not fully compliant with hive (wich cause probleme with liquidbase)
So I try to found a module not too complex in order to migrate our indexima schema in a safe way.
In this early release, I just trying to do the job.
Setup
Requirements
- Python 3.7+
Installation
Install this library directly into an activated virtual environment:
$ pip install yoyo-indexima
or add it to your Poetry project:
$ poetry add yoyo-indexima
Usage
Hive connection
- backend ui must start with
indexima://
- If you have trouble to obtain an hive connection, please read http://dwgeek.com/guide-connecting-hiveserver2-using-python-pyhive.html/
Migration
You could see a complete sample under 'example' folder.
using python client
usage: yoyo_indexima [-h] [-s SOURCE] -u URI {show,apply}
example:
yoyo_indexima apply -s $(pwd)/example/migrations/ -u "indexima://admin:super_password@localhost:10000/default"
Commands:
- show: pring pending migration
- apply: apply ending migration
within python code
If your migrations script are under directory migration
folder
import os
from yoyo_indexima import get_backend, read_migrations
if __name__ == "__main__":
# obtain IndeximaBackend
backend = get_backend('indexima://admin:super_password@localhost:10000/default')
# Read migrations folder
migrations = read_migrations(os.path.join(os.getcwd(), 'migrations'))
print(f'migrations: {migrations}')
if migrations:
# apply migration
with backend.lock():
backend.apply_migrations(backend.to_apply(migrations))
License
Contributing
See Contributing
Next step
- production usage in order to see how this tool made the job
- more documentation in code
- purpose few change in 'yoyo' like set all SQL command on Backend class
- ...
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for yoyo_indexima-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9b7eed561195c092e1bb753153005f37283f74534127afb92eb12506303019df |
|
MD5 | 63a9ceb6d2140f8e5b6fb8d30d095602 |
|
BLAKE2b-256 | ce63abffec94195221749c104967d82853dc4dfa12ec85cf7509f5e66cf36edb |