Artsy Docker development toolkit
Hokusai is a Docker + Kubernetes CLI for application developers.
Hokusai "dockerizes" applications and manages their lifecycle throughout development, testing, and release cycles.
Hokusai currently only supports Kubernetes deployments on AWS, configured to pull from ECS container repositories (ECR), although other providers may be added in the future.
At Artsy, as we began working with Kubernetes, while impressed with its design, capabilities, and flexibility, we were in need of tooling we could deliver to agile development teams that addressed the day-to-day tasks of application development, delivery, introspection and maintenance, while providing a clean and uncomplicated interface.
Transitioning teams to the Docker / Kubernetes ecosystem can be intimidating, and comes with a steep learning curve. We set out to create a Heroku-like CLI that would shepherd the application developer into the ecosystems of Docker and Kubernetes, and while introducing new tooling and concepts, outlining a clear practice for dependency management, local development, testing and CI, image repository structure, deployment and orchestration.
# Only if you don't already have pyenv installed brew install pyenv pyenv install -s
Before installing pythons via pyenv, make sure you are using brew-installed libraries
readline and xcode-installed
zlib and these libraries are correctly linked. For example:
brew install openssl If you need to have firstname.lastname@example.org first in your PATH run: echo 'export PATH="/email@example.com/bin:$PATH"' >> ~/.bash_profile For compilers to find firstname.lastname@example.org you may need to set: export LDFLAGS="-Lemail@example.com/lib" export CPPFLAGS="-Ifirstname.lastname@example.org/include" For pkg-config to find email@example.com you may need to set: export PKG_CONFIG_PATH="/firstname.lastname@example.org/lib/pkgconfig"
Now, when installing pythons you should see the following output.
$ pyenv install 3.5.8 python-build: use openssl from homebrew python-build: use readline from homebrew Downloading Python-3.5.8.tar.xz... -> https://www.python.org/ftp/python/3.5.8/Python-3.5.8.tar.xz Installing Python-3.5.8... python-build: use readline from homebrew python-build: use zlib from xcode sdk Installed Python-3.5.8 to $HOME/.pyenv/versions/3.5.8
Note: If you want to create a PyInstaller distribution (by running
make build) you need to install Python with development dylibs. Use the environment variable
PYTHON_CONFIGURE_OPTS="--enable-framework" on Darwin and
PYTHON_CONFIGURE_OPTS="--enable-shared" on Linux when running
pip install virtualenv virtualenvwrapper mkvirtualenv hokusai workon hokusai
pip install poetry
Install dev dependencies:
poetry install --no-root
Update dev dependencies:
To install hokusai package dependencies from
setup.py in "editable" mode, run
pip install -e .
Other installation methods can be found in the project's repo
If you use homebrew on OSX, install Docker for Mac with:
brew tap caskroom/cask && brew cask install docker
If you installed Docker for Mac,
docker-compose is also installed. Otherwise install with:
pip install docker-compose.
If you're on OSX, the preferred installation method is via homebrew:
$ brew tap artsy/formulas $ brew install hokusai
If you've previously installed hokusai via an alternate installation method, you may need to force the
link step. If you installed hokusai via pip, you may also want to cleanup that installation:
$ pip uninstall hokusai $ brew link --overwrite hokusai
If you are on Linux:
curl -sSL https://raw.githubusercontent.com/artsy/hokusai/master/get-hokusai.sh | sudo bash
Alternate Installation Methods
Note: If installing via pip fails due to pip failing to upgrade your system Python packages, try running
pip install hokusai --ignore-installed.
Note: You may need to adjust the target destination to match a directory in your
curl --silent https://artsy-provisioning-public.s3.amazonaws.com/hokusai/hokusai-latest-$(uname -s)-$(uname -m) -o /usr/local/bin/hokusai && chmod +x /usr/local/bin/hokusai
- Configure your AWS credentials.
hokusai configure --kubectl-version <kubectl version> --s3-bucket <bucket name> --s3-key <file key>. You'll need to provide the kubectl version matching your Kubernetes deployments, as well as the S3 bucket name and key of your org's kubectl config file. System administrators: see Administering Hokusai for instructions on preparing AWS, Kubernetes, and publishing a kubectl config file. Artsy devs: see these artsy/README docs for the current way to install and configure hokusai.
To enable bash autocompletion:
eval "$(_HOKUSAI_COMPLETE=source hokusai)"
See Getting Started.md to start using Hokusai for your project.
A full command reference can be found in Command Reference.md.
To work on Hokusai itself, set up your local development environment like so:
- As above, install
To install the Hokusai package in "editable mode" from a checkout of this repository, you can run
pip install --editable . This works well in combination with Virtualenv/Virtualenvwrapper as you can install the project in editable mode within a virtualenv, and from a release in your default system environment.
Hokusai is currently tested on Pythons 2.7.16 and 3.5.8.
Install poetry (see above).
poetry install --no-root.
All tests can be run with
Only run unit tests:
Only run integration tests:
Tests for specific modules, TestClasses, or even methods can be run with
python -m unittest test.unit.test_module.TestClass.test_method
DEBUG=1 environment variable to print boto logging
Hokusai can be used to simplify the process of spinning up a "review app" instance of your project, based on a feature branch or pull request.
Full details are in the Review App reference.
Merges to master automatically distribute Pyinstaller versions for beta testing at https://artsy-provisioning-public.s3.amazonaws.com/hokusai/hokusai-beta-Darwin-x86_64 and https://artsy-provisioning-public.s3.amazonaws.com/hokusai/hokusai-beta-Linux-x86_64 respectively.
To create a new release, bump the version by editing the
./hokusai VERSION file, create an entry in CHANGELOG.md and open a PR from
master to the
release branch of this repo.
The project is named for the great Japanese artist Katsushika Hokusai (1760-1849).
This project is the work of engineers at Artsy, the world's leading and largest online art marketplace and platform for discovering art. One of our core Engineering Principles is being Open Source by Default which means we strive to share as many details of our work as possible.
You can learn more about this work from our blog and by following @ArtsyOpenSource or explore our public data by checking out our API. If you're interested in a career at Artsy, read through our job postings!
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size hokusai-0.5.13-py2-none-any.whl (52.0 kB)||File type Wheel||Python version py2||Upload date||Hashes View|
|Filename, size hokusai-0.5.13.tar.gz (30.0 kB)||File type Source||Python version None||Upload date||Hashes View|