d2s 0.2.2

A Command Line Interface to orchestrate the integration of heterogenous data and the deployment of services consuming the integrated data. See https://d2s.semanticscience.org

A Command Line Interface to orchestrate the integration of heterogenous data sources under a common RDF Knowledge Graph using CWL workflows, and the deployment of user-facing services over the integrated data using Docker (SPARQL, BioThings APIs, GraphQL-LD, OpenAPI, Web UI).

Installation

Complete documentation about the d2s-cli on the d2s documentation website 📖

Install with pipx

pipx install d2s cwlref-runner

Use pip, pip3 or pipx depending on your system preferences.

Requirements:

• Python 3.6+ (built using python:3.6)
• docker-compose
• git, time (bash)

More detailed documentation to install pipx and d2s on various platforms available at d2s.semanticscience.org

See those instructions to install d2s on Windows using the Chocolatey package manager and pipx.

Some features support are still a work in progress on Windows, such as CWL workflows (see the official CWL Windows documentation to make it work)

Enable autocompletion

Enable commandline autocompletion in the terminal

Recommended, it makes d2s much more user-friendly

• ZSH: add the import autocomplete line to the ~/.zshrc file.

echo 'eval "$(_D2S_COMPLETE=source_zsh d2s)"' >> ~/.zshrc

Set your terminal to use ZSH by default:

chsh -s /bin/zsh

A oh-my-zsh theme can be easily chosen for a personalized experience. See the zsh-theme-biradate to easily install a simple theme and configure your terminal in a few minutes.

• Bash: add the import autocomplete line to the ~/.bashrc file.

echo 'eval "$(_D2S_COMPLETE=source d2s)"' >> ~/.bashrc

To be tested.

Install d2s for development

Install d2s as executable in local for development.

d2s will be updated directly on change in the code.

pip install --editable .

Try it

Display the default help command

d2s

Create a d2s project in the given folder

d2s init project-folder-name

All d2s commands are designed to be run from the project folder, move to it

cd project-folder-name/

Project settings stored if .d2sconfig file.

Upgrade d2s

With pipx:

pipx upgrade d2s

With pip:

pip install --upgrade d2s 

Uninstall

With pipx:

pipx uninstall d2s cwlref-runner

With pip:

pip uninstall d2s cwlref-runner

Add new package

Add new packages to use to setup.py (to be imported during the build).

And install it locally for dev:

pip install <package>

Build and publish

Publish using Docker

To publish a new version on pypi:

• upgrade the version in setup.py (e.g. from 0.2.1 to 0.2.2)
• use the following script to build and publish automatically using Docker:

./publish_pip.sh

A test will be run using Docker before publishing to make sure d2s init works.

Build locally

Building and publishing can be done locally:

# Build packages in dist/ folder
python3 setup.py sdist bdist_wheel
# Publish packages previously built in the dist/ folder
twine upload --repository-url https://upload.pypi.org/legacy/ dist/*

Additional instructions to install twine locally (not needed)

pipx install twine

If you experience issues with Bash or ZSH because d2s is not defined when installing for dev. Then add pip install --editable develop/d2s-cli to .zshrc

You might need to install Python3.6 for dev (dev with python3.6 should work though)

sudo apt-get install python3.6 python3.6-venv python3.6-dev
# Set python3 to use 3.6
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
sudo update-alternatives --config python3

vim /usr/bin/gnome-terminal

#!/usr/bin/python3.6

If you face issue uploading the package on pypi:

twine check dist/d2s-*-py3-none-any.whl