d2s 0.3.0

A Command Line Interface to orchestrate the integration of heterogenous data and the deployment of services consuming the integrated data. See https://d2s.semanticscience.org

A Command Line Interface to help orchestrate the integration of heterogenous data sources under a common RDF Knowledge Graph using Python, RML mappings, Bash, and GitHub Actions workflows (YAML).

We also provide guidelines and help to deploy of user-facing services over the integrated data using Docker (SPARQL, BioThings APIs, GraphQL-LD, OpenAPI, Web UI).

Installation

Complete documentation about the d2s-cli on the d2s documentation website 📖

Requirements:

• Python 3.6+ (built using python:3.6)
• Git
• Optional: Java 11+ to use d2s sparql upload
• Optional: oc command line tool for deploying to the DSRI OpenShift cluster (for Maastricht University academics and students)

Install from pypi

pip install d2s

Use pip, pip3 or pipx depending on your system preferences.

You can also install it from the master branch, if you want the latest updates:

pip install git+https://github.com/MaastrichtU-IDS/d2s-cli.git@master

See those instructions to install d2s on Windows using the Chocolatey package manager and pipx.

Install d2s for development

Install d2s as executable in local for development.

d2s will be updated directly on change in the code.

pip install -e .

Try it

Display the default help command

d2s

Create a d2s project in the given folder

d2s init project-folder-name

All d2s commands are designed to be run from the project folder, move to it

cd project-folder-name/

Project settings stored if .d2sconfig file.

Update d2s installation

pip install --upgrade d2s 

Uninstall

pip uninstall d2s

Enable autocompletion

Enable commandline autocompletion in the terminal

Recommended, it makes d2s much more user-friendly

• ZSH: add the import autocomplete line to the ~/.zshrc file.

echo 'eval "$(_D2S_COMPLETE=source_zsh d2s)"' >> ~/.zshrc

Set your terminal to use ZSH by default:

chsh -s /bin/zsh

A oh-my-zsh theme can be easily chosen for a personalized experience. See the zsh-theme-biradate to easily install a simple theme and configure your terminal in a few minutes.

• Bash: add the import autocomplete line to the ~/.bashrc file.

echo 'eval "$(_D2S_COMPLETE=source d2s)"' >> ~/.bashrc

To be tested.

Build and publish

Publish using Docker

To publish a new version on pypi:

• upgrade the version in setup.py (e.g. from 0.2.1 to 0.2.2)
• use the following script to build and publish automatically using Docker:

./publish_pip.sh

A test will be run using Docker before publishing to make sure d2s init works.

Build locally

Building and publishing can be done locally:

# Build packages in dist/ folder
python3 setup.py sdist bdist_wheel
# Publish packages previously built in the dist/ folder
twine upload --repository-url https://upload.pypi.org/legacy/ dist/*

Additional instructions to install twine locally (not needed)

pip install twine

If you experience issues with Bash or ZSH because d2s is not defined when installing for dev. Then add pip install --editable develop/d2s-cli to .zshrc

You might need to install Python3.6 for dev (dev with python3.6 should work though)

sudo apt-get install python3.6 python3.6-venv python3.6-dev
# Set python3 to use 3.6
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
sudo update-alternatives --config python3

vim /usr/bin/gnome-terminal

#!/usr/bin/python3.6

If you face issue uploading the package on pypi:

twine check dist/d2s-*-py3-none-any.whl