Ensembl Python Template

Ensembl ValueSet API Project.

This repo contains code for the ValueSets API app. The ValueSets API expose gRPC end points for providing information about Ensembl Value Sets.

An Ensembl ValueSet is a collection of key/value pairs that are related to a given topic. For example, biotypes, APPRIS annotation terms, or transcript support level (TSL) definitions.

A ValueSet item should have the following fields:

• accession_id (str): value set unique identifier - e.g. "mane.select"
• label (str): short human friendly name - e.g. "MANE Select"
• value (str): short machine friendly name - e.g. "select"
• definition (str): short description - e.g. "A Transcript which is matched between Ensembl/GENCODE and RefSeq as part of the MANE project"
• description (str|None): optional and verbose definition - e.g. "The Matched Annotation from NCBI and EMBL-EBI is a collaboration between Ensembl/GENCODE and RefSeq. The MANE Select is a default transcript per human gene that is representative of biology, well-supported, expressed and highly-conserved. This transcript set matches GRCh38 and is 100% identical between RefSeq and Ensembl/GENCODE for 5' UTR, CDS, splicing and 3'UTR."

Requirements

• pyenv (with pyenv-virtualenv plugin)

• Python 3.10+

• grpcio

• grpcio-tools 1.30+

• openpyxl 3.*+ (optional)

• poetry

• Python packages

  • Test
    • pytest
    • pylint
    • mypy
  • Docs
    • Sphinx
    • mock
    • sphinx-rtd-theme
  • Dev
    • ipython
    • black

Getting Started

Clone this repo:

git clone --depth 1 -b main https://github.com/Ensembl/ensembl-valuesets-api.git

Install the python part (of the pipelines) and test it:

pip install ./ensembl-valuesets-api

# test
# do something!

Run the server, listening on default port 50051:

python scripts/run_server.py

Test the server with the provided basic client:

python scripts/client.py

Optional installation

If you need to install "editable" python package use '-e' option

pip install -e ./ensembl-valuesets-api

The following files (or folder names) will need to be modified:

• docs/conf.py: Change accordigly to project's name
• docs/install.rst: Change installation instructions accordingly
• README.md: Write a meaningful README for your project

Once done with the basic customisation, create the initial commit:

cd <NEW_PROJECT_NAME>
git add .
git commit -m 'Initial commit'

Installing the development environment (with Pyenv)

pyenv virtualenv 3.8 <VIRTUAL-ENVIRONMENT-NAME>
cd <NEW_PROJECT_NAME>
pyenv local <VIRTUAL-ENVIRONMENT-NAME>
pip install -r requirements-dev.txt
pip install -e .

Testing

Run test suite:

cd <NEW_PROJECT_NAME>
pytest

To run tests, calculate and display testing coverage stats:

cd <NEW_PROJECT_NAME>
coverage run -m pytest
coverage report -m

Generate documentation

cd <NEW_PROJECT_NAME>
./scripts/build_docs.sh

Open automatically generated documentation page at docs/_build/html/index.html

Automatic Formatting

cd <NEW_PROJECT_NAME>
black --check src tests

Use --diff to print a diff of what Black would change, without actually changing the files.

To actually reformat all files contained in src and test:

cd <NEW_PROJECT_NAME>
black src tests

Linting and type checking

cd <NEW_PROJECT_NAME>
pylint src tests
mypy src tests

Pylint will check the code for syntax, name errors and formatting style. Mypy will use type hints to statically type check the code.

It should be relatively easy (and definitely useful) to integrate both pylint and mypy in your IDE/Text editor.

Project setup with poetry

Clone this repo:

git clone --depth 1 -b main https://github.com/Ensembl/ensembl-valuesets-api.git

Navigate to repo directory and create virtual environment

poetry env use python3

Verify when virtual environment is activated or not

poetry env list

Install the dependencies

poetry install

Run the grpc and rest server from project directory

poetry run python3 -m src.grpc.ensembl.server
poetry run python3 -m uvicorn src.rest.server:app

Test the rest endpoints

curl --location --request GET 'http://localhost:8000/api/valuesets/accession_id/mane.select'
curl --location --request GET 'http://localhost:8000/api/valuesets/value/amino_acid_alphabet?is_current=true'
curl --location --request GET 'http://localhost:8000/api/valuesets?is_current=false'

Installing from pypi

Resources

Python Documentation

• Official Python Docs

Python distributions and virtual environments management

• Pyenv docs
• Pyenv virtualenv docs

Auto-generating documentation

• Spinx Docs

Linting, type checking and formatting

• Pylint
• Mypy
• Black
• Google Python Style Guide

Testing

• pytest
• Coverage

Development tools

• IPython

Distributing

• Packaging Python
• Apache 2.0 License
• Semantic Versioning