geneEcomparison 0.0.95

Z-score comparison of gene-expression by species

gene-E-comparison

Project setup (using Visual Studio Code)

As always when using VS Code, make sure to add the setting described here to avoid an Execution_Policies-error when running Python scripts in general: https://stackoverflow.com/questions/56199111/visual-studio-code-cmd-error-cannot-be-loaded-because-running-scripts-is-disabl

For the AllenSDK to be able to run, Microsoft Build Tools for C++ is required. Please install them using this link: https://visualstudio.microsoft.com/de/visual-cpp-build-tools/

Make sure to include the optional MSVC package, then restart your computer.

Python 3.6.8 is required. The Python version has to be < 3.7 and > 3.6.4 due to the AllenSDK requiring the tables-library, which has only been precompiled for Python 3.6 (see: https://stackoverflow.com/questions/53643594/unable-to-install-library-due-to-error-with-hdf5). You can try to build your own whl-file using a newer Python-version. Otherwise, download and install Python 3.6.8 from here: https://www.python.org/downloads/release/python-368/

Create a new virtual environment using this command:

py -m venv .venv

Once you ensured these prerequisites, install the required packages using the provided requirements.txt:

pip install -r requirements.txt

Note that the AllenSDK unfortunately requires quite an old version of pandas. After installing the AllenSDK, you need to upgrade pandas to a newer version:

pip install pandas==1.1.5

If you are debugging with VS Code, you might want to use https://pypi.org/project/ptvsd/ for much better debugging-performance:

pip install ptvsd

As this project is provided as a package, you need to start it as a module instead of a top-level script (check out this blog-post: https://fabiomolinar.com/blog/2019/02/23/debugging-python-packages-vscode/). To do this, define "module": "geneEcomparison" in your launch.json:

"configurations": [
        {
            "name": "Python: geneEcomparison",
            "type": "python",
            "request": "launch",
            "module": "geneEcomparison",
            "console": "integratedTerminal",
            "cwd": "${workspaceFolder}",
            "justMyCode": false
        }
    ]

Else, the package won't be available and the relative imports (https://stackoverflow.com/questions/14132789/relative-imports-for-the-billionth-time, https://realpython.com/absolute-vs-relative-python-imports/) won't work. You need to run the project from the debugging-tab instead of the green arrow in the toolbar (which only runs the current script-file). The debugger will execute the package (https://riptutorial.com/python/example/18555/making-package-executable) or - to be more precise - the script defined in the package's __main__.py-file.

Testing

Just call tox from the command-line to run the tests:

tox

Packaging

Creating a Python-package is quite complicated and requires knowledge of some conventions. Most of these conventions have already been abided in this project for it to be deployed. Please read the python-documentation for some background-information on this topic: https://packaging.python.org/tutorials/packaging-projects/

To deploy the package to pypi.org, you need to build it first. Pypi does not support overwriting existing versions of a package, so make sure to increment the version-/revision-number in setup.cfg accordingly. Then, clean the dist/-folder in order to remove existing builds that would only take time for uploading just to be refused by pypi anyways. Build the package with this command:

py -m build

You need to create an account for pypi (you need separate accounts for each environment - testing: https://test.pypi.org/, production: https://pypi.org/) to upload the package. Upload the package using your account (you will be asked for username and password in the command-prompt):

py -m twine upload --repository testpypi dist/* --skip-existing

NOTE: The skip-existing-flag only makes sure that no error occurs due to any attempts of uploading already existing versions - in case you forgot to clean the dist/-folder (https://stackoverflow.com/questions/52016336/how-to-upload-new-versions-of-project-to-pypi-with-twine).

Testing the package

The package is now ready to be installed. To test its installation in a clean environment, I recommend using https://mybinder.org/. Mybinder allows you to use a variety of sources (e.g. a GitHub-repository, as done here) to set up a Python-environment and access it online. This Jupyter-notebook provides the correct environment for gene-e-comparison: https://mybinder.org/v2/git/https%3A%2F%2Fgithub.com%2Fchristoph-hue%2Fpy-dist-test/HEAD?filepath=test.ipynb

The main block is intended for testing the installation using pip (make sure to change the geneEcomparison-version-number according to your recently deployed package):

pip install -i https://test.pypi.org/simple/ geneEcomparison==0.0.19 --extra-index-url https://pypi.org/simple

Be aware that the extra-index-url-argument makes sure that all dependencies are requested from the production-environment of pypi, as its test-environment does not provide all necessary packages and should thus not be used. You can omit this parameter when using the production-environment.

Deploying in pypi.org's production-environment:

py -m twine upload --repository pypi dist/* --skip-existing

Usage

Running this package requires a permanent internet-connection.

IMPORTANT: Note that the AllenSDK-dependency unfortunately requires quite an old version of pandas. After installing the geneEcomparison-package, you need to upgrade pandas to a newer version:

pip install geneEcomparison
pip install pandas==1.1.5

Once the installation has finished, you can start the web-application using this code-block:

from geneEcomparison import App
# You can provide your own list of genes to be selectable in dropdowns. Then, you also need to define defaults for the dropdowns
# App.setAvailableGenes(["Gabra4", "Gabra5", "Gabrb1", "Gabrb2", "Gabrb3", "Gabrd", "Gabrg2"], "Gabra5", "Gabrb1", "Gabrb2")
App.start(port=5000) # default is 5000. provide a port other than that, in case it is already in use

This will start the flask-server, making the app available on the localhost, port 5000: http://127.0.0.1:5000/

Please note that any (or most of the) requested and processed data will be cached for performance-reasons.

Region-assignments

The mapping-file region-assignments.csv assigns diverging structures of different species to a common conceptual structure, which helps making the data comparable. Multiple structures can be mapped to a single concept. If you do not provide this file in your working directory, a mapping-file shipped with the package is used.

Example for region-assignments.csv:

Human,Mouse,Name
"level_3;pons","level_4;Pons","Pons"
"level_3;hypothalamus","level_4;Hypothalamus","Hypothalamus"
"level_3;thalamus","level_4;Thalamus","Thalamus"
"level_4;cerebellar cortex","level_3;Cerebellar cortex","Cerebellar cortex"

Shutdown

You can dispose the flask-server by pressing CTRL+C in the Python-console that hosts the web-application. There is currently no interface for shutting it down.