Storage support for 4DN Data Portals.
DCIC Snovault is a FORK of snovault created at the ENCODE DCC project at Stanford. Our fork supports other projects of the 4D Nucleome Data Coordination and Integration Center (4DN-DCIC). Although this software is available as open source software, its primary function is to support our layered projects, and we are not at this time able to offer any active support for other uses. In particular, this fork does not purport to supersede the original snovault. we just have a different use case that we are actively exploring.
DCIC Snovault is a JSON-LD Database Framework that serves as the backend for the 4DN Data portal and CGAP. Check out our full documentation here.
This repository contains a core piece of functionality shared amongst several projects in the 4DN-DCIC. It is meant to be used internally by the DCIC team in support of Fourfront, the 4DN data portal, and at this point in time it is not expected to be useful in a standalone/plug-and-play way to others.
Installation in 4DN components
DCIC Snovault is pip installable as the dcicsnovault package with:
$ pip install dcicsnovault``
However, at the present time, the functionality it provides might only be useful in conjunction with other 4DN-DCIC components.
NOTE: If you’d like to enable Elasticsearch mapping with type=nested, set the environment variable “MAPPINGS_USE_NESTED” or set the registry setting “mappings.use_nested”.
Installation for Development
Currently these are for Mac OSX using homebrew. If using linux, install dependencies with a different package manager.
Step 0: Install Xcode
Install Xcode (from App Store) and homebrew: http://brew.sh
Step 1: Verify Homebrew Itself
Verify that homebrew is working properly:
$ brew doctor
Step 2: Install Homebrewed Dependencies
Install or update dependencies:
$ brew install libevent libmagic libxml2 libxslt openssl postgresql graphviz python3 $ brew install freetype libjpeg libtiff littlecms webp # Required by Pillow $ brew cask install adoptopenjdk8 $ brew install email@example.com
If installation of adtopopenjdk8 fails due to an ambiguity, it should work to do this instead:
$ brew cask install homebrew/cask-versions/adoptopenjdk8
If you try to invoke elasticsearch and it is not found, you may need to link the brew-installed elasticsearch:
$ brew link --force firstname.lastname@example.org
If you need to update dependencies:
$ brew update $ rm -rf encoded/eggs
If you need to upgrade brew-installed packages that don’t have pinned versions, you can use the following. However, take care because there is no command to directly undo this effect:
$ brew update $ brew upgrade $ rm -rf encoded/eggs
Step 3: Running Poetry
To locally install using versions of Python libraries that have worked before, use this:
$ poetry install
To update the version dependencies, use:
$ poetry update
This command also takes space-separated names of specific packages to update. For more information, do:
$ poetry help update
Managing poetry.lock after update
There may be situations where you do this with no intent to check in the resulting updates, but once you have checked that the updates are sound, you may wish to check the resulting poetry.lock file.
Normally, a successful build on a tagged branch will cause publication automatically. It should not be necessary for you to manually use:
$ poetry publish
Also, you would need appropriate credentials on PyPi for such publication to succeed. As presently configured, these credentials need to be in the environment variables PYPI_USER and PYPI_PASSWORD. If you attempt to do this manually, be sure the version is set properly!
To run specific tests locally:
$ bin/test -k test_name
To run with a debugger:
$ bin/test --pdb
Specific tests to run locally for schema changes:
$ bin/test -k test_load_workbook
Run the Pyramid tests with:
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size dcicsnovault-4.7.6-py3-none-any.whl (245.3 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size dcicsnovault-4.7.6.tar.gz (207.8 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for dcicsnovault-4.7.6-py3-none-any.whl